3

Now-a-days, literature in deep learning is increasing at a rapid rate.

One of the key difference with other domains is that almost all research papers in deep learning provide corresponding code in GitHub.

It is (relatively) easy for me to understand the (mathematical, analytical aspects of) research paper. But, I personally feel it difficult to understand existing codes.

in this context, I want to know the common practice among researchers regarding the existing codes. Do researchers in deep learning understand the existing codes in detail? Or they just use the code as a module to execute?

Is there any recommended approach towards complex existing codes?

2
  • 1
    Out of curiosity what is your background and level of training? Jan 7 at 8:51
  • @NAMcMahon Completed course on DL and at literature survey level.
    – hanugm
    Jan 8 at 6:09
4

Do researchers in deep learning understand the existing codes in detail? Or they just use the code as a module to execute?

A little bit of both. Some thoughts:

  • I only look at the code on a tiny fraction of the papers that I read. Most of the papers explain their "idea" well enough and have results on standardized tasks. So, there is no need to look at the code unless:

    1. I cannot understand exactly what the "idea" is (and I am sufficiently motivated to continue trying to figure it out), or
    2. I want to implement the idea myself.
  • In general, most of the codebase is the skeleton for ETL (extract, transform, and load), training, validation, tensorboard, etc. I ignore this. Instead, I look for the small part of the codebase where the new idea is implemented, and I study that in detail. Even if I want to run the code, I will generally add their new idea into one of my existing codebases, rather than running their entire codebase off-the-shelf.

  • Of course, new students will have different needs than more experienced people. After you've seen enough codebases, you can quickly distinguish between "the usual stuff" and "hmm, why did they do that?" When you are just starting out, everything will look unfamiliar, and you may need to go much more slowly to understand the different pieces. This is the same as any other discipline, really.

  • An added complication is that these papers come in a wide variety of languages. Recently, research papers have been converging on PyTorch, which I am very happy to see. But this is another difference: students generally get stuck on the software details of each language (which is reasonable, since their software engineering skills are less developed), while experts will only care about the underlying math.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.