How good is an LoR based on a correspondence on one's paper?

Question

I am applying for Fall 2018 for Ph.D. Programs in Physics. I came across a paper in my late sophomore year which I found extremely interesting so I wrote a mail to the author expressing my thoughts about his paper which included a slightly new interpretation of one key constant appearing in his paper. He was very glad to read my mail and responded quite enthusiastically--further suggesting that I should send my note to a Journal. I didn't send it to any Journal because I thought it was too unimportant to be separately published.

Then, coincidentally, I met with the guy when he visited my University to deliver a guest-talk, in my early final year. Since I knew he was coming, I had been playing around with the calculations and arguments related to his paper and I found a newer way to derive what he had derived in his paper using fewer axioms. So, I met with the guy and told him about my results. This time also he was very enthusiastic and glad to meet me. He again strongly suggested communicating my results to a Journal. We then had many discussions over email regarding my new calculations and his suggestions. We recently found an even more simpler proof of the same results and I am in the process of submitting a paper on the same to a Journal. I have also been in contact with the guy regarding several versions of the draft for this paper.

Thus, I thought it would be nice if he writes me a reference letter indicating what he liked about my thinking and arguments, and how important he thinks are my results. Moreover, it would show that I like to explore papers out of my interest and work on them without any academic obligations. But when I wrote to him about it, he told me that he would write me a reference letter if I want him to but he would not be able to write anything quantitatively (to quote, "for example, you were in top 10% of my class or something like that") and thus, he doubts whether the LoR would be really helpful. I think that even a qualitative LoR will do good, thus, I am thinking of asking him to write an LoR anyway.

So, before I do that, I would like to know whether such an LoR would do any good or not? And worse, can it somehow backfire? Also, I would like the answer to consider both the scenario -- this being my fourth LoR and this being one of my three LoRs.

Answer 1

From your description, it sounds like this individual has effectively functioned as your research mentor for this project. A letter he writes for you could certainly be quite valuable as part of your application. Graduate schools are looking, more than anything else, for the potential to do cutting-edge research as a Ph.D. student and after.

So I would recommend definitely using his letter. However, keep in mind that this letter will not be able to cover much beyond the quality of your research output. Your contacts with him have been much more limited than would be typical if you were working with an on-site research supervisor. Most letters of recommendation submitted through online application management systems require the recommender to give quantitative evaluations of students' skills, relative to some other population of undergraduates. This professor is saying that he is going to have to select the "don't know" option in multiple categories when he submits his recommendation.

That is not a big deal, but it's not nothing either. Moreover, this professor is not going to have any way to comment on your classroom performance. So if you do get a letter about him that points to the high quality of your research, you should make sure that you have at least one other strongly complementary letter that addresses the matters which the first letter cannot.