I met a professor (a big name in our field) at a workshop and we worked on the initial idea for a paper as part of a group project. We stayed in touch for a couple of weeks but our conversation ceased as the other group members stopped working on the paper and the project ran out of steam. I was applying for postdoc fellowships at the time and one of the fellowships required four recommendation letters. I only knew three researchers in our field well enough that I could ask them for recommendation letters. Since I felt like I impressed the professor from the workshop while working on the paper with some good results, I sent him an email explaining my situation and asking whether he would be willing to write the fourth recommendation letter. I never heard back from him, so it's most probably a no. However, I now worry that his impression of me has been tarnished since I feel like I asked him for a letter after doing only a small amount of work. How do professors look upon these things? How badly have I messed my future prospects given that he is a big name in the field and sits on various funding committees? I don't think I will be able to see him face-to-face for a while.
Recommendation letters for students one knows well are relatively easy to write.
Recommendation letters for students one doesn't know well are very difficult to write and potentially take twice as long as the ones for students one knows well. Asking someone who doesn't know you to write one was a stretch, and though it would have been more polite of that person to kindly tell you "no", this was very likely a silent "no".
Think of it this way: recommendations are sort of a covenant we have with the students we teach, postdocs we supervise, or colleagues we work closely with. It is part of the reward that comes with doing well in your studies or research, you don't just get your marks or your degree, you get the recommendation too. If you haven't earned that with someone, it's just a time suck for them to try to come up with something to say about you based on the CV you give them.
I doubt you have messed anything up permanently; they may not even remember this happened. I don't think ill of anyone that I've opted not to write a recommendation for. I'd just be hesitant to ask anything more of that person until you've earned it via collaboration.