No, the delay didn't spoil your chances, and no, don't write a followup.
The prof wasn't waiting for your reply, and didn't expect it to come the next day. If he is "well known", he may get dozens of applications every week, plus hundreds of other emails. He likely didn't notice the delay, but just processed the daily flood of messages.
Even if he did remember when your first mail came, he knows that compiling a good application with references takes time. You usually have to write a reference yourself and find a professor who knows you at least by sight to sign it. You could have been out of town for a job interview, ill, etc., there are a million very legitimate reasons why you didn't have it ready by the next day.
If he actually has an open position in which you might've fit, he has surely checked your references, and didn't like them. If not, there is little you, or he, could do.