At this point, you should probably give up on your hope that the professor will send the letter. If he hasn’t done it three weeks after the deadline despite repeated reminders, then I think a revision of your assessment that “he is very willing to help” is in order. Without trying to psychoanalyze him, all that one can say is that for all intents and purposes he is behaving like someone who is not at all willing to help, so that should be your working assumption from now on. Sadly. :-(
My recommendation is to look ASAP for someone else who can write you a decent letter of recommendation. At the same time, it may be a good idea to contact the department, explain the situation (including enough specific detail about the professor’s strange behavior to make it likely that they will be sympathetic to your situation and willing to be forgiving about this), apologize profusely, and say you are working on getting a letter from someone else as soon as possible, and that you hope that they will still be able to consider your application.
Good luck.