This is a delicate situation, which you should handle as carefully as you can. In particular, I strongly advise against publishing this paper without first discussing it with seniors mentors who know all the details of the situation, and then having a (possibly awkward but necessary) discussion with the speaker if your mentors think publishing could be the right idea.
The problem is that there's a tricky balancing act:
If the speaker plans to publish within a reasonable time period, then trying to scoop him by publishing first would be somewhere between incredibly unfriendly and unethical (even if you give him full credit for the theorem statement itself and anything else you learned in the talk). It's not dishonest, but it's still a violation of the usual norms of professional behavior. It would be a major blow to your reputation, and you'd be much better off not trying to publish.
If the speaker doesn't plan to publish it, or at least not without unreasonable delay, then it's OK for you to publish a proof while explaining the situation and giving appropriate credit. But in the case of unreasonable delay there's an implied criticism of the speaker, which makes the situation extra contentious. (Don't do this unless you're confident that the community will side with you.)
Unfortunately, nobody can say with any certainty where to draw the line for reasonable delay. I think it's safe to say nobody would consider ten years reasonable, while many mathematicians would consider one year reasonable. (It's common to give talks on work that you plan to write up for publication only after completing a few more things.) A several year delay in writing something up is long but not unheard of. There are also personal factors that can affect what's reasonable: you'll look really bad if the speaker goes around telling people "I intended to write up my proof of this theorem as part of a longer paper, but I got delayed due to my cancer treatments. After 18 months someone who had been in the audience of my talk sent me his own write-up and threatened to publish it himself if I didn't quickly produce a paper of my own."
Maybe none of this matters. It could be that the speaker doesn't care or has no intention of publishing this result, and would be happy to give you his blessing to publish it yourself. It could be that he would be happy to write a joint paper (although you should be very reluctant to propose this, since it too involves awkward issues). The only way to find out what he has in mind is to talk.
the optimal solution is to contact the speaker, but he does not seem to reply
Even if the speaker is reluctant to correspond in general, it seems likely that he would discuss this issue. If you send a reasonable e-mail outlining your perspective and asking what his plans are, and you still hear nothing, then you can try to get in touch by other means. For example, maybe a mutual acquaintance could put you in touch (such as an organizer of the seminar/conference at which he spoke). Of course you need to handle this tactfully, but if you'd like to publish then you shouldn't give up on talking with the speaker if your first attempt fails, since there's too much at stake.