In the final exam of a course, a student has solved some problems in a really ingenious way. He gets perfect score anyway. What is the appropriate way to inform him about my appreciation of his solutions? (Also what if I don't have the occasion to meet him again?)
Just write the student an email and express your appreciation. Don't overthink this. Then again, don't go over the top ("you are the best student I ever had") - you never know how he will take this.
Depending on how well you know the student and on whether you would actually be willing to do so, inform him that you would be happy to write a glowing recommendation for him if he ever needs one, e.g., for admission to grad school.
And as Compass comments, think about how you can help the student develop even further. Do you have any follow-up courses, maybe even advanced ones that the student would usually not take but that you think he would be able to follow? Tell him. Can you give him a research project? Take him on as an Research or Teaching Assistant?
At some point, this turns from "helping the student grow" to "nurturing your own future grad student", and that's OK. This is a mutually beneficial thing, after all. And it's never too early to snap up the promising students. If you don't do it, others will.
Well, as a student, I will be glad if my work gets appreciated. Recognition like these act as confidence booster. But, don't overdo it, as it may make student think too high of himself.
Assuming that the final exam sheets are returned to the students, you can attach a (very) short note appreciating his ingenuity. Trust me, I get excited even on a simple "Good Job!" on my exam sheet (my professor does it rarely though). Other option could be to write him a short email. You may invite him to have a discussion on what further he can do with the ideas he demonstrated in the answer sheet. Depending on how high you think of his subject knowledge, you can offer him a TA job, as pointed by Stephen Kolassa.
But, most importantly, you should make sure that the student does something big with his talent, by referring him a book, an online course, or any other course in your or other department.
I know a professor who writes a special note to top 5 students in his class. Something along the line of :
"congratulations on a great job in my class ABD255 ... specially I appreciate the way you handled assignment 5.".
He also offers any recommendation letters that the student might need in the future and an open invitation to do master thesis with him. These are some good ways of showing your appreciation as they will likely count towards the student's career.