But I don't understand if it is ethical because he was the one who identified the problem. I wouldn't have thought of the answer if he hadn't asked the question. I am an undergraduate and this is my first time in research. I do not understand the process, I think. But is this how research works? What is the proper way to proceed now?
There is no ethical issue. You should be proud of your accomplishment and publish the paper. It may be, in fact, that the professor saw the problem but not the solution and needed it himself.
In fact, it is fairly common for a dissertation advisor for the doctorate to provide the problems to his/her students that result in their degrees. That isn't universal, but happens regularly in some fields.
On the other hand, in some fields (mathematics, say) the ability to ask the right questions is extremely important - more so than finding the eventual answers in many cases. But to be a complete researcher you need to be able to do both. However, it isn't expected that you get this ability naturally. So, in a way, the professor is "kick starting" you on your way to being a good researcher.
Of course, you can also say in the paper that the problem was proposed by Professor X of University Y. The professor can give you advice about the advisability of that, of course.
Moreover, the question posed by the professor got you thinking. You got to thinking about the solution. But I also guess that it got you thinking about the next question that needs to be asked. If so, you are on your way.
I should also note, for completeness, that there have been occasions in which someone has posted important unsolved problems as "exercises" in books and elsewhere. This has sometimes been done without comment. The idea is to get the problem out to a wide variety of people who know the topic and might, somehow, have the insight to solve it. Don Knuth, for example, has done this in Computer Science:The Art of Computer Programming.
Nice work! There is certainly no ethical issue here, and not really much gray area either because this professor, who is the only other person there would be any measure of shared credit to consider, has advised you to send the work to a conference. Since they did not propose it as a shared submission, it seems like they most likely view you as the sole author of the work.
If you want to completely cover your bases, it would be appropriate to ask that professor for further advice just like you have asked us:
I am new to research and unsure of proper conventions. Since you suggested the problem, do you think I need to acknowledge your contribution in some way? If so, how?
Answering these sorts of questions is exactly what an academic advisor is for.
You may also consider asking them for advice on a proper conference to submit to, since this can be quite tricky for someone inexperienced.