I am first going to point out how the actual application of any answer to this question is very dependent on the culture that you are in, and afterwards, I am going to suggest a few abstract ways how to do this.
As I said, answers to this question will be extremely culture-dependent. Starting with your example of giving an apple to an elementary school teacher, this could be seen as bordering on corruption in some places (especially in elementary school, where it can be surmised that the child did not buy the apple themselves, but was given the apple by their parents and asked to hand it over to their teacher). (Aside from the more practical question, unless you expect to be treated in a special way compared to your classmates, which would leave an extremely sour taste concerning your little apple gift, what the teacher is supposed to do with some 20 (or whatever the size of your class) apples.)
Signs of appreciation, and the degree at which appreciation is expressed, vary a lot between cultures. I grew up in Germany, and seen with a German cultural background, most praise expressed in English-speaking cultures sounds way over the top. Conversely, Germans are often perceived as cold and uncaring by people from English-speaking cultures, as a simple "Thank you for the interesting class." would already be quite a strong praise in German, but is not necessarily perceived as such by native English speakers. (Some of these differences are outlined quite well in an essay that was translated to English by Thorsten S in his answer on Travel SE.) There is even a region in Germany that is especially well-known for that trait, and one of their sayings goes "Absence of ranting is sufficient praise." While it's not actually quite that bad and that somewhat humorous saying should be taken with a grain of salt, it demonstrates well different cultural attitudes toward expressing praise.
The same applies to more material ways of expressing praise. Depending on the culture and circumstances, the way in which a gift (for saying thank you or otherwise) is presented might matter more than the gift itself. To cite just two examples:
- In some cultures such as in China, a gift might be simple, but needs to be well-chosen. As pointed out by the graphic linked in a Travel SE question, an apple might be an ok gift, but by no means switch it for a pear.
- Wedding gifts in Germany typically involve some money, but the most important aspect (the one that will actually be appreciated and memorized) is that a lot of fantasy and crafting went into them. cf. this, this, or this.
After establishing that the actual contents and wording is very culture-dependent, there might still be some generic suggestions for how to place the statement of appreciation:
- At the end of the last lesson, when you see the professor for the last time: If you hope to collaborate with the professor in the future, just talk to the professor and express that you'd be interested taking more classes/participating in projects in the research area of the professor or their department. You can then easily start such a conversation by pointing out that one of your main motivations for that goal is that you found the current class very interesting (or, if the topics do not match a lot, at least that it was very well organized and/or well-taught).
- Once you have something specific in mind, such as a project, point out that you found the class very interesting then. You can postpone your praise till that time; being on the professor's "gave praise, but didn't have anything concrete" batch will probably not help a lot, anyway.