If a university highly values teaching then such letters can be helpful and can be solicited ethically as long as there are certain considerations met.
A former student, with whom you no longer have any supervisory (or grading) relationship is fine. Such letters should be sent independently and their sender's identity and content not revealed to you. Both conditions should be met.
Alternatively, simply announcing that you are up for tenure and any student who wants to comment can send a letter to the committee is welcome to would be fine, as long as student anonymity is maintained.
A student who has graduated and with whom you will no longer have any possibility of a supervisory relationship is fine, even if not anonymous.
But everyone needs to be assured that there was (and can be) no pressure applied, nor any bribery possible.
And, yes, I think it is common enough in teaching colleges and some others. Probably more common is for a candidate to supply a list of names of students that the committee can contact independently.
However, if you contemplate doing this, I suggest that you discuss it with the department head and/or the dean before you act to assure that it is acceptable at your place as well as any additional restrictions. In particular, including letters that you have actually seen might not be allowed, or even proper.