Does it matter if a faculty member teaches a small class or large class? Does anyone cares about the size of the class or only the number of classes the faculty teach each semester?
That depends a lot on local policy. Here (UTFSM, Chile), for load assesment, if a class is small (less than 10 finish it) it doesn't count (unless special circumstances apply, like the last few students taking a discontinued class), if it is very large it counts a bit more (or you might get more TAs or be relieved from other duties). For teaching evaluation, each class counts the same (the class averages are just averaged).
At my university, the typical number that's looked at is the credit hour production. Basically, it is calculated for each class as [number of students] times [credit hours awarded for that class]. Since my department doesn't have TAs or other forms of co-taught classes, I'm not sure how this number is divvied up for super-large lecture classes.
Often, to "enable" smaller senior seminar classes, we'll occasionally have the instructors for such classes also teach during the same year a larger lower-level class to balance things out. But normally for us, at least, these numbers are looked at at a departmental level rather than an individual level.
For evaluations, the only major issue is that if there are less than X students that respond, no one can see their evaluation for (a) fear of identifying the students and (b) terrible sample size = terrible statistics. IIRC our system will still aggregate them into a semester pool and an all-time pool for the numerical scores, but the comments are gone for good. Certain aspects of the course size will of course be taken into consideration. Class of 8 and you took three weeks to get their test back and students dinged your for it? Not good. Class of 120 and they complain about not having their 10 page paper returned by the next class session? "meh". But got the 8-student class's test back the next day? "meh". The 120's papers? Pretty great. Same idea applies for other aspects of the class (office hours, speed of response to email, willingness/ability to provide individualized attention, etc).