Short answer: yes, absolutely!
I'm in chemistry, but I have participated in a few physics searches. It's customary to include a letter for many of the reasons given in other answers. We want to know that you've taken the time to consider our department specifically and given some thought to how you'd fit, collaborators, etc. Otherwise with a generic cover letter, there's a nagging feeling that the candidate is either:
- Blindly sending out masses of applications.
- Applying to you, but your department is very low on their priority list.
Neither is a good first impression.
More importantly, I think is that it would look very strange to not have a cover letter.
I have been on roughly a dozen searches, each with at least 80-100 candidates each. I cannot recall someone who didn't include at least a short cover letter.
It really only needs to be 2-3 paragraphs, with maybe a few sentences on the specific department.
I can sympathize that it's yet another hoop to jump through in the application process. But I strongly recommend including one, if only because you don't want to leave an odd impression when we get to your file.