I don't really like other people seeing my source file. Are there good reasons to upload the source file (.tex) on arXiv over uploading just the PDF?
Arxiv mandates uploading the source of any PDF file generated from Latex code. You will not be able to upload only the PDF version in this case (unless you succeed to game the system).
Why do the require the source? This saves quite a lot of space (50k-100k source vs. 500-900k PDF) and there are some additional reasons. As the other comments mention - the full explanations are in http://arxiv.org/help/faq/whytex
At the start, I'd like to direct attention to @Noah Snyder's comment on the OP: http://arxiv.org/help/faq/whytex
If for no other reason, I'd be happy to see ArXiV and others steer away from the ubiquitous PDF format because it's proprietary (see various places in the Wikipedia article).
One GPL alternative I've had experience with is the ultra-lightweight DjVu format.