I am a communications engineering undergraduate, and I was thinking of studying pure math courses will this help me later on in graduate school, and if it will help what courses of pure math will be related to my major?
I'm in a similar situation to you. I'm finishing my masters in digital communications and doing my thesis right now. Math is an issue for me definitely.
In my experience, a bit more complicated math is helpful if you're going to be developing systems that do statistical signal processing (which seems to be very popular now) and anything that has "smart" in its name.
What exactly do you need depends on what you want to study in detail. It might be a good idea to first dip your toes in the fields you want to be working with first and the pick math courses you want to take. You could try looking at some books or papers about say adaptive filters or modern receivers or estimation theory. This should give you an idea what's usually needed. If you're lucky enough to have approchable professors, do ask them about what will be useful to you.
I find Paul Garrett's recommended courses rather ambitious for a non-math major undergrad, if the context is the U.S.
You would be well served by covering most of the basic math classes needed for a physics or engineering major:
- the calculus, including multi-variate
- an intro to ordinary differential equations -- this is a classic course and will prepare you for the Fourier transform
- linear algebra -- will make a lot of things easier once you've had this; also, as long as your course is rigorous and not too cookbook, you will gain mathematical maturity by doing lots of proofs
- if you enjoy it, you could have some fun with a numerical methods class (although this is not a pure math course)
- intro to probability or graph theory (for something totally different)
- if you want more pure math, then take "advanced calculus" (but not without a rigorous linear algebra class under your belt)