I've had the same feeling in the past and I still have it to some extent. However, for me it is within areas of engineering. Personally, I switched fields within my major and started over again.
I was a Mechanical Engineering PhD student studying experimental fluid mechanics and at some point I realized I'm more interested in instrumentation, sensors, signal processing, etc. I also realized that I enjoy coding. I eventually got a master's degree and left the program. My plan was to work for a year or two and take my time to make up my mind about whether or not I want to go back to graduate school.
Getting a job was so difficult that it really did make me envy computer science graduates (since there were many more opportunities for them and as an undergrad I was very close to choosing CS). Once I got a job, I really didn't like the work that much anyway so I was really encouraged to go back to grad school. My biggest problem was that I was afraid of changing major since I felt like I didn't have the necessary background to study EE or CS. I also didn't think I would be admitted to any CS or EE program and I didn't have the extra cash to apply to too many grad programs. I ended up applying to ME programs again and once I joined a graduate program I tried to join research groups which were multidisciplinary. I eventually ended up in a group which does robotics work which combines all the areas I'm interested in. Personally, I'm extremely glad I made the change. I like the field much better and my research is much more interesting; however, I am also always aware that I'm a bit older than my peers and I'm also aware that I left a well paying job and a decent career for this and I'm silently afraid that I'll end up having trouble finding a job I like again (because I feel CS and EE majors get most the robotics jobs) and I'll end up settling for something similar to what I had in the first place....And that leads back to the envy thing you were talking about...I still envy CS and EE majors. Although not nearly as much as used to.