The kind of math you'd get exposed to with your proposed BSc would be very much of the applied variety (calc, differential equations, that sort of thing) but it's not going to drive you to any of the backing theory (abstract algebra, real/complex analysis, topology, etc.). It's unlikely a Master's in Aerospace Engineering is going to do you any favors in this regard either. Hence, by the time you want to do your PhD in Math, you will be woefully underprepared for it.
Hence, if you really want a PhD in Math, it's a very, very good idea to do a BA/BS in Math, so I would highly, highly recommend that you dual-major. The good news is that there will be some overlap in classes when you do this, so it's by no means an insurmountable goal. Plus, this will give you a strong sense as to whether to pursue the PhD in Math.
Another option is that, if you just do a BSc in Aerospace Engineering, when you go for your Master's you might consider switching to Math at that time. From a practical standpoint, I'm not sure a Master's in Aerospace Engineering would open many more doors than the BSc would, but a Master's in Math would definitely broaden your knowledge and give you more general appeal. It's possible to proceed with a PhD from there.