The problem you will have is that for advanced material, the likely market is very small. If you aren't widely known already, there is the possibility that the market is zero. So, publishers will be wary. They like to publish first year textbooks where the risk may be there, but the potential upside is large.
I wouldn't expect an advance payment (which is likely a charge against royalties in any case). A book that would sell for over $100US might carry a royalty of around $5 per copy, less any marketing charges that are assessed against the book.
Publishers also don't generally spend much on promoting books beyond the first few years. If the book "sells itself", as some do, then you can expect a royalty stream for several years, otherwise it isn't all that lucrative.
If you have a way to do it, try to figure out what the annual sales might be for the book. How many places teach a course for which the book would be useful? Are there already good books that are used? What sorts of enrollment in those courses? Have your materials been tested? Publishers have a good handle on some things, but especially lower level courses.
I have a friend who wrote a (professional) book that we all predicted would be a huge success, but it turned into a bomb. It's just that the market turned elsewhere as there was nothing wrong with the book (I bought one, actually). On her next book deal the publisher (large and well known) required that they recoup their losses on the previous one before they would pay her any royalties at all.
If you want to publish, do it for its own sake. You will get some recognition in, perhaps, a small circle, but it is unlikely to pay for your retirement. The fraction of highly successful books that sell well over several years is very (very) small.
But, you might get lucky and you will probably learn some things just by doing it.
