Is it OK to ask my advisor to contribute more to the paper (other than just critiquing)?
Yes, I think so.
If so, how does one go about doing this without insulting their mentor/boss?
When I was in similar situations in the past, I asked my advisor specific questions about what it was that I needed help on, but I did not necessarily try to sugarcoat the request or anything like that. For instance, if a paragraph needed rewording, I would email my advisor something like "Paragraph Y seems a little wonky to me. While I work on Section X some more, can you come up with a better explanation for Paragraph Y?"
I found that the success rate of the above varied depending on how busy my advisor was. My advisor seemed to "get better" (from my perspective :) ) the more times I asked for specific help. Also, I wasn't afraid to speak up when my advisor offered general tips rather than the specific help/contribution I was after: I would reply and ask again. If on multiple iterations I did not get what I was looking for, I would piece together what they did give me and work with that. That's life.
As a side note, I personally found that trying to word emails etc. just so to avoid coming off as "insulting" etc. required too much effort on my part for little/no gain (you don't even really know if your "properly" worded emails come off the wrong way, so what's the point?). In the end, I came to the conclusion that my advisor and I were two adults working on a research project together. I like to think that our advisor/advisee relationship could be described as direct and respectful, and I think that should be the goal.