It can be an issue, but it is doesn't have to be a large one. If all of your work is joint with senior people, many people will wonder to what extent you contributed to these projects, and to what extent you were just following others. Particularly at top schools, people want to hire leaders in their field, and certainly people who will be productive without mentors/advisors alongside them.
Who you're collaborating with (the very top people or virtually unknown people, just your grad school/postdoc advisors or people at other institutions, etc) and what the results are also play a role in how this is viewed.
However, this can definitely be addressed in letters, and thoughtful letter writers should address this issue. Another thing that is helpful is if you go forth and talk to many people about math (at conferences etc) so that at many places where you are applying someone will be able to address how well you "know your stuff."