I have two suggestions that you might consider. Neither is a panacea but one or the other might help.
If this happens frequently, or is likely to, have a colleague who has "native" speaking skills and with whom you are in frequent successful communication listen for questions and relay them to you as needed. They can understand the speaker and you can understand them. I'm very deaf and have trouble understanding almost every conversation. But I can communicate well with my spouse, as I hear her everyday and know how her mouth moves when she is speaking. She often has to interpret for me in any social situation, even among friends where accent and language are not an issue. Some deaf speakers have colleagues or others use sign language to relay questions.
The second way, that works in some, but not all, situations, is to ask the questioner to meet with you after the talk in a quieter environment where conversation is easier and you can have some back-and-forth that is hard in a public space. Questions asked in auditoriums can be hard to hear even in the best case. A disadvantage, of course, is that others don't get to hear your answer. But even here, giving "I'm not a native speaker" as the reason for the request to meet is probably a good idea.
But I would avoid any suggestion that the other has an accent or other possible communication issue. Own the problem yourself.