153

I've attended a lot of seminars and lectures by now, and it's typical for the chair of the seminar to offer audience members a chance to ask questions of the speaker about their work after the presentation.

Most of the time, the questions seek clarification of some aspect of the presentation or focus on a more comprehensive understanding of the research involved.

However, occasionally, I've noticed that questions are purposefully designed to embarrass the speaker. Things along the lines of "That method won't work at all for what you're trying to do. Your results are completely invalid" or "So-and-so's group already did that work years ago. Did you not read their paper?"

Perhaps more disturbingly (I just got back from a really large conference if you can't tell), is that women seem to be more harshly criticized than men, and over trivial issues. For instance, in a few sessions I went to, female graduate students were given really hard times over their presentations while the male grad students were not. All presentations were about the same quality. I guess I'm a little shocked; I'd heard of sexism in academia but hadn't actually seen it (or noticed it) until this conference.

Oddly enough, I've observed that it's typically prestigious professors or researchers that are asking these ostentatious questions. I suppose they figure they have enough "fame" or whatever that their job isn't in jeopardy, and there's no easy way to really prove they're being rude or sexist.

I don't know what's going on here, but it seems to me that these questions, even if they do have technical merit, should be held until after the seminar, where they can be discussed privately with the researcher.

My question here, specifically, is what can be done to minimize these (uncomfortable for everyone) instances? My thoughts are that a session chair should remind the audience to refrain from questions that are accusatory in nature. As a presenter, I'm not sure what can be done in advance to preempt and avoid these questions. Any ideas?

  • 21
  • 32
    Would you give us better examples? It seems to me your second one "So-and-so's group already did that work years ago. Did you not read their paper?" is point frank and you're somewhat obligated to answer it. – scaaahu Nov 24 '14 at 6:31
  • 16
    Professor Cheng-Ning Yang, a 1953 Nobel Laureate in Physics, is good at dealing with this. I noted that he would use "I do not know how to answer this question" to decline the "unconstructive" questions. For example, you can read his popular books on his own research and teaching experiences and will see how he delt with Pauli's relentless questions. – Megadeth Nov 24 '14 at 8:27
  • 14
    This will sound like a joke, but it's not: binge-watch all of The West Wing and study how C.J. handles heckling in her press briefings. – Lightness Races in Orbit Nov 24 '14 at 12:36
  • 20
    It is simply rude for senior people to "mess with" junior people in a public forum. It does seem to be popular with some people, but it is not "accepted style" in the U.S., at least not within a reasonable distance of me. Yes, to my perception, junior women are often perceived (by bullies) as more vulnerable to bullying. – paul garrett Nov 24 '14 at 13:33
69

Firstly, I should note that the examples you give are certainly somewhat agressively formulated (more so than would be common in my field), but not in themselves invalid questions. It is certainly "allowed" to be critical of the presented work, and there is nothing the session chair can or should do about this.

As a speaker, it always helps to think in advance what kind of "negative" question there could be, and prepare for them. You know that your work is very close to Foo et al, 2001? Good, have a backup slide that highlights your contribution over them. You think people would argue that a much simpler standard approach would have also worked instead of your super-complicated new custom method? Have a backup slide which compares the results of the two methods (you did test the standard method first, right?). If your answer does not convince the person asking the question, then much of what I wrote in this related answer also applies here. Specifically:

(...) try to explain calmly why you did what you did. Yes, maybe that person asking the question will disagree, but so what? The fact that your actual peer reviews are good shows that there are a non-trivial amount of researchers that actually agree with you. The person asking the question is not your supervisor, you don't need to agree with him/her specifically on your research agenda or approaches.

As a session chair, you need to step in as soon as a question starts to become an ad hominem attack. That is, "I think this has been done years ago." is still fine, while "How the heck did you even get a PhD?" isn't. Further, a session chair should interrupt a series of questions as soon as he feels that the discussion isn't of interest to the larger audience anymore. I do not think that there is much that can be done in advance. As a session chair, you typically want to foster discussions, not preemptively set out the ground rules.

About the parts regarding sexisms and whether senior professors tend to be mean: I don't really have much to say about this. Not being a woman, I have not yet noticed any particular pattern about how females get asked questions versus males. The same is true for whether senior professors ask more critical questions - personally, I rather have the impression that fellow grad students and young postdocs or assistant professors are more likely to be extremely critical of other person's work than more senior researchers.

  • 14
    @O.R.Mapper With that kind of comment the person asking the question would really only embarrass himself. – xLeitix Nov 24 '14 at 9:32
  • 14
    @O.R.Mapper In the "you don't know" situation, it might be good to say a bit more than what xLeitix suggested, perhaps something like "I was not aware of that work. Thank you for bringing it to my attention. I'll look into it." My reasoning is that (1) even if the questioner intended to be nasty, (s)he has given you information that is likely to be useful, and (2) by being as polite as possible, you make any further nastiness from this questioner look particularly bad to the rest of the audience. – Andreas Blass Nov 25 '14 at 4:17
  • 14
    I don't understand this comment: "Not being a woman, I have not yet noticed any particular pattern about how females get asked questions versus males." After all, being a man, haven't you had precisely the same opportunity to notice patters in how males get asked questions versus females? – Tom Church Nov 25 '14 at 15:14
  • 8
    But we're talking about two groups of people! It's not meaningful to say "one group is treated the same and the other group is treated different"; the two groups are treated differently from each other. – Tom Church Nov 25 '14 at 16:11
  • 11
    @TomChurch It is, if there is one group that is very large (males) and one group that is at most 20% (females), as in my field. – xLeitix Nov 25 '14 at 16:12
47

As others have noted in the comments, there's a difference between questions that have legitimate content but an unnecessarily aggressive tone (e.g., "Isn't it obvious that won't work for reason X, you fool?"), and questions that are purely verbal attacks with no real substance ("How did you even get accepted to this conference?").

As a speaker, the best way to respond to the first kind of question is just as if it lacked the harassing component. If you calmly address the substance of the question without getting your dander up, the questioner is the one who will look like a jerk for adopting such a confrontational stance. Of course, it's especially nice if you can cleanly dispatch their question with a concise and accurate rejoinder (e.g., "Actually, Smith and Jones showed in a paper last year that this method works quite well"), but if your research is sound, even a fairly garden-variety response is probably adequate (e.g., "The jury is still out on that question, but our results show it's worth investigating further").

As for the second kind of question, at least in my experience it seems to often take the form of a sort of rambling rant by the questioner, directed less at the current speaker than at some whole research area or methodology. (For instance someone saying, "But don't these kinds of studies always run up against the problem of. . .") I have seen people come of well in responding to these kinds of questions by waiting patiently and then responding good-humoredly but pointedly with something like "I'm not sure I caught what your actual question was."

If the question really is a direct attack or a "gotcha" attempt (e.g., quizzing the person on one particular paper), you'll rarely be faulted for just saying something like "Maybe we can discuss those details individually later" (i.e., in the coffee break or whatever). I think this is the ace-in-the-hole response to many questions that try to derail the question session and turn it into a one-upmanship contest. Of course, it's best only to use it when the question really is out of bounds (or too large in scope to be answered in the question session), because if you try to deflect legitimate questions this way you'll look like you haven't done your homework.

As an audience member, I've noticed that usually when someone asks a confrontational question, it seems to make the audience uncomfortable as well. Often other people have questions and would like to ask them before the question time is consumed by verbal posturing. So it helps to remember that, if the question is really uncalled-for, the audience is probably "on your side" and will not think less of you for simply deflecting it and moving on.

  • 2
    +1 for the "discuss those details individually later" approach. – R.. Nov 25 '14 at 5:44
  • 1
    -1 for the "discuss those details individually later". If someone says that your results are completely invalid, this is a very weak answer. Your response might be short, but you shouldn't be oblivious to such statement. – Tomas Nov 29 '14 at 16:34
35

As a presenter I believe that the best advice I can give is be honest, be yourself, and be prepared.

This goes for your presentation and any research/material surrounding your lecture.

I have given many presentations/lectures/seminars in the field of computer science. Many of these at one point in time revolved around trade automation within direct exchange fields.

When I first started doing this, I was in my mid 20s. The audience was 90% men, almost all of them 40+. I have to say at first I cringed when we opened things up to questions. Things were anywhere between uneasy to hostile - mainly due to my age and lack of time in the field.

Some general tips:

  • Understand that some things you say people will take that as a threat to whatever their "job" is. Change, new technology or new data can make people fear their abilities. I once discussed a way to cut two milliseconds off an exchange execution. This technique threw out a standard industry mechanism. My boss promptly got several calls from executives at two firms wanting me fired. A year later that industry mechanism was a thing of the past.
  • Your topic might be interesting but there is a 99.99% chance that it will have little effect on the history of mankind. You can't put off an aura of godliness and not expect a few smartass questions.
  • Give credit where it is due. In my case I combined several theories and current technology to produce something that worked efficiently together. I did not at any point in time act like I created everything. I also (depending on audience) liked to give some shout-outs to those who helped or those whose work influenced what I did.
  • Know your audience. I like to know who might attend my sessions. I know that people from certain groups might have certain types of questions. Also at the same time try to acknowledge this audience during your time. Often you can neutralize a harsh person by giving a strategic compliment before they have a chance to ask a question.

Quick bits on your specific questions:

That method won't work at all for what you're trying to do. Your results are completely invalid

Answer: Your session should have been clear enough to answer or refute this. If it can be asked without everyone rolling their eyes then you have not done your job as a presenter getting the facts across. If someone asked me this I would calmly tell them why I disagreed (in less than a minute).

So-and-so's group already did that work years ago. Did you not read their paper?

Answer: If your work is just an extension of theirs then this should be part of the presentation. And then during the presentation you would tell how yours differed (you don't have to "compare" the entire time) and give your additions. Again if you did not do this you have failed as a presenter and it is a valid question.

that women seem to be more harshly criticized than men, and over trivial issues

Answer: People always judge others by how they look. This is a fact of life. I was an ex-football player. I get lots of dumb-jock looks and remarks. You can't get hung up on this. Just answer the questions and keep your opinions of their motives to yourself. (You can despise the person and still answer their question nicely)

it's typically prestigious professors or researchers that are asking these ostentatious questions.

Answer: Be self-deprecating, make sure that others know that you don't think you are the smartest person in the world. This seems counter-intuitive but they will soon see that they just spent two hours listening to someone who doesn't think that he/she is smart, so where does that leave them?

As for the chair... People should be allowed to ask questions even if tough. I don't believe in asking certain questions behind closed doors unless the question is truly personal (which I didn't think your examples were). However, the chair should step in if the people asking the questions are keeping others from asking something or if they are just taking too long.

  • @xLeitix - Sorry I left without saving! – blankip Nov 24 '14 at 8:41
  • 7
    Regarding "So-and-so already did that work," one cannot always anticipate it. It's quite possible that they really did do all that, and you just weren't aware of it because they published it in an obscure journal, or one aimed at a different field, with different journals and jargon, that just happens to occasionally overlap yours. (For instance, there are plenty of results in applied math that could apply equally well to physics, economics or mathematical biology, or might just be published in a math journal.) ... – Ilmari Karonen Nov 25 '14 at 8:54
  • 7
    ... Or it could be that the questioner is wrong, and so-and-so actually did something superficially similar but unrelated. In any case, you cannot always anticipate these questions, so you'll need to have ways to deal with them when they come "out of the blue". ("That's interesting, let's discuss it during the break" is often a good one.) Ditto for "that's all wrong!" -- maybe it is, or maybe the questioner just doesn't get it, or maybe they dozed off and missed a crucial point. In any case, the public Q&A session at the end of the presentation is not right time to settle such disputes. – Ilmari Karonen Nov 25 '14 at 8:59
  • @IlmariKaronen - Those are good points. If I didn't know. I would simply ask for information around their points and tell them I would get back to them. At the same time I would feel a little embarrassed for not knowing - I am sure this can happen easier in some fields compared to others. I feel like if someone is coming to listen to me for 2-4 hours I should really know my stuff. – blankip Nov 25 '14 at 16:09
  • It would be however, completely appropriate for the chair of the session (or audience members) to point out sexism if they believe it is a problem in their session. However, that is their decision and there is little the speaker can do to remedy this. – WetlabStudent Jan 21 '15 at 6:41
20

My method seems to be a bit more... devious... than the methods proposed by other answers. This is something that was handed down in a roundabout fashion from other researchers I know. No one talks about it directly but I've had one on one conversations when it comes up.

As a presenter I try to make sure to lead the audience to what might seem like a couple of 'gotchas'. Just as I control the method and content of the presentation, I think it's important to control, to a limited extent, the questions and post-talk discussion.

What does this mean? I never give a talk without a pretty good idea of the worst questions I could get about it. This requires having a group, preferably in your lab, who is willing to give you harsh, constructive feedback. This requires an advisor or supervisor who is willing to listen to your presentation and put their 'asshole' hat on and help you identify those questions. This requires being willing to be told that your baby is ugly. A lot of research groups are very positive and that's good for the most part but when it comes time to present you need honesty and a little bit of appropriate 'meanness'.

Additionally, I typically try to lead the audience to questions during my talk. The 'jerk' questions tend to pop up when others don't have questions. By making a talk that is engaging and encourages questions in a specific vein the presenter is less likely to receive an unhelpful, insincere question. This is a bit more subtle and there's a delicate balance. A presenter should never leave pertinent information out but... there's a line right? With experience a presenter can both know what questions to expect and encourage some questions over others.

It's a bit weird, isn't it? One would think that a 'perfect' presentation would involve precisely describing the topic in such a way that questions are unnecessary but the real world of presentations having questions, having people be engaged is beneficial.

Of course nothing will remove the 'jerk' questions entirely. I once gave a presentation on a visual classification system. A faculty member, a well known faculty member albeit in a different subfield, raked me across the coals. Why? Because "Training data is stupid, you just need to tell the robot what a chair is. Why are you spending so much time on something stupid!" Which, for anyone who understands classification systems is... not accurate or very helpful. Sometimes you just gotta let that crotchety jerk get their poison out, say a couple of 'mhmms' and nod while finishing with a "That's very interesting. I'd love to talk more about this with you. Why don't you email me at..."

  • 3
    Some of the best academic presentations I've seen use this method of inviting questions that propel your presentation along. Even if nobody asks the question, you can always say, "right now some of you may be wondering how we got X. Well, in my next slide..." Prepping a significant collection of "discussion" slides that may or may not be relevant also helps shut down gotcha-seekers marvelously well. As soon as the first one is rebuffed with "I'm so glad you asked that, let me show you this graph in my appendix slide," others tend to stay quiet. – Calvin Fisher Nov 26 '14 at 17:33
  • Reminds me about leaving a minor typo in a report prior to boss review, or a simple problem in a home prior to inspection. Some people have a need to pick at something, for whatever reason, and by giving them a simple problem to solve, they've scratched the itch and won't feel like digging too deeply. For an academic presentation, though, you'd want, and should welcome, any and all comments, opinions, and questions in order to improve. While controlling the presentation is important, I suspect people who follow this tactic are depriving themselves of some valuable feedback. – Adam Davis Nov 26 '14 at 17:40
  • I think this is also a good scientific habit, and preventing ugly questions is only a positive side effect: There is always some aspect to inquire further, something to keep the science business running. This is what conference talks are for. They are not for silent admiration. I think especially elder professors might feel offended if they think a speaker is just looking for admiration. So they ask the "jerk questions". – Horst Grünbusch Nov 26 '14 at 22:29
  • @AdamDavis - Perhaps I should have been more clear. The goal is to encourage constructive, helpful questions that engage the audience and, in doing so, minimize the unhelpful 'jerk' questions. From my experience it's not the 'jerk' questions that end up providing valuable feedback, typically those are being asked to tear down the presenter and make the asker feel/appear more important/impressive. – Nahkki Nov 28 '14 at 22:39
19

A neutral way to effectively bypass the question could be to ask "may we have a chat about that in the break?".

Those sorts of questions are often not productive to answer in front of an audience, particularly if you don't think it's a valid concern. If you have an answer on hand, then you should answer the question - but you don't want to get into an argument on stage with a well-known professor if you cannot give a concise answer.

By doing this, you're also encouraging the audience to ask questions that are concise and clearly answerable.

  • 3
    I agree. If a question completely derails you - e.g. "Foo et al did this years ago," when you've never heard of Foo et al, say something like "Thank you, I'm not familiar with that work. Perhaps you have time to talk about it later?" – Jeremy Miles Nov 25 '14 at 17:33
  • 1
    Unfortunately an all too common result of this approach is that the presenter avoids the person asking the question for the rest of the conference and problem with the work is never discussed. A direct answer (if you have one) is much better than bypassing it. If you don't have an immediate answer then you should say so (before suggesting an off-line discussion). – Dikran Marsupial Nov 26 '14 at 9:31
13

First, a general method for softening rudely-posed questions is that the session chair can rephrase the question into a more productive form. That is, the useful and constructive content can be separated from possibly hostile tone or affect of the questioner... especially if, as the session chair can probably judge by the affect of the speaker, the speaker is flustered.

This mediation is potentially relevant both in the cases where (apparently) the question is sheer bullying, or (apparently) the speaker is actually mistaken or ill-informed or ... and cases in-between.

10

"That method won't work at all for what you're trying to do. Your results are completely invalid."

Stay calm and kindly answer:

"Thank you, I am very grateful for your input. Could you please be more specific about why you think it wouldn't work?"

This way you will embarrass them - you will show that you react like a kind person and scientist, while they weren't able to be specific enough. If they don't specify their critique, you are fine. If they do, you turned the emotional argument into rational one, and you can respond factually.

"So-and-so's group already did that work years ago. Did you not read their paper?"

Stay calm and kindly answer:

"We did a lot of research of the published studies, but all of them were inappropriate for our case because of A, B or C. But we could miss something, so I would be very grateful if you give me the reference afterwards."

This way you present several things: that you actually did the research, that you can accept the critics and that you are working like a scientist.

  • 2
    I had a question once like the second and I exactly answered something like this and it was well received by the audience. – Mikaël Mayer Nov 26 '14 at 0:21
  • 2
    -1 The first suggestion has the dramatic ability to backfire if the asker then public outlines several reasons why the method is invalid. – user20640 Nov 27 '14 at 23:59
  • 5
    @Lego you have to turn the emotional and vague critique into particular reasons! First, because you are scientist and you are actually interested in those reasons, and second, if you don't do it you are silently accepting the critique. Your approach "Please don't tell me why, I don't wanna know, and I am afraid of being embarrassed in front of these people" is not scientific and will only embarrass you. – Tomas Nov 28 '14 at 7:13
8

My question here, specifically, is what can be done to minimize these (uncomfortable for everyone) instances?`

I disagree with your premise, that one should do anything to minimize these instances. These questions (excluding sexist behavior) are not only legitimate, but are a useful mechanism of quality control that forces researchers to perform their research more rigorously. A bad researcher can simply ignore most other kinds of criticism of their work without negative personal impact. But they cannot do that in a public forum.

If your research is solid, you should have no problem to defend it against any legitimate accusations, even in a public forum such as a conference. If, however, the person asking that question has a point, it is their duty as a researcher - and the real point of a conference - to point out your mistakes, so that you and others in the room can learn from it.

1

To the person asking a malicious question regardless of his/her status depicts lack of etiquette. I think the Chair or Moderator should quickly intervene by asking to discuss the issue after the end of event so that the speaker can adequately respond in more details.

  • Why should etiquette (or the lack thereof) be a reason to interfere with a scientific debate? – Robert Buchholz Nov 30 '14 at 14:44
  • 2
    Personal attacks and rudeness waste time and needlessly mucks up the atmosphere of discussion and debate. Moderation is about making sure that you have a good debate, and keep your audience interested. I love a good argument or debate, but I have increasingly little time for people who are rude because they can be, and I'm hardly alone. If the gathering's goal is to further scientific or other learning, this kind of grandstanding is illogical and wasteful, and turns off the audience. – Tail-Kinker Dec 4 '14 at 0:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.