I'm going to a conference in a few days. Among the presentations I'd like to listen to, there's one paper that is nicely written but is extremely flawed methodologically. In short, the authors violate at least three assumptions of causal inference like it's no big deal - and they don't even discuss these assumptions. Due to that, any attempt to interpret the results is laughable. (I'll edit in more details, if needed).
The point is, if it were a good paper, I'd love to keep contact with the authors. I'm familiar with their previous work, they are usually good in doing what they are doing. We work in neighboring sub-fields. It's unlikely we'd work together - we just happen to use similar methods for different things. But it is likely we'll keep bumping into each other at other conferences or will review each other's papers.
Right now I see three options:
- go to their panel, raise my hand during the Q&A time, list everything that's wrong with their paper. The presenter is really not going to like it (I know I wouldn't).
- talk to the presenter before or after the panel over coffee, list everything that's wrong with the paper, but save him public humiliation. Problems: we are not presented to each other, so he might go on the defensive and stop listening to my comments right after the first "Man, you can't do it this way." Obviously, if he reacts like this, any "let's stay in touch" would be impossible, too.
- say nothing, hoping that reviewers would say it instead. Problems: I don't establish any contact with the authors; it might be quite some time before the paper goes through the review process; the reviewers might not be that rigorous and familiar with the methodology (the latter depends on the submission venue, I guess).
If I decide to give feedback, how to use these options to make this feedback useful without sounding like I'm trying to trash these authors' work?
Additional details that might affect the responses:
- Quick googling shows that the paper now has a "working paper" status and is under submission somewhere. And the results have been already featured in The Financial Times. Oh, well...
- Seniority: both the presenter and I are at the post-doc/assistant professor level. Two other coauthors of the paper are senior professors.
- "Are you sure you got the flaws right"? I'm sure.