Are TED presentations academically credible?

Question

Many people, including myself, are fascinated by TED speakers. They provide us a new perspective. For example, see the TED talk The best stats you've ever seen by Hans Rosling.

However, when I talked about this to an anthropologist, he said that the statistics in this TED talk are unverifiable. He suggested that the speaker doesn't give us his sources, so therefore the talk is no longer a scientific talk. As a result, he suggested, real scientists are clever enough to stay away from these presentations.

Do TED talks have the necessary rigor and foundation to be citable in papers and other "scholarly" publications?

Answer 1

I think you are asking two slightly different questions at the same time:

1. “Are TED speakers and their ideas credible? (as in "likely to be correct")” I'd say yes, since TED's process for inviting speakers is at least as strict as that of many scientific publications. Most TED speakers are invited to speak at TED because they've become renowned scientists through their peer-reviewed publications.

2. “Are TED talks scientific (as in "usable to base your own science on")?” Here the answer is definitely "no", for the reasons you mentioned: they lack the information needed to be independently verifiable.

Note that this does not mean that their content is wrong or unscientific; it just means that the talks are incomplete from a scientific point of view, and thus not by themselves verifiable. But most TED presenters will previously have published the findings they present at TED in a regular peer-review scientific publication in a scientific format.

The bottom line is: you should treat TED talks like you should treat Wikipedia: use it to quickly understand new and interesting concepts. But before you actually apply one of these concepts, verify them using scientific literature.

Answer 2

TED talks are for popularizing ideas in science, technology and arts, as emphasized in their tagline:

Ideas worth spreading.

Very often they are a nice starting point to get interested into a certain idea and they serve as a general food for thought.

However, don't treat them as revelations, since:

• they are not comprehensive (in 5-20 min you can't give a comprehensive introduction to anything (some things require hours to explain, others - years of studies); moreover, if a talk is for the popular audience (as in case for TED talks), many crucial details need to be simplified or dropped),
• it is not rare that the presented idea is not considered mainstream (they do value originality; sometimes minority ideas can turn out to be a gold seam, which needs audience, in other cases it may turn out to be incorrect or inconclusive).

So, if you want to use the content of TED talks for anything beyond discussions over a beer, do consult other sources (e.g. scientific papers by the authors and check if the they are recognized in their field).

Also, very often checking out a relevant Wikipedia page gives insight into the status of a given view (is it present at all? is it disputed? is it this year's discovery or does it date back to ancient times?).

Or, as you did, asking experts (sure, they can be wrong too, but at least can be more objective than in a 15 min pitch).

Answer 3

Not exactly an answer to the question, but I'd just like to correct a factual error in the question.

The speaker doesn't give us his sources, therefore the talk is no longer be a scientific talk.

The time limit on TED talk probably prevents the speakers from disclosing all sources. I cannot speak for all of them, but at least for Hans Rosling (the one featured in your question), all the data source he uses are listed here.

he said that these statistics are unverifiable

This, I agree. Even sources are disclosed, we may not know if the sampling is well designed and executed, or if the measurement techniques are the most accurate. Some more exploration on our own in the data providers' site is perhaps necessary.

And the real scientists are clever enough to stay away these bullshit presentations.

To be honest, I feel that your anthropologist seems no better. It takes only a few clicks to verify the data source, and yet instead of doing that, he/she decided to call the talk "bullshit." I'd recommend you to be more careful about what he/she has to say on all other "scientific matters." Better to verify them yourself.

Answer 4

I can only speak to my area of expertise (chemistry). That being said, some of the few TED talks on chemistry are "far-out," to put it politely.

For example, in this talk (from 2011), the speaker proposes creating artificial (by that I mean inorganic-based, not the Craig Venter sort of thing) life (an idea that has been pursued for decades) in two years (skip to 14:00 for the Q/A). Needless to say, we still don't have artificial life (which is probably a good thing).

Even optimistic experts agree we are many decades from approaching an understanding of how life emerges from a network of chemical reactions.

In this context, I feel that TED talks offer anything from fantasy to reproducible scientific "fact" (or the closest thing to it), depending largely on the speaker (what their goals are) and what's "marketable" to the TED audience.

Answer 5

TED is Technology, Entertainment, and Design. While speakers may use science to illustrate their topic, and may be presenting some or all of a scientific presentation, this is not a venue intended to disseminate scientific information.

Like Wikipedia, TED might be a place to provide pointers to the people, papers, and research that is ongoing in a specific area, but you should rarely find that the talk itself is a good source of scientific information. Generally such talks are snippets from experts, and what you really want is the expert or paper being discussed, not the discussion itself.

Answer 6

So for a concrete case to consider, the first statistics Rosling presents are about relative child mortality rate in 5 selected pairs of countries. The second are the result of a small informal test of his own students' knowledge of the first.

I don't think it takes an anthropologist to analyse whether Rosling's numbers are verifiable or not. He certainly does not provide evidence in the TED talk that they are correct, in that sense they clearly are not verifiable from the talk. If that's what your friend means then fair enough. A TED talk is not an academic publication.

I'm not aware that the field of anthropology rejects in principle the notion of child mortality being a real thing. So Rosling's comparisons could be checked against other sources. It would not be appropriate to cite Rosling in a paper if you needed a source for the relative child mortality rates of those pairs of countries. You would go to WHO and/or national medical reporting, and pay close attention to their methodologies. In that sense they can be checked, and either verified or falsified. Will your friend give you long odds that Rosling has those pairs of countries in the right order according to WHO or his preferred source? Thought not ;-)

Rosling also hasn't really demonstrated anything about the state of knowledge of Swedish students, although the insinuation is that he has. He (serious-jokingly) says that there's a role for him to teach them something. His small trial is sufficient to support his small claim. His methodology is simple and obvious enough that the test is repeatable with other groups. There's no problem of fundamental science here. I don't know (and I don't think it matters) whether he's ever published on that particular result, but it's probably never appropriate to cite because it's such a limited and specific claim of no general interest. And might be cherry-picked.

Next he shows off his visualization software. The important thing to note here is that this is not an attempt to publish academic conclusions on fertility or life expectancy. He's using UN data (about which a great deal has been written elsewhere), to motivate the use of a particular tool, to combat what he believes are out-of-date general intuitions about that data. His hurried narration makes no scientific claims beyond the fact that a large group of what would be called "developing" countries used to have high fertility and low life expectancy, and now don't. And that AIDS reduced life expectancy in Africa. Neither of those is really controversial enough to really warrant further justification in this context: like the comparisons between pairs of countries it just provides something for you to go away and look up if you want serious corroboration.

There is nothing really to verify or falsify beyond his claim that anyone holds this intuition about developing countries in the first place. That is substantial, it's presented as novel and, even worse than his informal student study, he doesn't quantify it, let alone support it. He's not properly publishing a scientific conclusion, but then I'm not persuaded that he's pretending to.

And so on. In the next comparison US vs. Vietnam, I think he says "by the end of the year" when he means "by the end of the decade". Slip of the tongue, should never be allowed to stand in a carefully produced, copy-edited, reviewed scientific publication, but there it is. So in a sense, no, public speaking is not credible at all since such errors are far more common. In another sense, does this mean there's something wrong with public speaking?

One cannot cite (or even trust) the content of a TED talk as if it were the content of a peer-reviewed journal. TED doesn't do that. Neither is there AFAIK any fact-checking other than what the speaker does or arranges. You can treat the content of TED talks the same way as you'd treat the content of any public address by that speaker. So if you were writing a paper that for some reason needed to know what Rosling specifically says in public, then using his TED talk as a source might be reasonable. Otherwise, not so much, but then why would you want to?

the talk is no longer a scientific talk, and the real scientists are clever enough to stay away from these presentations

I think that's akin to saying that TV documentaries are not scientific talks and real scientists are clever enough to stay away from them. It's true that TV documentaries are not journal articles. It's true that there can be fakery and stupidity in their vicinity. However I don't think it's true that no real, clever scientist can get involved. They must distinguish the activities of "publishing research" and "popular education", and avoid claiming that one is the other, or applying the standards of one to the other.

TED talks are as credible as the individual speaker. The fact that TED has "chosen" them should lend no authority at all but probably, unfortunately, does. As for the credibility of TED as a forum, I don't recall what wit here in the UK observed that the country has very many people who would turn down an honour in principle, but tragically are never offered one. I suspect the same may be true of TED -- there are various reasons you might not want to do it, among them that the content of the talks tends to be over-trusted by people who enjoy TED and its speakers. These reasons are not universally applicable, and in any case apply most strongly to those who won't be invited.

Aside from giving talks, experts in the field might choose to watch or not watch the presentation on the same basis they choose any popular presentation of their field. On the one hand they might be interested to know what outsiders are hearing. They might enjoy the speaker. On the other hand, what are they going to learn? Everything shown will be either commonplace in their field or else the specific and perhaps controversial views of the speaker, that an expert could better assess by reading their publications than by seeing the simplified popular version.

Answer 7

Let us work out the following analogy: TED as a journal, Talk as a published article.

When one meets a questionable article with unverified claims, they may of course doubt the conclusions. A usual thing to do then is to write back to the Journal and raise your concerns to them. The Editor may possibly come back to author inviting them to write an erratum explaining / correcting / substantiating the conclusions of the Paper. This process is known as "scientific discussion" and is a blood flood of academia.

As long as the Journal maintains and empowers such procedure, I would see it as a completely trustworthy source, despite occasional mistakes and errors in published articles, which are probably inevitable in real life.