22

When I look into a specific problem over Google Scholar and simple Google Search, I find many related publications from publishers other than IEEE, ACM, Elsevier, Springer etc.

Usually the authors come from Indian, Chinese, Arabic institutions. My initial instinct is to ignore them, however I always feel as if there might be something important.

What is the right thing to do in such cases?

  • 16
    It should be noted that coming from some major publishers does not necessarily imply that the journal isn't junk. – Fomite Oct 13 '14 at 15:48
  • 7
    @Fomite... as evidenced by Chaos, Solitons, & Fractals, published by Elsevier. – S. Kolassa - Reinstate Monica Oct 13 '14 at 16:06
  • 9
    What is the right thing to do in such cases? — Read the papers yourself. Of course, that was the right thing even before Google Scholar. – JeffE Oct 14 '14 at 1:22
  • 1
    Is it right to believe that accepted or high-voted answers are correct? Is is right not to read answers that did not (yet ?) get votes? Use your brain. – babou Oct 14 '14 at 21:48
  • "Is it right to believe that accepted or high-voted answers are correct?" -> Yes! (more likely than the other answers). This is highly dependent on the credibility of the voters! – Kristof Tak Oct 16 '14 at 8:39
30

There are a lot of good publications in the world that are not published by mega-publishers. Some of them you've never heard of because they are regional or specific to certain subfields, but are still very good. It's also often difficult for people from certain countries to publish in mainstream conference venues due to visa issues. For example, IEEE and ACM usually require at least one author to be present in person, and that may simply be impossible for, say, a group of Iranian authors and a conference being held in the U.S. Likewise, the cost of travel is often prohibitive for authors from the developing world.

That said, there's also a lot of junk publications in junk venues, and even something like the IEEE stamp doesn't mean you're reading a real paper.

So how should you evaluate a paper in a dubious venue? Just like you would any other paper:

  • Is it on target with what you are looking for?
  • Are the results significant?
  • Is the data credible?
  • Is it part of a network of related papers building toward the presented result?

Publications in dubious venues are just much more likely to fail these tests.

  • 3
    IEEE and ACM always require at least one author to be present in person — Not exactly true. I have presented papers at ACM conferences on behalf of authors who could not attend. – JeffE Oct 14 '14 at 1:23
  • True, there are occasionally exceptions: I've seen a remote talk as well. But it's always some sort of special case and they definitely require a conference fee. Now, whether the organizers properly obey the regulations they have been given is sometimes a different question... – jakebeal Oct 14 '14 at 2:12
  • 1
    Strictly speaking, the visa issues are not a total obstacle. What IEEE and ACM essentially require is the registration. If, by the time the conference takes place, the authors/presenters do not turn up, the proceedings are already finalized and printed/burnt on CD. To my understanding, it's generally too late to remove any paper at that point. – O. R. Mapper Oct 14 '14 at 6:48
  • 1
    Having just served as publications chair for a conference, I can tell you that IEEE, at least, specifically asked us afterwards if anybody failed to register and present as required. They might end up on the USB drive, but would not be archived in Xplore. – jakebeal Oct 14 '14 at 7:00
  • and they definitely require a conference fee — Again, not always. In at least one case, the authors I spoke for could not attend and therefore did not register. IEEE may have draconian registration policies, but I've never heard of such from ACM. – JeffE Oct 14 '14 at 9:20
11

Google has recently done an analysis of citation trends, and found that citations to "less" prestigious journals are actually increasing:

Rise of the Rest: The Growing Impact of Non-Elite Journals

An extract from the above study:

"... now that finding and reading relevant articles in non-elite journals is about as easy as finding and reading articles in elite journals, researchers are increasingly building on and citing work published everywhere. Considering citations to all articles, the percentage of citations to articles in non-elite journals went from 27% in 1995 to 47% in 2013. Six out of nine broad areas had at least 50% of citations going to articles published in non-elite journals in 2013."

Additionally, in my personal experience (15+ years of computer science research), I've found that the breadth of ideas is considerably enhanced if one makes some effort to go beyond the so-called "top" journals, while still staying aware of publications in "top" journals.

A lot of the elite-type publications can be quite political, in the sense that stuff won't get published in them (ie. get past the reviewers) unless it follows the fashionable-approach-of-the-day and cites the "right" papers. This can lead to a reduction of new ideas and/or only incremental improvements of existing methods.

As a consequence, some of the less "prestigious" journals can in fact be a breath of fresh air, where some of the more risky, newer, and/or alternative methods are explored.

7

Back in the old days before online publication and widespread indexing of journal articles, readers depended on journal publishers to curate the research papers and select the best papers for publication. Now, there are many more places to publish, and the number of papers being published has grown dramatically. More so than in the past, good papers are published in obscure journals and bad papers are sometimes published in prestigious journals.

In evaluating the quality of a paper, you're largely on your own. However, one thing that you can do now that you couldn't easily do in the past is to check how many other authors have cited the paper that you're looking at. If the paper that you're looking at is widely cited by other authors (and the citations are not negative ones), then there's a good chance that the paper is reasonably trustworthy.

  • 9
    I'm not sure that the citation count for an article is a reliable measure of its usefulness. At best it's a blunt measure. Many papers are simply cited because someone else cited them. – mtall Oct 13 '14 at 14:51
  • 1
    I agree that citation counts are a blunt measure and not the only way that you should judge a paper. The point is that it's readily available information, and by looking at papers that cite the paper in question you can see what impact the paper in question has had. – Brian Borchers Oct 13 '14 at 15:49
6

You should have a basic workflow for assessing the merit of a resource. If it's from an unknown journal or seems a bit dodgy, I would do a quick scan of the abstract, introduction, methods and conclusion (if they don't exist then there's a red flag). Also, what resources are listed in the references/bibliography? If they are all low-quality (the MSM, other unknown journals, Wikipedia) then I'd quickly move on.

But, if the writing is clear, the research aim, hypotheses/questions, analysis and findings appear credible, there are good quality references cited, and it is relevant to the research you are doing---always ask 'how is this relevant to my question?'---then I'd read deeper and errr... Google the author to find out a bit more about him/her and the institution to which s/he belongs.

As you get further into your reading and your field you will be able to assess the quality of the resource quickly.

3

At this point, for almost any field worth doing research in there is just too much literature for anyone, even an expert, to keep up with. In the field that I work in on the order of 100 papers are published a week - I could literally read all day and still not be caught up.

Instead of this, I have found much more use in following particular groups whose work I have found useful in the past (some outside of the US/Europe axis). So, if one of these (approximately 100) groups publishes a paper in a non-major journal, I am more apt to read it. Along with this, I don't read everything in the major journals of the field, either. This might help if you don't want to completely ignore non-major publishers but also don't want to spend your whole life reading.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.