Academia Stack Exchange is a question and answer site for academics and those enrolled in higher education. It only takes a minute to sign up.
Sign up to join this community
When I look into a specific problem over Google Scholar and simple Google Search, I find many related publications from publishers other than IEEE, ACM, Elsevier, Springer etc.
Usually the authors come from Indian, Chinese, Arabic institutions. My initial instinct is to ignore them, however I always feel as if there might be something important.
What is the right thing to do in such cases?
There are a lot of good publications in the world that are not published by mega-publishers. Some of them you've never heard of because they are regional or specific to certain subfields, but are still very good. It's also often difficult for people from certain countries to publish in mainstream conference venues due to visa issues. For example, IEEE and ACM usually require at least one author to be present in person, and that may simply be impossible for, say, a group of Iranian authors and a conference being held in the U.S. Likewise, the cost of travel is often prohibitive for authors from the developing world.
That said, there's also a lot of junk publications in junk venues, and even something like the IEEE stamp doesn't mean you're reading a real paper.
So how should you evaluate a paper in a dubious venue? Just like you would any other paper:
Publications in dubious venues are just much more likely to fail these tests.
Google has recently done an analysis of citation trends, and found that citations to "less" prestigious journals are actually increasing:
Rise of the Rest: The Growing Impact of Non-Elite Journals
An extract from the above study:
"... now that finding and reading relevant articles in non-elite journals is about as easy as finding and reading articles in elite journals, researchers are increasingly building on and citing work published everywhere. Considering citations to all articles, the percentage of citations to articles in non-elite journals went from 27% in 1995 to 47% in 2013. Six out of nine broad areas had at least 50% of citations going to articles published in non-elite journals in 2013."
Additionally, in my personal experience (15+ years of computer science research), I've found that the breadth of ideas is considerably enhanced if one makes some effort to go beyond the so-called "top" journals, while still staying aware of publications in "top" journals.
A lot of the elite-type publications can be quite political, in the sense that stuff won't get published in them (ie. get past the reviewers) unless it follows the fashionable-approach-of-the-day and cites the "right" papers. This can lead to a reduction of new ideas and/or only incremental improvements of existing methods.
As a consequence, some of the less "prestigious" journals can in fact be a breath of fresh air, where some of the more risky, newer, and/or alternative methods are explored.
Back in the old days before online publication and widespread indexing of journal articles, readers depended on journal publishers to curate the research papers and select the best papers for publication. Now, there are many more places to publish, and the number of papers being published has grown dramatically. More so than in the past, good papers are published in obscure journals and bad papers are sometimes published in prestigious journals.
In evaluating the quality of a paper, you're largely on your own. However, one thing that you can do now that you couldn't easily do in the past is to check how many other authors have cited the paper that you're looking at. If the paper that you're looking at is widely cited by other authors (and the citations are not negative ones), then there's a good chance that the paper is reasonably trustworthy.
You should have a basic workflow for assessing the merit of a resource. If it's from an unknown journal or seems a bit dodgy, I would do a quick scan of the abstract, introduction, methods and conclusion (if they don't exist then there's a red flag). Also, what resources are listed in the references/bibliography? If they are all low-quality (the MSM, other unknown journals, Wikipedia) then I'd quickly move on.
But, if the writing is clear, the research aim, hypotheses/questions, analysis and findings appear credible, there are good quality references cited, and it is relevant to the research you are doing---always ask 'how is this relevant to my question?'---then I'd read deeper and errr... Google the author to find out a bit more about him/her and the institution to which s/he belongs.
As you get further into your reading and your field you will be able to assess the quality of the resource quickly.
At this point, for almost any field worth doing research in there is just too much literature for anyone, even an expert, to keep up with. In the field that I work in on the order of 100 papers are published a week - I could literally read all day and still not be caught up.
Instead of this, I have found much more use in following particular groups whose work I have found useful in the past (some outside of the US/Europe axis). So, if one of these (approximately 100) groups publishes a paper in a non-major journal, I am more apt to read it. Along with this, I don't read everything in the major journals of the field, either. This might help if you don't want to completely ignore non-major publishers but also don't want to spend your whole life reading.
