How do you judge the quality of a journal?

Question

A colleague and I recently submitted a paper to a journal with an impressive-sounding name, the "International Journal of Artificial Intelligence and Computational Research". According to their website,

IJAICR is a referred [sic] journal in the field of computer science, artificial intelligence and soft computing methods.

It was accepted two days after we submitted it. That's too fast. We were suspicious. Although the journal said that all papers are peer-reviewed, we could not see how that was done in two days. Plus, we received no comments from the reviewers. Also, the submission guidelines didn't ask for a "blind" copy (without our names or any references to who we were).

But wait, there's more.

The acceptance letter asked us to send them US$300 to publish it. We did not. We've withdrawn our submission and will submit a new version of the paper to a more reputable journal in the coming months.

How might we make a better choice of respectable journals before we submit next time?

Answer 1

The way I usually choose journals is by looking at where people I trust/follow publish, and where previous work was published. It is usually not too hard to compare the quality your work to the quality of the work you are citing, and chose a target based on that. Unless your field is highly mutli-disciplinary, you will see the same journals/conferences popping up again and again in your references; submit to one of those.

Before submitting, however, it is always important to look at a few articles from previous issues. This will give you a second gauge of quality for the journal and also let you pick up on any formatting and presentations quirks that might be present in the publication.

Answer 2

You may want to check that the publisher is not on Beall's List of Predatory, Open-Access Publishers.

Surprisingly, the publisher in question is not. I've found Beall's list to be fairly comprehensive, but the vanity press industry seems to be booming.

Caveat: that list is just one guy's opinion. But it resonates with my own experience.

Update: As of late January 2017, Beall's list has been taken down.

Answer 3

The best way is by word-of-mouth: ask around the department, ask your PhD advisor, ask people you've worked with.

If it is a specialist journal, and you are a specialist, then the next best way is to look at the previously published issues of the journal and see what kinds of articles they accept.

Failing that, the Australian Government's Research Council puts out a ranking of journals and conferences every now and then. It is not perfect, but should give a rough idea of where a journal places in the eyes of the bureaucrats :-). Note that "new" journals (journals that have not been active for more than X years) are not ranked, so omission from the list does not necessarily mean that the journal itself is not worthwhile.

Answer 4

"How do you judge the quality of a journal?"

One answer is already known to you: Submit a paper to it and see how it responds. I once submitted a few papers to an online journal (no print equivalent) and the referee reports that I got clearly showed that they read and understood the paper and that they knew the subject well. They even suggested ways how the paper could be improved. The quality of the referees reflect the quality of the journal.

Answer 5

For some hard data I like eigenfactor, because I think there methodology makes a lot of sense. The default settings are a bit odd, you want the AI score not EF score. I also like the "eigenfactor category" under advanced search better than the "ISI category" that you can get on the main page. Certainly it's better to have a more in-depth understanding, but also sometimes you just want a quick ballpark guesstimate of how good the journal is, and eigenfactor does that.

Answer 6

I'm surprised nobody mentioned this yet, so I'm adding it as a separate answer:

Read some articles from the journal (in your subfield), and see how they feel like

When you look at a journal and check that the 5 or 6 latest articles in your domain are of the meh type, you probably don't want to publish there. If they make you feel “oh, I never quite thought of that, it's clever… I see how I may use it”, go for it!

Answer 7

I think that "quality" is a bit overrated, and that you should think most deeply about "fit" (i.e., how well your content matches the interests of a journal's reviewers and readership). With that said, when I'm looking for the best journal to put an article, I have three methods:

1. Look at my cites for similar work to my manuscript, then look at where they published (similar to Artem's comment). If your paper isn't citing related work, well... it probably has a pretty terrible lit review. If you're citing sources out of your league for that paper (i.e., Science, Nature), find the publication list for that lab to find their fallback journals.

2. Ask greybeards I collaborate with where they would publish such a paper. Good senior academics have a huge amount of expertise about the social networks and publishing networks in their fields.

3. Use journal rankings. I find Scimago to be the most convenient. That also charts things like cites/paper over time, so you can see if a newer journal is growing or stagnant. The Austrialian Research Counsel (mentioned by Willy Wong) is probably my #2 resource. Thompson's ISI stuff is also useful, but I've found it sometimes has glaring omissions and has been less convenient to me (annoying to log in). Conferences are harder to rank using indices. Google and Microsoft have ranking systems that catch those pretty well, at least for comparing citations within a topic. However, given the choice, I'd still go with option #1 or #2. I mainly use this approach for interdisciplinary research that doesn't have an obvious, natural target journal.

Answer 8

First of all, I look at the publisher. If you don' find a lot of references to the publisher on the web, it is suspicious.

Then, I look at the editor in chief, and at the board of editors. If you don't find many big names here, this is again very suspicious.

Finally, google the journal name and look who is publishing in the journal. Most of the time it becomes clear if the journal is serious or not.

Answer 9

"How do you judge the quality of a journal?"

Look at the editorial / advisory board. They've gotta be from the top schools or they've gotta be top researchers.

Answer 10

You can test them yourself. This procedure will speed up by your experience. There are signs during submission and during review which shows which journal is really good and which is not. But there is another way too: look at the objective information below. It is possible to judge a journal based on the COLLECTIVE information you obtain from these factors:

1. Their editorial. You don't know them? Ok pass to the next ones.
2. Their publishing country (India? no thanks unless the paper is not good or the journal is an exception).
3. Their publisher (yes good publishers usually select good journals, although some weak journals with a lot of money can again hire a good publisher)
4. The time passes since you submit and they respond
5. The above factor SHOULD be considered along with the amount of manuscript they receive. Good journals receive thousands of manuscript a year, but still do not waste authors' time by keeping them waiting for too long before a rejection decision. Bad journals receive sporadic manuscripts and keep the authors wait for months until they tell the author their decision. It is a pain when you see some of them have "lost" you paper, or some of them reject your paper with a couple of lines of comment, after 6 months. Good journals do the same in less than 24 hours.
6. Indexing databases. Look where the journal is indexed in. ISI Web of Science? Medline? or what? The scam journals usually are not indexed in any accredited databases (not Google Scholar or Scopus). If a journal is accepted to be indexed in ISI or to a lesser degree, Medline, it is unlikely to be able to have low quality. Otherwise, ISI would have booted them out.

There are other factors too. But these will give you 90% insight already.

Answer 11

Ask your institution's scientific librarian.

It's part of their job to know about journal reputation and quality.

Answer 12

Look at the journals responsible for the references you cite in your own manuscript. Upside, these are the journals publishing relevant and credible work in your area. Downside, there are plenty perfectly good journals that are newer, or less specialized, and so might be missed.

BTW, a very fast review time with no reviews is suspicious. But charging an article processing charge (APC) is not, in itself, a reason for worry. That said, do look to see that you are getting the full open access you pay for if there is an APC (i.e. no transfer of copyright to the publisher and the article is released under a CC-BY license). You can search for copyright policies by journal or publisher here: http://www.sherpa.ac.uk/romeo/

Answer 13

The Journal Ranking site (www.journal-ranking.com) aims to rank many (11K) journals, according to their measure of impact (mainly number of citation, but weighted according to the ranking of the citing journal), number of articles, etc. It also let you sort by field.

But again, this is only their point of view, and their way of measuring quality.

(and it only ranks journals listed in the ISI's SCI)

Answer 14

Generally, the scientific specialists' communities have traditionally identified journals having high editorial standards. That is the key: who are the editors and what are the standards. Are the breakthrough discoveries sent to that journal? From that, reputations are built and filtered down to preference and use by the community.

Answer 15

You can search for conferences/journals on Google scholar: https://scholar.google.com/citations?view_op=top_venues&hl=en

Search by the full name (not the abbreviation), and you'll see its h5-index, which is the h-index in the last 5 years. Although it is hard to judge a conference using its h-index, you can at least compare them to one another.

There is also a list on the left from which you can get the conferences/journals with highest h5-index.

Answer 16

The Impact Factor is usually a good independent means of establishing a journal's quality. Essentially it is the average number of citations of its articles. The higher, the better. The logic is that good quality articles are cited more, so a journal that only lets in better quality articles will have a higher impact factor than one which lets in practically anything.

The quality of the journal you submit to will reflect on the quality of your paper, so you should try to submit to the better ranked journals within your field. You should be able to readily establish a short-list of suitable journals just by listing the journals of the papers you cite.

There are a few things to be aware of when comparing the impact factors of journals:

• Impact factors also reflect the size of a field, so you cannot use it to compare the quality of journals from two separate and unrelated fields. A group of papers in a large popular field are naturally going to get cited more than those from a very small field.
• Almost all journals have impact factors, unless they have been excluded as a result of being in some way nefarious (such as predatory journals), or if they are newly established and therefore haven't been around long enough to get one.

I could not find an impact factor for the journal you mentioned, even though it seems to have been around for at least 5 years. You are right to be cautious of it. I would not recommend submitting to a journal which does not have an impact factor, even if the reason for this turns out to be quite innocent.

Answer 17

To make a better choice for submitting I would google and use Wikidata.

Googling might lead you to lists like Jeffrey Beall's https://scholarlyoa.com/individual-journals/ and provide you with context about the journal, such as editorial board, blog posts, etc.

Wikidata will provide you with pointers to indexing services and journal rankings. Consider Journal of Machine Learning Research at https://www.wikidata.org/wiki/Q1660383 . It indicates a BFI level of 2 which is the high category of the Danish journal ranking system. It shows that Scopus is indexing the journal. (Strangely the Australian ERA link does not seem to work). I do not find "International Journal of Artificial Intelligence and Computational Research" in Wikidata, which should call for suspicion

Answer 18

You could use a combination of some simple rules and common sense. Among the simple rules once could list:

• Presence in major databases, e.g. Web of Science, PubMed, etc
• Does anyone in your field cite papers in this journal?
• The quality of other papers in this journal
• Do your colleagues publish in this journal?
• Do they have typos in their webpage?

And common sense could help in a variety of ways. For example, today, I got the following letter, a rather nice one, seems like someone would have written it specially to me:

I hope this email finds you well. I wanted to get in touch with you about a paper you authored entitled "Using principal component scores reduces the effect of socially desirable responding". Firstly, thank you for taking the time to publish this, it was an interesting read. Have you continued working in this area? If you have any other articles or ongoing research I would love to know more.

I am hoping to discuss with you having a short follow-up article or perhaps a review article published in one of the next issues of the journal I serve as an editor for, the Medical Research Archives. I think our readers would be interested in a paper with information from any continued research or new data since this was published. It would be especially helpful if the article could be written for more of a general medicine audience so that many sub specialties could gain from it. The article would not have to be long, and any of your co-authors or colleagues would be welcome to contribute to it. I am happy to assist in any way I can, and there is no hard deadline.

Ok. First of all, they refer to a paper on psychometrics and ask me to publish something on the same topic in "Medical Research Archives". Makes no sense. Second, they tell me "thank you for taking time to publish this" (gracias de nada! this is my job!) and "it was interesting to read" (how on earth would you know - having not read it? and what is the point of calling it "interesting" if you have nothing specific to say about it?). At this point, it it quite clear that they have a script sending these letters, using a database with e-mails and titles of research papers. It is not written the way an actual editor of a bona fide journal would write, so I just delete the e-mail. (It was better written than most such e-mails, that is the reason why I even checked their webpage, - not that I would for a second consider sending a paper to their journal - but just for curiosity about the new tricks of predatory publishers.)

Answer 19

Quality of the journal is reflected in your own experience.

When you've reached the level when you start thinking about publishing in your field of expertise,

and you find out about a journal you've never heard before through several years of undergraduate and postgraduate studies,

it is a good enough reason to be suspicious and cautious.

Answer 20

Many good answers are provided here. Along with the editorial board, publisher, and impact factor, one should check http://www.scimagojr.com/. This site provides ranking for journals, and country. The ranking of journals based on Q1 through Q4 in the specified subjects ensures of the quality of journals in that issue. Q1 journals are the best in that subject and Q2, Q3, and Q4 are decreasing impact of the journals.