12

I self-archive my papers on a personal homepage hosted using GitHub Pages (sulir.github.io). Some time after the page was published, I had my papers properly indexed and displayed in Google Scholar. Next to each article, there was a link named [PDF] pointing to the file hosted on my site, or after clicking "All versions", I could see a link to the PDF on my website.

A few months ago, I noticed that all these mentioned PDF links disappeared. Furthermore, papers which were published only on my website disappeared completely from the search results.

Note that this problem is not specific to my site. I tried to search for other researchers' papers hosted on GitHub/GitLab and found out that Google Scholar do not display PDF links for them. I suppose it affects all files hosted on GitHub Pages and GitLab Pages (or at least that not using a custom domain).

To support this claim: I know the "site:" operator on Google Scholar is quite limited (only primary versions are returned), but anyway:
filetype:pdf site:github.io - 0 results
filetype:pdf site:gitlab.io - 0 results

Is Google actively blocking such platforms? Or is it related to technical details how these pages are served? I also tried contacting scholar-support@google.com, but I did not get any response.

Edit: I also checked for a robots.txt file - it is not present. Furthermore, traditional Google search can find these files, so this problem is specific to Google Scholar.

Edit 2: Suppose Google Scholar intentionally excluded the whole github.io and gitlab.io domains from indexing. What options do I have to have my PDF files indexed?

closed as off-topic by Fomite, user3209815, user2390246, Bob Brown, Florian D'Souza Dec 11 '17 at 17:48

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "The answer to this question strongly depends on individual factors such as a certain person’s preferences, a given institution’s regulations, the exact contents of your work or your personal values. Thus only someone familiar can answer this question and it cannot be generalised to apply to others. (See this discussion for more info.)" – Fomite, user3209815, user2390246, Bob Brown, Florian D'Souza
If this question can be reworded to fit the rules in the help center, please edit the question.

  • 2
    Presumably Google Scholar no longer considers GitHub a reliable source for research papers, which seems reasonable, albeit annoying for you. Maybe Google made this decision because its web crawlers were generating erroneous citations from GitHub. Perhaps you can host your papers on an institutional website. – user2768 Dec 11 '17 at 15:34
  • This question has now been marked off-topic. I presume this is because only Google can provide a definitive answer. (GitHub might be able to speculate on the latter.) Maybe the OP can revise the question so that it is on-topic. – user2768 Dec 11 '17 at 18:06
  • I added a reformulated question. – Matúš Sulír Dec 12 '17 at 13:21
  • @user2768 like when it was credible? – SSimon Dec 12 '17 at 13:33
  • @SSimon My use of "reliable source" is perhaps misleading; I don't know whether Google ever considered GitHub reliable. Its perhaps better to write: Presumably Google Scholar has blacklisted GitHub as a source for research papers. – user2768 Dec 12 '17 at 14:12