By Helena Korjonen, PhD
Postdoctoral Researcher, Sustainable Food Practices, University of Luxembourg
Google Scholar (GS) is the top search engine used by those who are looking for scholarly content1. There are many reasons for this, not least that it feels familiar 2, 3. However, a review of the literature reveals that there are significant limitations to its effectiveness.
Lack of curation
The internet is full of ‘scholarly’ content, information, and data in various formats and of various quality. As John Naisbitt said, “We are drowning in information, but starved for knowledge”
When Google Scholar trawls the internet to seek out scholarly content, it does not assess it. It does not try to make sense of it for researchers in a logical indexed structure. And, unlike many trusted curated tools4, Google Scholar does not have transparent indexing guidelines that define what they collect, and why they collect it and make it available.
Recent research shows that Google Scholar is “unsuitable as a primary review for resources”5. Other research argues that Google Scholar has flaws and lacks technological features to enable systematic searching. As a result, it fails to adhere to several FAIR principles (Findable, Accessible, Interoperable and Reusable) and cannot be classed as a professional searching tool6.
Google Scholar is not a curated controlled database. Content is harvested automatically.
Whilst there are some advantages to this, for example in the retrieval of grey literature7, there are also significant disadvantages.
Search results rankings
Research has shown that Google Scholar results can be dated. This presents an issue when most people don’t go beyond the first page of results when searching for papers.
The citation count of an article is a major factor in the Google Scholar results ranking8, 9. This benefits publications from high Impact Factor sources, which are cited or linked to extensively around the Internet.
Unfortunately, new papers have what’s called a citation lag, and older papers are cited more often10. Newly published papers may be retrieved by GS, but they will be further down the results list and you may not see them.
Papers can also be manipulated into a higher rank than it ‘deserves’ by placing words in titles or abstracts to increase recall or manipulate citing data3.
Should you need to include multi-lingual items in your search, there is a definite bias in Google Scholar towards English-language content. Documents published in languages other than English are relegated to positions that make them virtually invisible8.
Replicating your search
It is nearly impossible to replicate a search in GS, to retrieve the same references again. For this reason, it’s recommended to carefully document a search done in Google Scholar for full transparency6. That can be extremely time consuming.
References are often dropped unknowingly from GS5, 11, 12 and you won’t even know the ones that might be missing1.
Using your results
In Google Scholar, once you have the results, it does not allow you to see beyond 1,000 records or to download more than 20 references to your bibliographic tool at a time. Most curated databases allow you to run your search, save it, download the results to your tool and revisit it.
How to search more effectively
IFIS has published a guide called Best Practice for Literature Searching. This provides guidance on where to search; how to formulate research questions and evaluate results. Click here to access the guide
- Gardner T, Inger S. How readers discover content: Trends in reader behaviour from 2005 to 2021. Renew Consultants. 2021; 17. Available from: https://renewconsultants.com/wp-content/uploads/2021/07/How-Readers-Discover-Content-2021.pdf
- Hemminger BM, Lu D, Vaughan KTL, Adams SJ. Information seeking behavior of academic scientists. Journal of the American Society for Information Science and Technology. 2007; 58(14): 2205-2225. Available from: https://doi.org/10.1002/asi.20686
- Georgas H. Google vs. the library: Student preferences and perceptions when doing research using Google and a federated search tool. portal: Libraries and the Academy. 2013; 13(2): 165-185.Available from: https://doi.org/10.1353/pla.2013.0011
- Halevi G, Moed H, Bar-Ilan J. "Suitability of Google Scholar as a source of scientific information and as a source of data for scientific evaluation—Review of the literature. ." Journal of Informetrics. 2017; 11(3): 823-834. Available from: https://doi.org/10.1016/j.joi.2017.06.005
- Gusenbauer M, Haddaway NR. "Which academic search systems are suitable for systematic reviews or meta‐analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources." Research Synthesis Methods 2020; 11(2): 181-217. Available from: https://doi.org/10.1002/jrsm.1378
- Boeker M, Vach W, Motschall E. Google Scholar as replacement for systematic literature searches: Good relative recall and precision are not enough. BMC Medical Research Methodology. 2013; 13(1):1-2. Available from: https://doi.org/10.1186/1471-2288-13-131
- Haddaway NR, Collins AM, Coughlin D, Kirk S. The role of Google Scholar in evidence reviews and its applicability to grey literature searching. PloS One. 2015; 17;10(9): e0138237. Available from: https://doi.org/10.1371/journal.pone.0138237
- Beel J, Gipp B. Google Scholar's ranking algorithm: The impact of citation counts (an empirical study). Proceedings of the 2009 3rd International Conference on Research Challenges in Information Science, RCIS 2009; 439–446. Available from: http://doi.org/10.1109/RCIS.2009.5089308
- Rovira C, Codina L, Guerrero-Solé F, Lopezosa C. Ranking by relevance and citation counts, a comparative study: Google Scholar, Microsoft Academic, WoS and Scopus. Future Internet. 2019 Sep;11(9):202. Available from: https://doi.org/10.3390/fi11090202
- Khare R, Leaman R, Lu Z. Accessing biomedical literature in the current information landscape. Biomedical Literature Mining. 2014; 11-31. Available from: https://doi.org/10.1007/978-1-4939-0709-0_2
- Nicholas D, Boukacem‐Zeghmouri C, Rodríguez‐Bravo B, Xu J, Watkinson A, Abrizah A, Herman E, Świgoń M. Where and how early career researchers find scholarly information. Learned Publishing. 2017; 30(1): 19-29. Available from: https://doi.org/10.1002/leap.1087.
- Jamali HR, Asadi S. Google and the scholar: The role of Google in scientists' information‐seeking behaviour. Online Information Review. 2010 Apr 20.. Available from: https://doi.org/10.1108/14684521011036990.