Treffer: Cheap, Rigorous, and Transparent: How Web-scraping with Python can Improve Collecting Grey Literature for Systematic Literature Reviews.

Gathering non-conventional literature, such as grey literature, from web-based sources for use in a systematic literature review is at times an arduous task. Often, the processes used to do so are difficult for other researchers to repeat. Compounding this issue is the cost that researchers bear, in either paying for desktop-based applications, or paying external researchers who have programming experience to design applications and tools for them. To address these issues, this article presents a methodology for researchers to systematically gather grey literature from online repositories using the computer programming language Python. Utilising a well-known data extraction technique (Web-Scraping), this article exhibits the code used to scrape policy documents from the International Energy Agency's online policy database. A flowchart for the different stages of this process is also introduced to aid in addressing the technical, legal, and ethical elements of web-scraping that researchers must also be aware of when undertaking this approach. Finally, as a proof of concept, the results from the method are also presented. [ABSTRACT FROM AUTHOR]