get_citation_count() only grabbing results from first page, not all results

[kweav]

only getting the first page of data rather than all the data. Going to need rvest read_html to loop through all pages

[kweav]

Look at this medium guide: https://dimitryzub.medium.com/scrape-google-scholar-in-r-d521cfe0e8d

[kweav]

And how Ava has handled pagination for querying the anvil collection: https://github.com/fhdsl/AnVIL_Collection/blob/main/scripts/query_collection.R

[kweav]

Another source of inspiration: https://cran.r-project.org/web/packages/gcite/gcite.pdf, https://github.com/muschellij2/gcite/tree/master/R

[kweav]

Pivoting to using semantics scholar and crossref which have APIs. Especially with needing to look at more pages of results, we get blocked more. And this way we're using APIs, rather than web scraping a platform that doesn't want us to.