Web scraping etiquette

[sarahmcclymont]

Might want to add a quick little bit about when web scraping is/is not appropriate and how to tell which is the case (eg: robots.txt and what that is). This is a pretty good blogpost that could be linked in the quick description: https://benbernardblog.com/web-scraping-and-crawling-are-perfectly-legal-right/

(Apologies if this is discussed somewhere and I just haven't come across it!)