Lockbit scraper fixed (now uses playwright) #74

[biligonzales]

Describe the changes

Lockbit 2.0 now uses a ddos protection mechanism hence the regular http get method is no longer working.

As a workaround I have implemented the playwright Microsoft library which behaves as if a proper browser did the request.

Summary of the changes:

1. lockbit.py: replaced the use of requests by playwright
2. requirements.txt: added playwright
3. Dockerfile: added playwright chromium support as well as required libraries.

I have also upgraded at the top of the Dockerfile from python3.9-buster to python3.10-bullseye.

Related issue(s)

It fixes Issue #74

Note that the scraping engine for lockbit has been left untouched as it is still perfectly working. Only the web page retrieval method has been altered.

How was it tested?

• docker-compose build app
• docker-compose up --abort-on-container-exit
• Checked that Lockbit entries have been inserted into the database

[captainGeech42]

Hey @biligonzales, thanks for working on this.

I've got a large refactor/fix collection from an anonymous contributor who maintains a private fork that I'll be merging in this weekend (just waiting to hear back from them on something), that I believe covers this and your other PR.

Once I get that merged in and assess what is still an issue, I will let you know here.

Regardless, I appreciate the PRs, thank you! Will be in touch.

[ocbrollingpaper]

@captainGeech42 is this coming or nah? Asking because I have fully refactored scrapers and rest of the stuff to be ran in real-time without need for cronjobs

EDIT:
I also have go version

[captainGeech42]

@ocbrollingpaper I guess not (was waiting to receive it from 3p, never came), feel free to PR those. Thank you!