Skip to content

NaiveInvestigator/bracu_website_scrapper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

FastAPI + MySQL Setup

This project uses FastAPI, MySQL, and supporting Python libraries for scraping, data processing, and API development.


📦 Installation

Install the required Python packages:

pip install bs4 cloudscraper lxml mysql-connector-python pdfplumber "fastapi[standard-no-fastapi-cloud-cli]" uvicorn sqlalchemy pydantic

Optional (in case mysql-connector doesn’t work):

pip install pymysql

Note: Installing pdfplumber will also install pdfminer.six automatically. No need to install them seperately.


🚀 Running the API

Run the main API (main.py) using FastAPI:

fastapi dev api.py

🗄️ Useful SQL Commands

SHOW TABLES;
DROP TABLE People;
SELECT * FROM People;
SELECT url, image_url FROM People;

🐳 MySQL Docker Setup

Follow CVAT’s Docker setup guide if you are on ubuntu 22. Or check the Docker official site Or use Dockers Convenience Install Script

Start a MySQL container:

docker run --name mysql-server -e MYSQL_ALLOW_EMPTY_PASSWORD=yes -p 3306:3306 -d mysql:oraclelinux9

Enter the container:

docker exec -it mysql-server /bin/bash

Access MySQL:

mysql -u root

🔄 Restarting MySQL After Reboot

If the container is stopped or the system reboots, you don’t need to run the full command again. Just restart the container:

docker start mysql-server

About

A bracu info scraper that gives structured JSON data from bracu.ac.bd

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors