A JavaScript-based web scraping tool powered by Puppeteer. This tool extracts news articles from popular Nepali news sources, allowing users to retrieve the latest articles across categories such as finance, sports, politics, art & culture, and the valley. Users can also scrape individual articles by providing the article URL from supported sites.
- Retrieve news articles by category from supported sites.
- Scrape individual articles with a specific URL.
- Data includes metadata such as title, subtitle, author, publication date, and content.
The following sources are in the pipeline for support:
- Onlinekhabar English
- Nepali Times
- The Himalayan Times
- MyRepublica
- New Spotlight Magazine
- Nepal Press
- Enewspolar
- Khabarhub
- Clone the repository:
git clone https://github.com/Aayush-khadka/Nepali-News-Scraper.git
- Navigate to the project directory:
cd Nepali-News-Scraper - Install the required dependencies:
npm install
The following is a sample of the structured data output from a scraped article:
{
"title": "Article Title",
"subtitle": "Article Subtitle",
"authorName": "Author Name",
"authorProfileLink": "https://kathmandupost.com/authors/author-name",
"articleLink": "https://kathmandupost.com/politics/2023/10/10/example-article",
"articleImage": "https://kathmandupost.com/images/example-image.jpg",
"publishedTime": "2023-10-10T12:00:00",
"updatedTime": "2023-10-10T14:00:00",
"updatedPlace": "Kathmandu",
"articleText": "Full text of the article...",
"articleType": "category"
}This project is developed for educational purposes only. The developer is not responsible for any misuse of the tool. Please adhere to the terms of service of the websites being scraped and use the tool responsibly.
For any issues or feedback, feel free to reach out:
- GitHub Issues: Submit Issues
- Email: khadkaaayush90@gmail.com