Skip to content

README.md

Latest commit

 

History

History
156 lines (115 loc) · 4.73 KB

README.md

File metadata and controls

156 lines (115 loc) · 4.73 KB

LinkedIn Scraper

A Python-based LinkedIn profile scraper that uses Selenium to extract profile information including name, work experience, and education details.

Prerequisites

Before running this project, make sure you have the following installed:

  1. Python 3.7+
    Download from python.org

  2. Google Chrome Browser
    Download from google.com/chrome

  3. ChromeDriver

    • Download ChromeDriver that matches your Chrome browser version from chromedriver.chromium.org
    • Place chromedriver.exe in the project root directory

Installation

  1. Clone the repository

    git clone https://github.com/AKProgramer/Linkedln-Scraper.git
cd Linkedln-Scraper

  2. Install required Python packages

    pip install -r requirements.txt

Configuration

Set LinkedIn Credentials

You need to provide your LinkedIn credentials to run the scraper. Use one of the following methods:

Option 1: Environment Variables (Recommended)

Windows (PowerShell):

$env:LINKEDIN_USER="your_email@example.com"
$env:LINKEDIN_PASSWORD="your_password"

Linux/Mac:

export LINKEDIN_USER="your_email@example.com"
export LINKEDIN_PASSWORD="your_password"

Option 2: System Environment Variables (Permanent)

Windows:

  1. Search for "Environment Variables" in Windows search
  2. Click "Edit the system environment variables"
  3. Click "Environment Variables" button
  4. Under "User variables", click "New" and add:
    • Variable name: LINKEDIN_USER
    • Variable value: your_email@example.com
  5. Add another variable:
    • Variable name: LINKEDIN_PASSWORD
    • Variable value: your_password
  6. Restart your terminal after setting these

Usage

Running the Sample Script

  1. Make sure you've set your LinkedIn credentials (see Configuration section above)

  2. Run the scraper:

    python -m samples.scrape_person

  3. The script will:

    • Open Chrome browser
    • Log into LinkedIn with your credentials
    • Navigate to the specified profile
    • Extract and display profile information

Sample Output

The script outputs the person's profile information including:

  • Name
  • Work experiences (position, company, duration, location, description)
  • Education (institution, degree)

Customizing the Profile URL

Edit samples/scrape_person.py and change the LinkedIn URL on this line:

person = Person("https://www.linkedin.com/in/your-profile-url", driver=driver)

Project Structure

linkedin_scraper/
├── linkedin_scraper/      # Main scraper package
│   ├── __init__.py
│   ├── person.py         # Person profile scraper
│   ├── company.py        # Company scraper
│   ├── actions.py        # Login and navigation actions
│   └── selectors.py      # CSS selectors for scraping
├── samples/              # Sample scripts
│   ├── scrape_person.py
│   └── scrape_person_contacts.py
├── chromedriver.exe      # ChromeDriver (you need to download this)
├── requirements.txt      # Python dependencies
└── README.md            # This file

Troubleshooting

Common Issues

  1. "Python was not found"

    • Install Python from python.org
    • During installation, check "Add Python to PATH"

  2. "ModuleNotFoundError: No module named 'linkedin_scraper'"

    • Run the script using: python -m samples.scrape_person
    • Or install the package: pip install -e .

  3. "Unable to obtain driver for chrome"

    • Download ChromeDriver from chromedriver.chromium.org
    • Place chromedriver.exe in the project root directory
    • Make sure the ChromeDriver version matches your Chrome browser version

  4. "ConnectionResetError" or login issues

    • LinkedIn may block automated logins
    • Try using a VPN or different network
    • Ensure your credentials are correct
    • LinkedIn may require CAPTCHA verification

  5. AttributeError for certain attributes

    • Some profile attributes may not be available for all users
    • The script handles missing attributes gracefully

Important Notes

⚠️ Legal and Ethical Considerations:

  • Web scraping LinkedIn may violate their Terms of Service
  • Use this tool responsibly and only for permitted purposes
  • Consider LinkedIn's rate limits and avoid excessive requests
  • Respect user privacy and data protection regulations

License

See the LICENSE file for details.

Support

For issues or questions, please open an issue on the GitHub repository.