Automatic Cross-Repository File Sync with Merge Request Creation

Project Overview

This project automatically syncs files from one GitLab repository (Repo A) to another repository (Repo B) whenever a merge happens.

The system listens for GitLab webhook events, detects changed files, copies them to the target repository, creates a new branch, pushes the code, and automatically creates a Merge Request.

Workflow

Repo A Merge
↓
Webhook triggers app
↓
Detect changed files
↓
Copy files to Repo B
↓
Create branch
↓
Push code
↓
Create Merge Request automatically

Features

Detects GitLab merge events automatically
Syncs only changed files
Preserves folder structure
Creates sync branch automatically
Pushes code to Repo B
Creates Merge Request automatically
Supports excluded paths
Prevents unwanted overwrites
Logging support included

Prerequisites

Before running the project, install the following:

Required Software

Python 3.10+
Git
GitLab Account
ngrok Account

Required Access

You need:

GitLab Personal Access Token
Access to both repositories

Project Structure

repo-sync-automation/
│
├── src/
│   ├── main.py
│   ├── git_manager.py
│   ├── sync_engine.py
│   ├── webhook_handler.py
│   ├── mr_creator.py
│   ├── conflict_handler.py
│   ├── logger.py
│   └── config_loader.py
│
├── config/
│   └── config.yaml
│
├── tests/
│
├── logs/
│
├── requirements.txt
├── docker-compose.yml
├── Dockerfile
└── .env

Clone the Repository

git clone <repo-url>
cd repo-sync-automation

Create Virtual Environment

You can use:

CMD
PowerShell
VS Code Terminal

Windows

python -m venv venv
venv\Scripts\activate

Linux/macOS

python3 -m venv venv
source venv/bin/activate

Install Requirements

pip install -r requirements.txt

Environment Variables Setup

Create a file named .env in the project root folder.

Example:

GITLAB_TOKEN=your_gitlab_token
WEBHOOK_SECRET=your_webhook_secret

How to Generate GitLab Token

Open GitLab
Go to:

Profile → Preferences → Access Tokens

Create token with permissions:

api
read_repository
write_repository

Copy the token
Paste it inside .env

Configure config.yaml

Open:

config/config.yaml

Example configuration:

app:
  host: 0.0.0.0
  port: 8000

source_repo:
  name: repo-a
  url: https://gitlab.com/username/repo-a.git
  branch: main

target_repo:
  name: repo-b
  url: https://gitlab.com/username/repo-b.git
  branch: main

sync:
  working_dir: repos
  sync_branch_prefix: sync

  excluded_paths:
    - .git/
    - node_modules/
    - __pycache__/
    - docs/

merge_request:
  target_branch: main
  title_prefix: "Auto Sync"

logging:
  file: logs/app.log

Run the Application

Open terminal inside the project folder.

Run:

python src/main.py

If using FastAPI + Uvicorn:

uvicorn src.main:app --host 0.0.0.0 --port 8000

ngrok Setup

Why ngrok is Needed

GitLab webhooks cannot access:

localhost:8000

ngrok creates a public URL that forwards requests to your local machine.

Step 1 — Download ngrok

Download from:

https://ngrok.com/

Step 2 — Create Account

Sign up
Login
Copy your auth token

Step 3 — Authenticate ngrok

ngrok config add-authtoken YOUR_TOKEN

Step 4 — Start Tunnel

ngrok http 8000

You will get a URL like:

https://abc123.ngrok-free.app

Webhook URL:

https://abc123.ngrok-free.app/webhook

Copy this URL.

GitLab Webhook Setup

Steps

1. Open Repo A

Go to:

Settings → Webhooks

2. Paste ngrok URL

Example:

https://abc123.ngrok-free.app/webhook

3. Select Trigger

Enable:

Merge Request Events

4. Save Webhook

Click:

Add Webhook

How to Get Project ID of Repo B

Method 1 — GitLab UI

Open Repo B.

On the project homepage, look at the right sidebar.

You will see:

Project ID: 12345678

Copy this ID.

Method 2 — GitLab API

curl --header "PRIVATE-TOKEN: <token>" \
https://gitlab.com/api/v4/projects

Where to Use Project ID

Paste the Project ID inside:

.env
config.yaml
or wherever your MR API logic uses it

How the Workflow Works

Step-by-Step Flow

1. Merge happens in Repo A

Developer merges code into main branch.

↓

2. GitLab webhook triggers Python app

GitLab sends webhook payload.

↓

3. Changed files detected

Application compares commits.

↓

4. Files copied to Repo B

Only modified files are synced.

↓

5. New branch created

Example:

sync/20260515

↓

6. Commit pushed

Application pushes changes automatically.

↓

7. Merge Request created

MR gets created using GitLab API.

Core Modules Explanation

main.py

Creates FastAPI server.

app = FastAPI()

Webhook endpoint:

@app.post("/webhook")

GitLab sends webhook events here.

git_manager.py

Handles Git operations.

Responsibilities

Clone repository
Pull latest code
Create branch
Commit changes
Push branch

Example logic:

if repo exists:
    git pull
else:
    git clone

create_branch()

Creates branch like:

sync/20260515

commit_changes()

Runs:

git add .
git commit

push_branch()

Runs:

git push

GitPython allows Python to control Git.

sync_engine.py

Responsible for file synchronization.

Responsibilities

Copy modified files
Delete removed files
Preserve folder structure

Git Status Types

Status	Meaning
A	Added
M	Modified
D	Deleted

Example:

('M', 'hello.txt')

Means:

Modified file

Uses:

git diff old_commit new_commit

For comparing commits.

Uses:

shutil.copy2()

To preserve:

file content
metadata

mr_creator.py

Creates Merge Request automatically using GitLab API.

Without API:

Manual MR creation needed

create_merge_request()

Sends HTTP POST request:

{
  "source_branch": "sync/123",
  "target_branch": "main",
  "title": "Auto Sync"
}

conflict_handler.py

Prevents overwriting files blindly.

Scenario:

Repo B already changed same file

Automation should avoid overwriting directly.

Merge Detection Improvement

Initial approach:

Webhook SHA payload

Can sometimes be inconsistent.

Better approach:

Local HEAD comparison

OR

GitLab Merge Request Changes API

Comparison Table:

Method	Reliability
Webhook SHA payload	Inconsistent
Local HEAD comparison	Stable

Example Run

Example terminal logs:

[INFO] Webhook received
[INFO] Cloning source repository
[INFO] Detecting changed files
[INFO] Copying updated files
[INFO] Creating sync branch
[INFO] Committing changes
[INFO] Pushing branch
[INFO] Creating Merge Request
[INFO] Sync completed successfully

Troubleshooting

ngrok URL not working

Fix

Ensure ngrok is running
Verify correct port

Example:

ngrok http 8000

Invalid GitLab Token

Fix

Verify token permissions:

api
read_repository
write_repository

Webhook not triggering

Fix

Check:

GitLab → Settings → Webhooks

Verify:

Correct URL
Merge Request Events enabled

Git Permission Denied

Fix

Reconfigure Git credentials.

Check:

git config --global user.name
git config --global user.email

Merge Request not creating

Fix

Check:

Project ID
Token permissions
API endpoint

Useful Commands

Activate Virtual Environment

Windows

venv\Scripts\activate

Linux/macOS

source venv/bin/activate

Install Requirements

pip install -r requirements.txt

Run Application

python src/main.py

Run FastAPI with Uvicorn

uvicorn src.main:app --host 0.0.0.0 --port 8000

Start ngrok

ngrok http 8000

Running with Docker

Build Docker Image

docker build -t git-sync-automation .

Run Docker Container

docker run -p 8000:8000 git-sync-automation

Application will run on:

http://localhost:8000

Docker Compose

Start Containers

docker-compose up --build

Stop Containers

docker-compose down

Notes

Before running Docker:

Ensure .env file exists
Ensure config/config.yaml is configured properly
Docker Desktop should be running

Future Improvements

Docker deployment
Kubernetes support
Multiple repo sync support
Better conflict resolution
UI dashboard
Retry queue system
Slack notifications

Author

Developed for automated GitLab repository synchronization and Merge Request creation.

Designed for DevOps automation, CI/CD workflows, and repository management.

Name		Name	Last commit message	Last commit date
Latest commit History 3 Commits
.github/workflows		.github/workflows
config		config
src		src
tests		tests
.env.example		.env.example
.gitignore		.gitignore
.gitlab-ci.yml		.gitlab-ci.yml
Dockerfile		Dockerfile
README.md		README.md
docker-compose.yml		docker-compose.yml
requirements.txt		requirements.txt

Folders and files

Latest commit

History

Repository files navigation

Automatic Cross-Repository File Sync with Merge Request Creation

Project Overview

Workflow

Features

Prerequisites

Required Software

Required Access

Project Structure

Clone the Repository

Create Virtual Environment

Windows

Linux/macOS

Install Requirements

Environment Variables Setup

How to Generate GitLab Token

Configure config.yaml

Run the Application

ngrok Setup

Why ngrok is Needed

Step 1 — Download ngrok

Step 2 — Create Account

Step 3 — Authenticate ngrok

Step 4 — Start Tunnel

GitLab Webhook Setup

Steps

1. Open Repo A

2. Paste ngrok URL

3. Select Trigger

4. Save Webhook

How to Get Project ID of Repo B

Method 1 — GitLab UI

Method 2 — GitLab API

Where to Use Project ID

How the Workflow Works

Step-by-Step Flow

1. Merge happens in Repo A

2. GitLab webhook triggers Python app

3. Changed files detected

4. Files copied to Repo B

5. New branch created

6. Commit pushed

7. Merge Request created

Core Modules Explanation

main.py

git_manager.py

Responsibilities

create_branch()

commit_changes()

push_branch()

sync_engine.py

Responsibilities

Git Status Types

mr_creator.py

create_merge_request()

conflict_handler.py

Merge Detection Improvement

Example Run

Troubleshooting

ngrok URL not working

Fix

Invalid GitLab Token

Fix

Webhook not triggering

Fix

Git Permission Denied

Fix

Merge Request not creating

Fix

Useful Commands

Activate Virtual Environment

Windows

Linux/macOS

Install Requirements

Run Application

Run FastAPI with Uvicorn

Start ngrok

Packages