VEIL: Video Face Exchange with Identity Locking

VEIL is a target-aware video face-swap pipeline that preserves selected target identities and replaces non-target faces in video using face detection, tracking, recognition, and InSwapper.

What VEIL Does

VEIL processes an input video frame by frame. It detects faces with YOLO, tracks them with BoT-SORT, identifies protected target people with InsightFace/ArcFace embeddings, and swaps only non-target faces with a configured virtual face. Target identities are left unchanged.

The pipeline also writes per-frame tracking metadata, face metadata, logs, and an annotated output video. When a non-target face is too small, occluded, or otherwise unsuitable for swapping, VEIL falls back to blur instead of forcing a low-quality swap.

Demo

Source video: Magnific

Virtual face source: This Person Does Not Exist

Original Input

Virtual Face

VEIL Output

Project Layout

veil/
├── README.md
├── media/
│   ├── input_test.gif
│   ├── fake_face.jpg
│   └── output_target5.gif
├── scripts/
│   └── download_weights.sh
└── models/
    ├── requirements.txt
    ├── boxmot/
    │   └── models/
    └── veil/
        ├── main_hybrid.py
        ├── config.py
        ├── detector_tracker.py
        ├── face_identifier.py
        ├── face_inswapper.py
        ├── face_utils.py
        ├── metadata_manager.py
        ├── crop_manager.py
        ├── target/
        ├── virtual_face/
        ├── videos/
        ├── weights/
        └── outputs/

Requirements

• Python 3.10 or newer is recommended.
• Python dependencies are managed in models/requirements.txt.
• NVIDIA GPU + CUDA are recommended for practical runtime.
• CPU execution is possible by setting device = "cpu" in models/veil/config.py, but it will be significantly slower.
• models/requirements.txt includes ONNX Runtime GPU, CUDA, and TensorRT-related packages. VEIL uses the available provider stack and falls back when a provider is unavailable.
• curl or wget is required for scripts/download_weights.sh.

Setup

git clone https://github.com/jiminbae/veil
cd veil
python -m venv .venv
source .venv/bin/activate
pip install -r models/requirements.txt
bash scripts/download_weights.sh

download_weights.sh downloads the required model files into models/veil/weights/:

• yolo26x-face.pt
• inswapper_128.onnx

The BoT-SORT ReID weight is stored at models/boxmot/models/osnet_x0_25_msmt17.pt.

Inputs

Place your files in these paths before running the pipeline:

• Input video: models/veil/videos/test.mp4
• Protected target images: models/veil/target/target*
• Replacement face image: models/veil/virtual_face/fake_face.jpg

Target images can be .jpg, .jpeg, .png, .bmp, or .webp. You can provide multiple target images as long as their filenames match target*.

Use clear target images where the protected person's face is visible and easy to detect. The replacement face image should also contain one clear face, preferably front-facing or near front-facing.

You can change these paths and thresholds in models/veil/config.py.

Run

cd models/veil
python main_hybrid.py

Each run automatically chooses the next output index by scanning the existing files under outputs/.

Outputs

VEIL writes generated files under models/veil/outputs/:

• Annotated result video: outputs/result/output_target{N}.mp4
• Runtime log: outputs/log/tracking_target{N}_log.txt
• Face metadata: outputs/metadata/face_metadata{N}.json
• Tracking metadata: outputs/metadata/tracking_metadata{N}.json

The result video includes face boxes and labels. The metadata files include frame numbers, raw track IDs, stable face IDs, bounding boxes, target match status, similarity scores, quality labels, and fallback reasons.

Configuration

The main settings live in models/veil/config.py:

• VIDEO_PATH: input video path.
• TARGET_DIR and TARGET_PATTERN: protected identity image gallery.
• TARGET_IMAGE_PATH: replacement face image used by InSwapper.
• ENABLE_FACE_SWAP: turn face swapping on or off.
• MAX_SWAP_FACES_PER_FRAME: cap the number of non-target faces swapped per frame.
• SIM_THRESHOLD and TARGET_THRESHOLD: identity matching thresholds.
• FACE_SWAP_BATCH_SIZE: batch size used by the swap stage.
• USE_BOTSORT_REID: enable or disable BoT-SORT ReID features.
• DETECTION_CONF: face detection confidence threshold.
• SWAP_MIN_FACE_SIZE and SWAP_MAX_FACE_AREA_RATIO: decide when a detected face is suitable for swapping.
• ENABLE_MOUTH_PASTE: keeps the original mouth area when blending a swapped face.

Pipeline Summary

1. Detect faces using YOLO face weights.
2. Track face boxes across frames with BoT-SORT.
3. Extract face embeddings with InsightFace/ArcFace.
4. Match tracked faces against target images.
5. Preserve target identities.
6. Swap eligible non-target faces using inswapper_128.onnx.
7. Apply blur fallback when swap quality would be unreliable.
8. Save output video, logs, and metadata.

Troubleshooting

• No target images found: add at least one image whose filename starts with target under models/veil/target/.
• No valid target embeddings loaded: the target image exists, but no face was detected. Try a clearer face image.
• Cannot open video: check that the input video exists at models/veil/videos/test.mp4, or update VIDEO_PATH in config.py.
• Cannot read target image: check models/veil/virtual_face/fake_face.jpg, or update TARGET_IMAGE_PATH in config.py.

Notes

• GPU execution is enabled by default through device = "cuda" in config.py.
• ONNX Runtime uses the available provider stack, including TensorRT/CUDA when installed and available.
• Downloaded model weights, local input assets, outputs, logs, and metadata are ignored by git.
• Use VEIL only with appropriate consent and for lawful, responsible video processing.

Developers

• jiminbae
• TELLEGIT
• creatingweb123
• jin23bsj