DiVE

This repository contains the official implementation of our paper "Diffusion-based Synthetic Data Generation for Visible-Infrared Person Re-Identification" accepted at AAAI 2025.

⚠️ Important Notice:
The weight file for generating infrared images has been uploaded and you can download it from the link in section 2.Finetune the SD model. If you want to use our weight to generate images, you should replace the huggingface data's dataset_info.json with the file with same name in dir Preprocess. Thie will make sure the camera description is consistent because the c2hf_revised.py generates rendom description each time.

Environment Setup

1. Clone this repository:

   git clone https://github.com/BorgDiven/DiVE.git
   cd DiVE

2. Create a conda environment using the provided environment.yml file:

   Please follow versions pinned in the environment.yml file! (My cuda version is 12.1.)

   conda env create -f environment.yml

3. Activate your newly created environment:

   conda activate <env_name>

Preprocess

1. Download the SYSU and Market-1501 datasets

2. Run market2sysu_revised.py (cd Preprocess) Convert the Market dataset into SYSU format (Market as one camera) and add Market into the SYSU dataset:

   python market2sysu_revised.py \
       --market_gt_bbox_dir /path/to/market_data/Market-1501-v15.09.15/gt_bbox \
       --sysu_dir /path/to/sysu_original_data \
       --max_sysu_id 533 \
       --min_images_per_id 12

   Parameters to modify:

   • market_gt_bbox_dir: gt_bbox directory in the Market-1501 dataset
   • sysu_dir: Path to the SYSU-MM01 dataset

   After running this, the Market data will be added to the SYSU data folder.

3. Run c2hf_revised.py (cd Preprocess) Convert the SYSU+Market dataset into a format ready for training:

   python c2hf_revised.py \
       --dataset sysu \
       --base_path /path/to/sysu_market \
       --output_path /path/to/sysu_market_huggingface \
       --num_cameras 7

   Parameters to modify:

   • base_path: Path of the SYSU+Market data
   • output_path: Path of the processed SYSU+Market data, compatible with Diffuser training

Data Generation

1. Download SD-1.5 model Using Hugging Face CLI

   Install the Hugging Face Hub CLI:

   pip install huggingface_hub

   Log in to Hugging Face:

   huggingface-cli login

   Go to https://huggingface.co/settings/tokens, generate a new personal access token, and use it when prompted by huggingface-cli login.

   Download a model:

   huggingface-cli download stable-diffusion-v1-5/stable-diffusion-v1-5 --local-dir ./stable-diffusion-v1-5
   Attention: add the "--local-dir-use-symlinks False" to directly download to specific path

   If you cannot access Huggingface, you can try using the mirror website https://hf-mirror.com/ (Please be aware of copyright issues!)

Change directory to: cd Data_Generation

2. Finetune the SD model (cd Data_Generation) Train the LoRA branch and textual embedding.

   cd Data_Generation
   source scripts/finetune.sh 
   export GPU_IDS=(0)
   finetune "/path/to/processed_data" "/path/to/sd_model"

   Parameters to customize:

   • "/path/to/processed_data": data generated by c2hf_revised.py
   • "/path/to/sd_model": the local SD model weights (e.g., /path/to/stable-diffusion-v1-5/stable-diffusion-v1-5)

   You could download the LoRA branch and textual embedding weight files:

   Netdisk link: https://pan.baidu.com/s/1OgCR4_vEtJHRALEOIxM7Lw?pwd=1c7c Password: 1c7c

   ⚠️ If you want to use our weight to generate images, you should replace the huggingface data's dataset_info.json with the file with same name in dir Preprocess. Thie will make sure the camera description is consistent because the c2hf_revised.py generates rendom description each time.

3. Generate the data (cd Data_Generation) Generate the infrared synthetic data.

   source scripts/sample.sh 
   export GPU_IDS=(0)
   sample "/path/to/processed_data" "/path/to/sd_model"

   Parameters to customize:

   • "/path/to/processed_data": data generated by c2hf_revised.py
   • "/path/to/sd_model": the local SD model weights (e.g., /path/to/stable-diffusion-v1-5/stable-diffusion-v1-5)

4. Add the generated data into SYSU-Market data (cd Preprocess)

   python move_synthetic_data.py -i /path/to/sysu_market_data -o /path/to/synthetic_data
   **/path/to/sysu_market_data must refer to the SYSU dataset folder integrated by market2sysu_revised.py Don't set to a new SYSU dataset folder**

Re-ID Training

Train SYSU data with DEEN method

1. Pre-process the dataset

    cd Re-ID_model/DEEN
    python pre_process_sysu.py

2. Train the Re-ID model

   Set the sh file:

    DATA_PATH="path/to/synthetic_sysu/" #add / at the end ! ! !

   Run training:

    sh train_sysu.sh

3. Test the Re-ID model

   Set the sh file:

    DATA_PATH="path/to/synthetic_sysu/" #add / at the end ! ! !
    RESUME="checkpoint_name" #sysu_deen_p4_n16_lr_0.1_seed_0_best.t
    N_CLASS="1406" # 395(sysu ids) + 1011(selected market ids, check it at synthetic_data/exp/id_mapping.json) 

   Run testing:

    sh test_sysu.sh