Skip to content

Improve documentation #4445

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ildyria merged 1 commit into from
Jun 22, 2026
Merged

Improve documentation #4445

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
179 changes: 141 additions & 38 deletions docs/specs/2-how-to/configure-facial-recognition.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
# How-To: Configure Facial Recognition (AI Vision)

**Author:** Lychee Team
**Last Updated:** 2026-03-22
**Last Updated:** 2026-06-22
**Feature:** 030-ai-vision-service
**Related:** [Feature 030 Spec](../4-architecture/features/030-ai-vision-service/spec.md)

## Overview

Lychee's facial recognition feature is powered by a sidecar Python service (`ai-vision-service`). When enabled, Lychee detects faces in photos, groups them into Person profiles, and lets users claim their own profile. This guide covers:
Lychee's facial recognition feature is powered by a sidecar Python service ([`lychee-facial-recognition`](https://github.com/LycheeOrg/Lychee-Facial-Recognition)). When enabled, Lychee detects faces in photos, clusters them, groups them into Person profiles, and lets users claim their own profile. This guide covers:

1. [Prerequisites](#prerequisites)
2. [Docker Compose setup](#docker-compose-setup)
Expand All @@ -16,47 +16,57 @@ Lychee's facial recognition feature is powered by a sidecar Python service (`ai-
5. [Enabling the feature in Lychee admin](#enabling-the-feature-in-lychee-admin)
6. [Permission modes](#permission-modes)
7. [Running a bulk scan](#running-a-bulk-scan)
8. [Service health check](#service-health-check)
9. [Troubleshooting](#troubleshooting)
8. [Clustering](#clustering)
9. [Maintenance operations](#maintenance-operations)
10. [Service health check](#service-health-check)
11. [Troubleshooting](#troubleshooting)

---

## Prerequisites

- Docker and Docker Compose v2
- A working Lychee deployment (see [docker-compose.minimal.yaml](../../../docker-compose.minimal.yaml))
- A **Supporter Edition (SE)** licence — AI Vision is an SE-only feature
- A working Lychee deployment (see [docker-compose.yaml](../../../docker-compose.yaml))
- The `lychee_worker` container running (face scans are processed through the queue)

---

## Docker Compose Setup

Add the `ai_vision` service to your `docker-compose.yaml`. The complete minimal example is in [docker-compose.minimal.yaml](../../../docker-compose.minimal.yaml). The key stanza:
Add the `lychee_facial_recognition` service to your `docker-compose.yaml`. The complete example is in [docker-compose.yaml](../../../docker-compose.yaml). The key stanza:

```yaml
services:
lychee_api:
# ... existing config ...
environment:
AI_VISION_ENABLED: "${AI_VISION_ENABLED:-true}"
AI_VISION_FACE_API_KEY: "${AI_VISION_API_KEY:-changeme}"
AI_VISION_FACE_URL: "http://lychee_facial_recognition:8000"
volumes:
- ./lychee/uploads:/app/public/uploads # Lychee upload directory

ai_vision:
build:
context: ./ai-vision-service # Build from source, OR use a pre-built image
container_name: lychee-ai-vision
- ./lychee/uploads:/app/public/uploads

lychee_facial_recognition:
expose:
- "${APP_PORT_AI_FACE:-8001}"
ports:
- "${APP_PORT_AI_FACE:-8001}:8000"
image: ghcr.io/lycheeorg/lychee-facial-recognition:latest
restart: unless-stopped
security_opt:
- no-new-privileges:true
cap_drop:
- ALL
environment:
VISION_FACE_LYCHEE_API_URL: "http://lychee_api:8000"
VISION_FACE_API_KEY: "${AI_VISION_API_KEY}"
VISION_FACE_API_KEY: "${AI_VISION_API_KEY:-changeme}"
VISION_FACE_VERIFY_SSL: "${AI_VISION_VERIFY_SSL:-true}"
VISION_FACE_PHOTOS_PATH: "/data/photos"
VISION_FACE_STORAGE_BACKEND: sqlite
VISION_FACE_STORAGE_PATH: "/data/embeddings"
volumes:
- ./lychee/uploads:/data/photos:ro # Shared read-only photos volume
- ai_vision_embeddings:/data/embeddings # Persistent embeddings store
- ./lychee/uploads:/data/photos:ro
- ai_vision_embeddings:/data/embeddings
networks:
- lychee
depends_on:
Expand All @@ -79,15 +89,15 @@ volumes:

## Shared Volume Configuration

The AI Vision service reads photo files directly from the filesystem — no HTTP file transfer. This requires both containers to mount the same upload directory:
The facial recognition service reads photo files directly from the filesystem — no HTTP file transfer. Both containers must mount the same upload directory:

| Container | Mount path | Mode |
|---|---|---|
| `lychee_api` | `/app/public/uploads` | read/write |
| `ai_vision` | `/data/photos` | read-only |
| `lychee_facial_recognition` | `/data/photos` | read-only |
| Host bind mount | `./lychee/uploads` | (source for both) |

**Critical:** The host path `./lychee/uploads` must be identical for both mounts. If you use an absolute path or a named volume for `lychee_api`'s uploads, apply the same source to `ai_vision`.
**Critical:** The host path `./lychee/uploads` must be identical for both mounts. If you use an absolute path or a named volume for `lychee_api`'s uploads, apply the same source to `lychee_facial_recognition`.

---

Expand All @@ -97,22 +107,49 @@ The AI Vision service reads photo files directly from the filesystem — no HTTP

Add these to `x-common-env` or the service's `environment` block:

| Variable | Description | Example |
| Variable | Description | Default |
|---|---|---|
| `AI_VISION_FACE_URL` | Internal URL of the AI Vision service | `http://lychee-ai-vision:8000` |
| `AI_VISION_FACE_API_KEY` | Shared secret used in both directions: Lychee sends it on scan requests to Python; Python sends it on callback responses to Lychee | `changeme-strong-random-value` |
| `AI_VISION_ENABLED` | Master kill-switch for the AI Vision subsystem | `true` |
| `AI_VISION_FACE_URL` | Internal URL of the facial recognition service | — |
| `AI_VISION_FACE_API_KEY` | Shared secret for mutual authentication (`X-API-Key` header) | — |
| `AI_VISION_FACE_RESCAN_IOU_THRESHOLD` | IoU threshold for preserving `person_id` on re-scan | `0.3` |
| `AI_VISION_FACE_STUCK_SCAN_THRESHOLD_MINUTES` | Minutes before a pending scan is considered stuck | `720` |

> Generate strong secrets with: `openssl rand -hex 32`

### AI Vision Service (`ai_vision`)
### Facial Recognition Service (`lychee_facial_recognition`)

| Variable | Description | Default |
|---|---|---|
| **Connection** | | |
| `VISION_FACE_LYCHEE_API_URL` | Base URL of the Lychee API (for callbacks) | — |
| `VISION_FACE_API_KEY` | Must match `AI_VISION_FACE_API_KEY` in Lychee | — |
| `VISION_FACE_VERIFY_SSL` | Verify SSL certificates when connecting to Lychee. Set to `false` for dev environments with self-signed certificates | `true` |
| `VISION_FACE_VERIFY_SSL` | Verify SSL certificates when connecting to Lychee | `true` |
| `VISION_FACE_SKIP_LYCHEE_CHECK` | Skip Lychee connectivity check at startup | `false` |
| **Logging** | | |
| `VISION_FACE_LOG_LEVEL` | Log level: debug, info, warning, error, critical | `info` |
| **Clustering** | | |
| `VISION_FACE_CLUSTER_EPS` | DBSCAN epsilon (max cosine distance); lower = tighter clusters | `0.6` |
| **Storage** | | |
| `VISION_FACE_PHOTOS_PATH` | Path where photos are mounted inside the container | `/data/photos` |
| `VISION_FACE_STORAGE_PATH` | Path for persisting face embeddings | `/data/embeddings` |
| `VISION_FACE_STORAGE_BACKEND` | Embedding store engine: `sqlite` or `pgvector` | `sqlite` |
| `VISION_FACE_STORAGE_PATH` | Directory for the SQLite embedding database | `/data/embeddings` |
| **Concurrency** | | |
| `VISION_FACE_THREAD_POOL_SIZE` | CPU threads for face detection inference | `1` |
| `VISION_FACE_WORKERS` | Uvicorn worker processes | `1` |
| **Queue** | | |
| `VISION_FACE_QUEUE_BACKEND` | Job queue backend: `database` or `redis` | `database` |
| `VISION_FACE_QUEUE_MAX_SIZE` | Max pending jobs (0 = unlimited); excess requests get HTTP 429 | `0` |
| **Detection thresholds** | | |
| `VISION_FACE_DETECTION_THRESHOLD` | Bounding-box confidence filter (0-1) | `0.5` |
| `VISION_FACE_MATCH_THRESHOLD` | Cosine-similarity cutoff for selfie match and suggestion candidates | `0.5` |
| `VISION_FACE_RESCAN_IOU_THRESHOLD` | IoU threshold for bounding-box matching on re-scan | `0.5` |
| `VISION_FACE_MAX_FACES_PER_PHOTO` | Maximum faces included in a callback payload (top-N by confidence) | `10` |
| **Quality filtering** | | |
| `VISION_FACE_MIN_FACE_SIZE_PIXELS` | Minimum face size in pixels (longest side); 0 = disabled | `0` |
| `VISION_FACE_BLUR_THRESHOLD` | Laplacian variance threshold; faces below this are discarded as blurry | `0.5` |

> See the full list of environment variables at the [Lychee-Facial-Recognition `.env.example`](https://github.com/LycheeOrg/Lychee-Facial-Recognition/blob/master/.env.example).

---

Expand All @@ -122,7 +159,17 @@ After starting the containers, enable the feature in **Admin → Settings → AI

1. **AI Vision enabled** — master toggle; set to `On`.
2. **Facial recognition enabled** — sub-toggle; set to `On`.
3. Configure optional settings (permission mode, batch size, etc.).
3. Configure optional settings:

| Setting | Default | Description |
|---|---|---|
| `ai_vision_face_permission_mode` | `restricted` | Who can view People, face overlays, and manage faces |
| `ai_vision_face_selfie_confidence_threshold` | `0.8` | Minimum confidence for selfie-based person claim |
| `ai_vision_face_person_is_searchable_default` | `On` | Default `is_searchable` flag for new Person records |
| `ai_vision_face_allow_user_claim` | `On` | Allow non-admin users to claim a Person |
| `ai_vision_face_overlay_enabled` | `On` | Show face bounding-box overlays in the UI |
| `ai_vision_face_overlay_default_visibility` | `visible` | Default overlay state when opening a photo (`visible` or `hidden`; toggle with `P` key) |
| `ai_vision_face_recognition_warning` | `On` | Show legal warning on Face Clusters and Face Maintenance pages |

These settings are only visible on Supporter Edition instances.

Expand Down Expand Up @@ -165,28 +212,74 @@ After setup, scan your existing photo library for faces:

**Via CLI:**
```bash
# Scan all unscanned photos
php artisan lychee:scan-faces

# Scan only a specific album
php artisan lychee:scan-faces --album={album_id}
```

Scanning runs asynchronously through the queue. Ensure the `lychee_worker` container is running. Progress is visible in the queue job history.

---

## Clustering

After faces have been detected, run DBSCAN clustering to group similar faces together. This helps with bulk assignment of faces to Person records.

**Via the admin UI:**
1. Navigate to **Admin → Maintenance**.
2. Find the **Run Face Clustering** card and click to trigger clustering.

Clustering is performed by the Python service. It reads all stored embeddings, runs DBSCAN (controlled by `VISION_FACE_CLUSTER_EPS`), and posts the results back to Lychee. Faces receive a `cluster_label`:

- `NULL` — not yet clustered
- `-1` — noise (not part of any cluster)
- `0, 1, 2, ...` — cluster ID

Cluster results can be reviewed and assigned to persons from the **Face Clusters** page in the admin UI.

---

## Maintenance Operations

All maintenance operations are available in **Admin → Maintenance**:

| Operation | Description |
|---|---|
| **Bulk Face Scan** | Enqueue all unscanned photos for face detection |
| **Run Face Clustering** | Trigger DBSCAN clustering on all face embeddings |
| **Destroy Dismissed Faces** | Hard-delete all faces marked as dismissed (also removes embeddings from the Python service) |
| **Sync Face Embeddings** | Synchronise embedding data between Lychee and the Python service |
| **Reset Face Scan Status** | Reset stuck-pending or failed photos so they can be re-scanned |

**Additional CLI commands:**

```bash
# Re-enqueue all failed scans
php artisan lychee:rescan-failed-faces

# Also reset photos stuck in pending for longer than 60 minutes
php artisan lychee:rescan-failed-faces --stuck-pending --older-than=60
```

---

## Service Health Check

The AI Vision service exposes a `/health` endpoint:
The facial recognition service exposes a `/health` endpoint:

```bash
# Inside the lychee network, from another container:
curl http://lychee-ai-vision:8000/health
curl http://lychee_facial_recognition:8000/health

# From the host (if you expose the port):
curl http://localhost:<MAPPED_PORT>/health
# From the host (default mapped port 8001):
curl http://localhost:8001/health
```

A healthy response:
```json
{"status": "ok", "version": "x.y.z"}
{"status": "ok", "model_loaded": true, "embedding_count": 1234}
```

Docker will also report the container's health status — wait for `healthy` before triggering scans:
Expand All @@ -195,37 +288,47 @@ Docker will also report the container's health status — wait for `healthy` bef
docker compose ps
```

Lychee's **Admin → Diagnostics** page includes an AI Vision service health check that verifies connectivity, health status, and configuration consistency.

---

## Troubleshooting

### AI Vision endpoints return 403

- Confirm the Lychee instance is a **Supporter Edition** licence.
- Check that `ai_vision_enabled = 1` and `ai_vision_face_enabled = 1` in admin settings.

### Photos are not scanned / `face_scan_status` stays `pending`

1. Verify the `lychee_worker` container is running (`docker compose ps`).
2. Confirm `QUEUE_CONNECTION` is not `sync` in the Lychee worker environment.
3. Check the AI Vision service health endpoint.
3. Check the facial recognition service health endpoint.
4. Review `lychee_worker` logs: `docker compose logs lychee-worker`.
5. If photos are stuck in `pending` for a long time, reset them:
```bash
php artisan lychee:rescan-failed-faces --stuck-pending --older-than=60
```

### AI Vision service cannot find photos
### Facial recognition service cannot find photos

- Compare volume mounts: the host `./lychee/uploads` path must be the same in both the `lychee_api` and `ai_vision` volume definitions.
- Compare volume mounts: the host `./lychee/uploads` path must be the same in both the `lychee_api` and `lychee_facial_recognition` volume definitions.
- Verify `VISION_FACE_PHOTOS_PATH` inside the container matches the volume mount destination.

### API key mismatch errors (401 from AI Vision / Lychee)
### API key mismatch errors (401 from either service)

- `AI_VISION_FACE_API_KEY` (Lychee) must equal `VISION_FACE_API_KEY` (Python service). The same key is used in both directions.
- `AI_VISION_FACE_API_KEY` (Lychee) must equal `VISION_FACE_API_KEY` (Python service). The same key is used in both directions via the `X-API-Key` header.
- Restart both containers after changing the secret.

### Selfie claim returns "no match found"

- Lower `ai_vision_face_selfie_confidence_threshold` (default `0.8`) in admin settings to accept less-certain matches.
- Ensure the photo library has been fully scanned first.

### Clustering produces too many / too few clusters

- Adjust `VISION_FACE_CLUSTER_EPS` (default `0.6`). Lower values create tighter, more numerous clusters; higher values merge more faces together.
- Re-run clustering from Admin → Maintenance after changing the value.

---

*Last updated: 2026-03-22*
*Last updated: 2026-06-22*
Loading