Flow is an enterprise-grade digital knowledge management and meeting intelligence platform designed to transform unstructured conversational audio into structured, searchable, and actionable digital assets.
Inheriting the speech processing and diarization pipeline from the FDSE 2025 research publication ("V-Scribe: Structured Transcription of Vietnamese Speech for Digital Knowledge Management", Springer CCIS Vol 2708)[cite: 6], Flow introduces an asynchronous multi-container architecture that automates meeting transcription, speaker identification, conversational segmentation, and semantic retrieval.
The system implements a Modular Monolithic Backend combined with an Isolated Asynchronous AI Worker Container orchestrated via Redis / BullMQ queues. The entire environment is containerized via Docker Compose behind an Nginx Reverse Proxy:
[ Client Browser (React SPA) ]
│
▼ HTTPS (Port 443 / 80)
┌────────────────────────────────────────────────────────────────────────┐
│ Virtual Machine (Host Environment) │
│ │
│ ┌────────────────────────────────────────────────────────────────┐ │
│ │ Nginx Reverse Proxy Container (TLS Termination & Rate Limiting)│ │
│ └───────┬────────────────────────┬───────────────────────┬───────┘ │
│ │ / │ /api/ │ /ai/ │
│ ▼ ▼ ▼ │
│ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ │
│ │ Frontend UI │ │ Backend API │ │ AI Service │ │
│ │ (React + Vite)│ │ (Node.js REST)│ │ (FastAPI App) │ │
│ └───────────────┘ └───────┬───────┘ └───────▲───────┘ │
│ │ │ │
│ Dispatch Job ▼ │ HTTP Task │
│ ┌───────────────────┐ │ Invocation│
│ │ Redis Message │ │ │
│ │ Broker (BullMQ) │ │ │
│ └─────────┬─────────┘ │ │
│ │ │ │
│ Consume Task ▼ │ │
│ ┌───────────────────────────────────┴───┐ │
│ │ Node.js Background Worker Container │ │
│ └───────────────────┬───────────────────┘ │
└────────────────────────────────────────────┼───────────────────────────┘
│ Outbound TLS / HTTPS
▼
┌─────────────────────────────────────────┐
│ External Managed BaaS (Supabase Cloud) │
│ • PostgreSQL: Structured Relational DB │
│ • pgvector: 512-dim Semantic Embeddings │
│ • Object Storage: Audios & Avatars │
└─────────────────────────────────────────┘
- Voice Profiling & Speaker Enrollment: Enrolls user voice samples to construct reference acoustic embeddings, eliminating manual speaker labeling during meeting post-processing.
- Automated Neural Processing Pipeline:
- Speaker Diarization: Pyannote.audio sequence labeling for Voice Activity Detection (VAD) and Speaker Change Detection (SCD).
- Vietnamese Speech-to-Text (ASR): High-precision transcription powered by fine-tuned
PhoWhisper-large. - Meeting Intelligence: Automated topic extraction, executive summarization, and speaker participation intensity metrics.
- Vector Semantic Search: Natural language conceptual queries (e.g., "budget decisions") powered by PostgreSQL
pgvectorHNSW indexes. - Contextual Playback (Click-to-Play): Interactive bidirectional synchronization between transcript segments and audio timestamps.
- Enterprise Security: "No-Download" view-only streaming, Supabase Row Level Security (RLS), and short-lived Signed URLs.
The system uses Supabase (PostgreSQL with pgvector extension) to enforce referential integrity across 13 domain tables:
account # User identity, authentication credentials, and profiles
setting # User preferences (theme, language, notifications)
voice_sample # Reference audio samples and voiceprint embedding vectors
meeting # Meeting metadata, processing status, and ownership
audio_file # Raw audio file references, duration, and formats
processing_job # Job-level lifecycle and progress tracking
processing_step # Step-level execution tracking (STT, diarization, analysis)
speaker # Meeting participants identified by acoustic matching
transcript_segment # Timestamped transcript segments mapped to speakers
semantic_segment # Topic-based semantic timeline segments
speaker_statistic # Speaking frequency and interaction intensity metrics
search_index # 512-dimension vector embeddings for semantic search
meeting_summary # AI-generated meeting summaries
Flow/
├── ai_service/ # Python FastAPI AI processing engine
│ ├── analysis/ # NLP summaries, topics, and behavioral analytics
│ ├── asr/ # PhoWhisper Speech-to-Text pipeline
│ ├── audio/ # Chunking, noise reduction, and preprocessing
│ ├── speaker/ # Pyannote diarization & ECAPA-TDNN embeddings
│ ├── Dockerfile # AI container specification
│ ├── main.py # FastAPI service entry point
│ └── requirements.txt # PyTorch, Transformers, Pyannote dependencies
│
├── backend-node/ # Node.js Modular Monolith & Async Worker
│ ├── src/
│ │ ├── config/ # AppConfig & environment management
│ │ ├── queues/ # RedisQueueManager (BullMQ orchestrator)
│ │ ├── services/ # Domain services (Auth, Meeting, User, Voice)
│ │ └── utils/ # Crypto, Storage, DateTime, Validation helpers
│ ├── server.js # REST API entry point (Port 9000)
│ ├── worker.js # Background job queue consumer
│ ├── Dockerfile # Backend container specification
│ └── package.json # Node.js dependencies
│
├── frontend/ # React.js Single Page Application (SPA)
│ ├── src/
│ │ ├── assets/ # Icons, illustrations, and static assets
│ │ ├── components/ # Reusable UI component library (Design System)
│ │ ├── pages/ # Auth, Landing, Dashboard, DiaryDetail, Profile
│ │ ├── services/ # REST API client & Supabase auth service
│ │ └── App.tsx # SPA routing & application layout
│ ├── nginx.conf # Internal Nginx web server config (Port 80)
│ ├── Dockerfile # Multi-stage production build container
│ └── package.json # React, Vite, Tailwind CSS dependencies
│
├── reverse-proxy/ # Nginx gateway reverse proxy configuration
├── docker-compose.yml # Multi-service container orchestration
├── .gitignore # Version control exclusion rules
└── README.md # Project documentation
git clone [https://github.com/sh1kaku59/Flow.git](https://github.com/sh1kaku59/Flow.git)
cd FlowEnsure the following base tools are installed:
- Node.js:
20.xLTS (Docker images usenode:20-alpine) - npm:
10+ - Python:
3.11.x(AI Docker image usespython:3.11-slim) - Docker Desktop & Docker Compose:
v2+ - Redis:
7.x(if running locally without Docker) - FFmpeg: Installed and added to system
PATH(required for local audio normalization)
cd frontend
npm install
npm run dev- Default local URL:
http://localhost:5173 - Note: Vite is configured with
envDir: "../backend-node"infrontend/vite.config.js, reading frontend environment variables directly.
In a new terminal window:
cd backend-node
npm install
npm run devRun the background worker in a separate terminal:
cd backend-node
npm run dev:worker- Backend health check:
http://localhost:9000/health
From the repository root:
# Create virtual environment
python -m venv .venv
# Activate environment
# On Windows:
.venv\Scripts\activate
# On macOS/Linux:
source .venv/bin/activate
# Install dependencies
pip install -r ai_service/requirements.txt
# Run AI service
python -m uvicorn ai_service.main:app --host 0.0.0.0 --port 8001- AI Service health check:
http://localhost:8001/health
- Create a project on Supabase Dashboard.
- Apply the database schema via Supabase SQL Editor for the 13 required tables.
- Create the following Supabase Storage buckets:
voice-samplesavatarsmeeting-audios
- Copy the Supabase URL and Service Role Key into
backend-node/.env.
VITE_API_BASE_URL=http://localhost:9000
VITE_SUPABASE_URL=[https://your-project.supabase.co](https://your-project.supabase.co)PORT=9000
SUPABASE_URL=[https://your-project.supabase.co](https://your-project.supabase.co)
SUPABASE_SERVICE_ROLE_KEY=your_supabase_service_role_key
HF_TOKEN=your_huggingface_access_token
FRONTEND_ORIGIN=http://localhost:5173,[http://127.0.0.1:5173](http://127.0.0.1:5173)
AI_SERVICE_URL=[http://127.0.0.1:8001](http://127.0.0.1:8001)
BACKEND_CALLBACK_URL=[http://127.0.0.1:9000/ai/progress](http://127.0.0.1:9000/ai/progress)
REDIS_URL=redis://127.0.0.1:6379
VOICE_BUCKET=voice-samples
AVATAR_BUCKET=avatars
MEETING_AUDIO_BUCKET=meeting-audios
SESSION_COOKIE_NAME=sid
SESSION_COOKIE_SECURE=false
SESSION_COOKIE_SAMESITE=lax
QUEUE_WORKER_CONCURRENCY=1
RATE_LIMIT_DISABLED=false
EMBEDDING_DIM=512Launch all containerized services with a single command:
docker compose up -d --buildService Endpoints:
| Service | Container Name | Port | Description |
|---|---|---|---|
| Frontend | frontend |
80 (Internal) |
React UI served via Nginx |
| Backend API | backend |
9000 |
Node.js REST API & Auth orchestration |
| Worker | worker |
Background | BullMQ Queue consumer |
| AI Engine | ai-service |
8001 |
FastAPI audio processing & STT |
| Redis | redis |
6379 |
In-memory message broker |
| Gateway | reverse-proxy |
80 / 443 |
Nginx reverse proxy & SSL manager |
To stop all services:
docker compose down- Redis connection refused: Ensure the Redis container is healthy or local
redis-serveris running, and verify thatREDIS_URLmatches the network. - Backend startup failure (Supabase error): Verify that
SUPABASE_URLandSUPABASE_SERVICE_ROLE_KEYare correct and that the key role isservice_role. - AI Service
HF_TOKENerror: Pyannote diarization requires an accepted Hugging Face model agreement and a valid user access token. - Audio decoding errors: Verify that
FFmpegis correctly installed on the operating system and accessible via terminal.
- Never commit
.envfiles or secret keys into version control. SUPABASE_SERVICE_ROLE_KEYmust never be exposed in client-side code or public repositories.- All storage files are accessed via temporary signed URLs with strict expiry policies.
- Institution: Faculty of Information Technology, Van Lang University, Ho Chi Minh City, Vietnam
- Project Mentor & Product Owner: Huu Nghia Huynh (Institute for Experiential Technology)
- Development Team (Group 22):