Debugging strategies and tools for troubleshooting Decabill issues.
# Billing manager API (local)
nx serve decabill-backend-billing-manager
# Docker Compose
cd apps/decabill/backend-billing-manager
docker compose logs -f backend-billing-manager
docker compose logs -f backend-billing-manager-worker
docker compose logs -f backend-billing-manager-schedulerConfigure log levels via environment variables where supported:
LOG_LEVEL=debug- Detailed debugging informationLOG_LEVEL=info- General informationLOG_LEVEL=warn- WarningsLOG_LEVEL=error- Errors only
Pass X-Correlation-Id or X-Request-Id on API requests to trace a single operation across API and worker logs. Access logs include [corr=…] in Nest output when inside the request context.
Common patterns to search for:
- Connection errors: Database, Redis, SMTP, Stripe, cloud APIs
- Authentication errors: Invalid API key, expired JWT, tenant mismatch
- Queue errors: BullMQ connection failures, job processor exceptions
- Payment errors: Stripe signature failures, checkout session errors
Secrets are redacted in access logs; do not rely on logs for full token values.
When QUEUE_BULL_BOARD_ENABLED=true on the API:
- URL:
http://localhost:3200/admin/queues - Credentials:
QUEUE_BULL_BOARD_USERNAME/QUEUE_BULL_BOARD_PASSWORD
Use Bull Board to:
- Inspect failed jobs and stack traces
- Retry stalled subscription billing or provisioning jobs
- Verify coordinator repeatable jobs are registered
Job names are defined in apps/decabill/backend-billing-manager/src/queue/job-registry.ts.
- Network tab: Monitor API calls, WebSocket upgrade, Stripe redirects
- Console tab: CSP violations, Angular errors
- Application tab: Storage and cookies for Keycloak sessions
docker exec -it billing-manager-api /bin/sh
docker exec -it billing-manager-worker /bin/sh
docker stats billing-manager-api# From host (compose default port)
redis-cli -p 6380 ping
# Inside compose network
docker compose exec redis redis-cli ping
docker compose exec redis redis-cli keys 'decabill-billing*'psql -h localhost -U postgres -d postgres
# Example tenant-scoped queries
SELECT id, tenant_id, status FROM subscriptions LIMIT 10;
SELECT id, tenant_id, status FROM invoices ORDER BY created_at DESC LIMIT 10;curl http://localhost:3200/api/healthcurl -H "Authorization: ApiKey your-api-key" \
-H "X-Tenant: default" \
http://localhost:3200/api/billing/subscriptionsAdjust path to match your OpenAPI specification.
Use Stripe CLI to forward webhooks to your local API:
stripe listen --forward-to localhost:3200/api/billing/stripe/webhookSet STRIPE_WEBHOOK_SECRET to the signing secret from stripe listen.
Billing WebSocket gateway listens on port 8082 with namespace billing.
Use browser DevTools Network tab or a Socket.IO client with:
- Correct origin (CORS)
- Auth token or session as required by your auth mode
X-Tenantheader on handshake for multi-tenant setups
- Check
subscription-billing.coordinatoris repeating in Bull Board - Verify worker is processing
subscription-billing.unitjobs - Review subscription status and billing dates in database
- Check
BILLING_SCHEDULER_INTERVALand batch size env vars
- Verify worker and API share
invoice_pdf_datavolume in compose - Check
BILLING_INVOICE_PDF_STORAGE_PATH(default/data/invoices) - Review worker logs during PDF generation jobs
- Confirm console sends correct
X-Tenant - For API key auth, review DR-002
- Verify user's
tenant_idin database matches expected tenant
- Monitor response times on heavy admin list endpoints
- Check database query plans for large tenant datasets
- Review rate limiting metrics if 429 responses appear
- Tune
QUEUE_WORKER_CONCURRENCYfor CPU and external API limits - Scale horizontally with additional worker containers
- Watch Redis memory as job history accumulates (jobs are not auto-removed)
- Index usage on
tenant_idfiltered queries - Connection pool sizing under scheduler batch loads
- Common Issues - Common problems and solutions
- Background jobs - Queue roles and job registry
- Environment configuration - Scheduler intervals
For specific issues, see Common Issues.