Feat #35: event-driven worker queue with Redis-backed job processing#54
Conversation
- Create app/lib/queue/ with types, queue operations, and job handlers - Redis sorted-set queue with priority ordering by scheduledAt - Exponential backoff retry (2^attempt * 5s, capped at 120s) - Dead letter queue after 3 failed attempts - Job types: evaluate_bounty, execute_task, process_payment, match_agent, settle_bounty - Cron worker endpoint (GET /api/cron/worker) processing 10 jobs per run - SSE status endpoint (GET /api/queue/status) for real-time monitoring - Add * * * * * cron schedule to vercel.json - 9 new unit tests for queue operations
|
Deployment failed with the following error: Learn More: https://vercel.link/3Fpeeb1 |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: dde8c7843c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const isCron = req.headers.get('user-agent')?.includes('Vercel Cron'); | ||
| const isAuthed = secret && auth === `Bearer ${secret}`; | ||
| if (!isCron && !isAuthed) { |
There was a problem hiding this comment.
Using user-agent as a trust signal allows any external caller to spoof Vercel Cron and run the worker without the shared secret, which can execute queued jobs (including payment-related handlers) on demand. This route should only trust a verifiable signal (for example authorization or a platform-provided cron header) before processing jobs.
Useful? React with 👍 / 👎.
| const results = await redis.zrange(JOB_QUEUE_KEY + ':order', 0, now, { byScore: true, count: 1, offset: 0 }); | ||
| if (!results || results.length === 0) return null; | ||
| const jobId = results[0] as string; | ||
| const jobData = await redis.hget(JOB_QUEUE_KEY, jobId); |
There was a problem hiding this comment.
The dequeue path reads the next ID via zrange and then marks the hash entry as processing, but does not atomically remove/claim the sorted-set member in the same operation. Concurrent worker invocations can therefore read the same job ID before either completes, causing duplicate execution (e.g., repeated task execution or payment processing).
Useful? React with 👍 / 👎.
1 participant
Summary
scheduledAttimestampevaluate_bounty,execute_task,process_payment,match_agent,settle_bounty— each with an async handlerGET /api/cron/workerprocesses 10 jobs per invocation, protected byCRON_SECRETor Vercel Cron UAGET /api/queue/statusstreams real-time queue depth + job lists*/1 * * * *cron schedule tovercel.json