Undergraduate in Cyber Science and Technology at Sun Yat-sen University. I have been contributing to open-source AI tooling since July 2026 and still send patches most days, mostly to evaluation and red-teaming frameworks, agent and MCP tooling, and the libraries they sit on.
Much of what I fix has the same shape. Something fails, nothing crashes, and the caller gets a value that looks like success. A failed page read comes back as if it were the page; a scanner reports that it wrote its findings when the write never happened.
| Area | Projects |
|---|---|
| Evaluation and red teaming | PyRIT · inspect_ai · inspect_evals · lm-evaluation-harness · opik · uqlm · trulens · rhesis · garak · lmms-eval |
| Agents, coding agents and MCP | MCP servers · fastmcp · serena · letta-code · pydantic-ai · livekit agents · haystack · llama_index · griptape · fantasy · mcp-context-forge · zotero-mcp |
| Structured generation and inference | xgrammar · outlines · vllm-metal |
| Tracing and observability | openinference · phoenix · openlit · langwatch |
| Security tooling | fickling · AI-Infra-Guard · agentic_security · agent-sweep |
| Data validation and ingestion | pandera · great_expectations · qdrant-client · unstructured |
- inspect_evals #2132. With more than one epoch, Humanity's Last Exam averaged each sample across epochs before computing calibration error, so errors in opposite directions cancelled and a maximally miscalibrated run scored 0.0. Calibration is now computed per attempt, and single-epoch results are unchanged. The issue was opened by a maintainer.
- PyRIT #2467. Gave the GCG optimisation loop explicit state types. In review the maintainer caught that my first version used
infto mean "not measured", which would have hidden a genuine non-finite loss; it now carries an explicit flag. The issue was opened by a maintainer. - uqlm #459. When one judge in a panel ran out of retries, every aggregate for that prompt became NaN and the run still reported success. I reported it and sent the fix together.
- lm-evaluation-harness #4039. MATH answer normalization turned tuple answers such as
0,1into01, so correct answers were marked wrong. The maintainer kept the leaderboard copy of the function frozen, because changing it would make historical scores incomparable. I had not thought of that, and it was the right call. - opik #8195. A system under evaluation could embed a verdict in its output that the judge would repeat, and the parser took the first one. My first revision escaped the values; the maintainer pointed out that rewriting the evaluated output distorts the evaluation itself, so the merged version isolates the output with namespaced delimiters instead and states the remaining risk in tests. The issue was reported by another contributor.
- xgrammar #834. Made
prefixItemspositional, as JSON Schema Draft 2020-12 specifies. An earlier draft accepted too much; the regression matrix caught it and it was replaced.
failroute is a static analyzer I wrote for one family of these bugs: Python exception handlers that return something a caller cannot tell apart from success. pip install failroute.
Measuring it was the most useful part. On eight pinned AI packages it flags plenty that standard linters miss, but in a sample labelled by LLM agents most findings were intentional fallbacks, and among findings the linters miss, about 1 in 56 was labelled a defect. The defects it did find are also caught by flake8 --select E722. The repository has the full numbers and the method.
Before filing anything I follow the call path in the real code. When I ran failroute over PyRIT it raised several dozen warnings; I traced twelve of them, all twelve were deliberate design decisions, and I filed nothing.
I work with coding agents inside a workflow I set up myself, with my own test gates, and pull requests where AI tools shaped the change say so. Early on I gave the agents too much rope: they lost context over long runs and opened some weak and duplicate pull requests. I closed those and tightened the workflow. If one of my pull requests turns out to be wrong or not worth your time, I close it; if you find one I missed, close it or tell me. Blunt review is welcome.
Most repositories on this account are forks for upstream work.


