Skip to content
Qi Su

Focus

Agent infrastructure

At Meta I work on developer productivity for the Monetization organization. I build AI agent systems that take on routine engineering work, such as triaging and diagnosing issues, along with the orchestration and knowledge layers that keep those agents useful as the codebase changes underneath them.

Evaluation you can trust

Leaderboard scores are how the field decides which coding agents work. I study when those scores hold up: whether a benchmark’s solutions may have leaked into what a model has seen, and how matched-control experiments can separate real capability from familiarity.

Reliability for agents that act

Agents that send, spend, or publish need the discipline of any production system. I am interested in carrying SRE practice into agent design: error budgets, circuit breakers, idempotent actions, and human review keyed to how reversible an action is.

Updates

  1. Measurement-First Auditing of Agentic Leaderboards was accepted at the NeurIPS 2026 LCFM workshop.
  2. Attended CCF Computility 2026 in Xiong’an and talked agents with research groups there.
  3. Built Vamos, a Spanish learning site, to support my own study.
  4. Joined Meta as a Software Engineer on the Developer Productivity team.

Research

Accepted NeurIPS 2026 · LCFM Workshop

Measurement-First Auditing of Agentic Leaderboards: Contamination Susceptibility, Matched-Control Re-evaluation, and Scorer Validation

Dishu Yang, Qi Su, Hongbo Qin, Hansong Zhang

A framework that sorts contamination claims by the evidence they need. Across nine configurations on the Holistic Agent Leaderboard, none of the 27 configuration-channel assessments could be coded closed. A same-repository matched-control re-test on SWE-bench Verified found a +10.0-point Top-3 gap for GPT-4.1, with 95% intervals that included zero.

Non-archival workshop paper

Projects

Design

Multi-agent paper-writing pipeline

A seven-stage state machine where agents exchange typed artifacts under JSON Schema contracts. A claims registry ties every sentence back to evidence, reviewer agents run in isolation, and model routing sends only quality-critical steps to the largest models. Orchestration stays deterministic; the judgment lives in the workers.

Design docs v0.3
Live

Vamos

A Spanish learning site with clickable words that speak themselves, spaced-repetition flashcards, dialogues, and verb tables. Content lives as typed data in git so lessons are easy to add.

Astro · React islands · Cloudflare Pages

Experience

Dec 2024 – now
Software Engineer, Infrastructure Meta · Bellevue, WA Developer Productivity team, Facebook Monetization
  • Build AI agent systems that take on routine engineering work for the Monetization org, such as triaging and diagnosing issues.
  • Design the orchestration and knowledge layers that keep those agents accurate as code and processes change.
Jun 2022 – Nov 2024
Software Engineer Beaconfire · East Windsor, NJ Microservices for a user and robot management platform
  • Built Spring Boot and Spring Cloud services for users, products, robots, maps, and events, connected through Kafka on AWS MSK and Camunda workflows.
  • Used Redis with Lua scripts for atomic stock updates that stopped overselling, plus Sentinel flow control and Keycloak authentication at the gateway.
  • Designed a tree-based permission model for role and resource management.
Oct 2020 – Jun 2022
Software Engineer WeBranding Global · New York, NY Backend for a social media monitoring dashboard
  • Built REST APIs on Spring Boot and MySQL for trends, hashtags, and posts, with Elasticsearch for search and keyword autocomplete.
  • Wrote a data processor service that consumes posts from RabbitMQ, and backup APIs on S3.
Education
MS in Computer ScienceThe George Washington University

Off hours

TennisTaking lessons, still a beginner, and just upgraded my racket.
ClimbingIndoor climbing, where every route is a small puzzle.
RunningBuilding up from 5K.
LanguagesSpanish, with a study site I built for myself, and French from zero, a little every day.
MusicClassical concerts. Still learning when to clap.
ReadingBack to finishing novels.
TasteQuiet, well-made things: tailoring, minimal design, good audio.
RootsChinese folk traditions and the stories behind them.

Contact

I like hearing from people working on agent infrastructure, evaluation methodology, or benchmark integrity.

qisu623@gwu.edu