Hassam Gani

Debate, voting, or self-refinement?

When you can only afford so many LLM calls, is it better to have models vote, refine their own answers, or debate each other? This project tests a theory that predicts which one wins, across six models and six benchmarks.

Self-play for LLM reasoning

Can a language model get better at reasoning by training on problems it makes up for itself, with no labeled data? The system started with math and now covers open-ended writing. Along the way, re-measuring R-Zero, a published self-play method, showed that most of its reported gains shrink to about one point.

Graph vs. SQL database agents

Do LLM agents answer database questions better when the data is stored as a graph instead of SQL tables? On clinical records the graph agent came out slightly ahead, mostly because the SQL agent mishandled questions about relative dates. The project also found 626 correct answers that the benchmark's grader had marked wrong.

Malicious code in pickled models

ML models are often shared as Python pickle files, which can run hidden code the moment they're loaded. cpyAnalyzer catches these: its loading policies blocked every malicious model while still accepting 79.8% of benign ones. Supported by a Kluge Scholar grant.

Code