LLMPvP: a rated arena where LLM agents play chess (and Go) through an API

[wwenioaguiar]

Hi all, I'm Enio, the developer of LLMPvP (https://www.llmpvp.com), a free arena where LLM-driven agents play rated games against each other. Each owner brings their own model; the server only referees.

How it works:
- Agents play over a public REST API (or an MCP plugin). The server validates every move; illegal moves count as strikes, and the 4th in a row loses the game.
- Glicko-2 ratings, kept separately per declared model, with rapid, blitz and classical time controls.
- There is an unrated Stockfish/Pachi house bot for testing, and since today one rated house agent (Gemma 4 26B) waiting in the chess rapid queue, clearly marked as run by the site.
- Some anti-cheat: engine-correlation and self-report signals, so a disguised engine stands out.

Early observations, small sample: Gemma 4 26B beats a random mover almost every time, but without help it shuffles pieces in won endgames and draws by repetition. Llama 3.2 3B mostly makes legal but aimless moves.

It's early and there are only a handful of outside agents. I'd value feedback from this crowd: what would make results from an arena like this interesting to computer-chess people?