Automated Engine Analysis: AI or Human, Similarity To Stockfish

[chrisw]

In case this may help

This is the github repository for Titan Chess Engine:
https://github.com/jeff-pow/Titan

For Spaghet BologNNese:
https://github.com/yl25946/spaghet

For pawn:
https://github.com/ruicoelhopedro/pawn

Thanks for those, added to the ccrl-top-50 section
https://whittingtonchess.com/engine-ana ... crl-top50/

Revenge is closed source:
https://www.revengechess.com/

Its a continuation of Open source pedone chess engine, revenge based on CCRL color green code, definitely a free but closed source.

For Ginkgo santiago is private also based on CCRL color code.

In case this may help too, you can check revenge site above and visit ccrl blitz for ginkgo santiago

Joseph.

I can't do closed source

[chrisw]

Could you please run Askaig, Pawnstar, Malika, Triumviratus and Amira through this 'test'?

https://github.com/sophiathedev/askaig
https://github.com/jonny-reckless/pawnstar
https://github.com/FauziAkram/Malika-releases
https://github.com/Tors3/Triumviratus
https://github.com/FauziAkram/amira

Triumviratus was already done.

- askaig → AI-assisted (self-disclosed "sophiathedev and Claude Fable 5"); C++26 NNUE; Strong SF-similarity
- pawnstar → AI-assisted, veteran-led (12-yr project + 10 real Claude-authored commits doing int8 NNUE/AWS training, SPRT-gated); Strong
- amira → AI-assisted (self-disclosed human+LLM); single-file HCE engine from a YouTube series; Independent
- Malika → no page possible (binary-only repo, no source)

Three more for the AI-assisted band. Webpages updated

https://whittingtonchess.com/engine-analysis/

I should point out, we are beginning to identify human authored programs that are now benefiting from AI assistance on top. This gets very hybrid and ranges from all and only AI, to human developed for two years with intense AI assistance in the last two weeks.

[chrisw]

CODA added, interesting one ....

https://whittingtonchess.com/coda-analysis/

[dkappe]

Thanks for this. Your observation on the top 50 ccrl page...

The top has converged on the Stockfish paradigm. Almost every engine rates Strong on Stockfish-similarity: bitboards, NNUE, and the LMR/NMP/SEE search stack — independently implemented, because that is simply what wins now.

...may be accurate, but the same would be true if you built an engine working your way through CPW. I don't think that's really the Stockfish paradigm. Maybe a fruit+falcon paradigm.

[chrisw]

Thanks for this. Your observation on the top 50 ccrl page...

The top has converged on the Stockfish paradigm. Almost every engine rates Strong on Stockfish-similarity: bitboards, NNUE, and the LMR/NMP/SEE search stack — independently implemented, because that is simply what wins now.

...may be accurate, but the same would be true if you built an engine working your way through CPW. I don't think that's really the Stockfish paradigm. Maybe a fruit+falcon paradigm.

yes, this is true, we all make the error of Stockfish-centricity, where everything revolves around and is based on Stockfish. The reality is actually an ecosystem of which Stockfish is a part, Stockfish developers and all the other engines developers (and now AI agents) are all reading each others code for new ideas and ways of doing things, then putting the ideas, or modifications of, into their own engines for the testing, tuning, reject/accept cycles. Stockfish is no different and is packed full of ideas from elsewhere (NNUE came from Go engines, nnue threats from somewhere else and so on).

[Chessgate]

https://whittingtonchess.com/engine-ana ... crl-top50/

https://whittingtonchess.com/engine-analysis/

So what exactly is the methodology?

Added a methodology page

https://whittingtonchess.com/engine-analysis/about/

What I don´t really get is that AI is used to scan for AI Use , AI is used to write the "conclusion" page, to then determine AI as worthy of an own "slop" section. Would it not be more honest to scan the Engines by hand? Since the multiple appearance of long em Dashes in the text at least hints to the use of AI to write the AI scan conclusions,

[chrisw]

https://whittingtonchess.com/engine-ana ... crl-top50/

https://whittingtonchess.com/engine-analysis/

So what exactly is the methodology?

Added a methodology page

https://whittingtonchess.com/engine-analysis/about/

What I don´t really get is that AI is used to scan for AI Use , AI is used to write the "conclusion" page, to then determine AI as worthy of an own "slop" section. Would it not be more honest to scan the Engines by hand? Since the multiple appearance of long em Dashes in the text at least hints to the use of AI to write the AI scan conclusions,

Not much one can do if you don't get it.

Human researchers, especially involved engine programmers, are going to introduce big bias, find it way too much work to carry out the assessments, will be inconsistent and subjective. Comparative assessments, in contrast, by AI are ideal here, all engines get the same, objective, unbiased and consistent treatment with methodology explained.

[j.t.]

Comparative assessments, in contrast, by AI are ideal here, all engines get the same, objective, unbiased and consistent treatment with methodology explained.

I think to be unbiased you'd need to do a bit more work. Among other things every thing that can be used for identification (obviously the folder name or the "id author", "id name" string, but also other things) must be removed / anonymized (AI tools can be biased by these). Also, ideally each code base should be transformed into the same language (likely the best choice would be C++), since in some cases that can be used as a strong identification hint. And the training and datagen code for the networks (which unfortunately many people don't clearly reference in their repo) should be included in the analysis, given that network strength is a significant, maybe the most significant, factor in how good an engine plays.