Fat Fritz 2

[dkappe]

I challenge you to write an engine yourself to use these weight files that would beat std stockfish even without using any nets at all for eval.. You and this A. Silver guy cannot of course do that, so what do you do? You rip of SF and then try to make a grab the money and run scheme before people realise the emperor has no clothes...

What do I do? I think you have me confused with someone else. I don’t sell any chess software.

I have written a few engines. Right now I’m exploring hybrid mcts/ab with a0lite, which incorporates mcts/nn and ab/nnue. I doubt it will ever catch stockfish, however. I’m having a hard enough time catching xiphos. It is encouraging that the mcts+ab hybrid is now stronger than each search technique on its own.

I also train and release non-stockfish nnue nets for my own entertainment and to see how different data impacts the playing style. (answer? A lot. Far more than the underlying search.)

Not everything has to be done to be “the best” or for money, you know. You seem obsessed by these things.

[connor_mcmonigle]

Are we supposed to shell out 100$ to the ChessBase profiteers to do a test that would satisfy you ?

FatFritz2 is only a different net and the binary is pure Stockfish dev. That's a known fact. Does the net provides more general strength ? Public test say no. Does the net provide significantly different move suggestions/ordering, thereby offering chess analysis value despite not being stronger ? The onus of the proof is onto ChessBase and FF2 supporters. They could use Ed Shroeder's sim tool on short and long searches and show FF2 not closely clustering with SF-dev... The default assumption is that it closely clusters with SF-dev.

CCRL feeding the ChessBase marketing deceptions by presenting FF2 as different from the Stockfish family (and FF1 as different from the Leela family for that matter, I didn't like that either) is shameful.

Some people might think Stockfish and FF2 are different engine families, but they are not. CCRL should not encourage mistaken beliefs in people using it as reference. But then Madeleine makes a good point that CCRL still doesn't give indication of Fire and Houdini's nature.

I’m not asking you to buy anything. Given that I’ve trained Toga, Frosty (ICE), Night Nurse, Dark Horse, Harmon and Dragon nets, all from different data sources, I think I can claim some expertise in this domain. In my experience, the net matters more than the engine.

I simply assumed with all the confident statements about FF2, someone here must have actually looked at the analysis of it and compared it with SFDev.

Have you performed sim tests to validate your rather ambiguous claim that "the net matters more"? It would be interesting to have some data here.
Even if the sim tool were to produce low similarity scores for your various, weaker, networks, that doesn't necessarily guarantee that the FF2 net would also yield low similarity scores as Alayan points out. It would be great if someone having acquired the commercial FF2 from ChessBase could perform a sim test.

In any case, there's some merit to the claim that the evaluation function is most relevant to the results of a similarity test (observe the high similarity score assigned between Houdini and Komodo as a consequence of Houdini featuring a decompiled version of Komodo's evaluation function). Still, I'm dubious of the claim that a (likely) weaker network is superior for human analysis. I'd be willing to bet that A. Silver wouldn't be able to discern the difference between Stockfish with FF2 and Stockfish dev if provided a large number of test positions and the respective networks' move preferences.

[Modern Times]

I'd also like to see results of Sim tests and solving ability in test suites. I've never done that sort of thing before though, so don't have the knowledge to do it myself. I'm far more interested in solving results of the two.

[twobeer]

I challenge you to write an engine yourself to use these weight files that would beat std stockfish even without using any nets at all for eval.. You and this A. Silver guy cannot of course do that, so what do you do? You rip of SF and then try to make a grab the money and run scheme before people realise the emperor has no clothes...

I have written a few engines. Right now I’m exploring hybrid mcts/ab with a0lite, which incorporates mcts/nn and ab/nnue. I doubt it will ever catch stockfish, however. I’m having a hard enough time catching xiphos. It is encouraging that the mcts+ab hybrid is now stronger than each search technique on its own.

And you do realize that almost anyone, using the training cookbook for SF could train a NNUE-net in 24h that together with SF Dev would beat your A0 Lite running the best Network out there?. right?

And Still claim the Network is more important than the Engine ... pffft.

[MikeB]

Latest from the blitz list.

Weird, why would you show two stockfish engines at the top. Isn't normal practice is to show just one version for each engine?

[dkappe]

Have you performed sim tests to validate your rather ambiguous claim that "the net matters more"? It would be interesting to have some data here.
Even if the sim tool were to produce low similarity scores for your various, weaker, networks, that doesn't necessarily guarantee that the FF2 net would also yield low similarity scores as Alayan points out. It would be great if someone having acquired the commercial FF2 from ChessBase could perform a sim test.

In any case, there's some merit to the claim that the evaluation function is most relevant to the results of a similarity test (observe the high similarity score assigned between Houdini and Komodo as a consequence of Houdini featuring a decompiled version of Komodo's evaluation function). Still, I'm dubious of the claim that a (likely) weaker network is superior for human analysis. I'd be willing to bet that A. Silver wouldn't be able to discern the difference between Stockfish with FF2 and Stockfish dev if provided a large number of test positions and the respective networks' move preferences.

The sim tests were done most recently by laskos (sp?) across a range of nnue and non nnue engines and posted in this forum.

My own test were with Toga III and SF at fixed depth across 20000 positions. At d12 with the same net, SF12 and Toga agreed on best move over 95% of the time. The 5% where they didn’t match looked to be imbalanced positions where hybrid (sf) or lazy (toga) kicked in, though I haven’t gotten back to this experiment to verify that hypothesis.

SF with the default net matched less than 50% of the time (42%?). I’m pretty sure a stronger net like Dark Horse would match more frequently with default SF.

The PV past the first move often didn’t match, though I didn’t gather stats on that. Below a pretty typical example from a Seer vs a0lite game.

[d]rn1q1rk1/pp2ppbp/2p2np1/8/3P3P/2N2N2/PPPBQ1P1/2KR3R b - h3 0 11

Code: Select all

SF12 + default net
info depth 12 seldepth 16 multipv 1 score cp 81 nodes 35781 nps 305820 tbhits 0 time 117 pv h7h5 d2g5 b8d7 g2g4 f6g4 g5e7 f8e8 e7d8 e8e2

SF12 + Toga 0.5
info depth 12 seldepth 16 multipv 1 score cp 51 nodes 48738 nps 288390 tbhits 0 time 169 pv b8d7 c1b1 d7b6 e2d3 f8e8 f3e5 b6d5 h1f1 d5c3 d2c3 d8d5

Toga III (same net)
info multipv 1 depth 12 seldepth 28 score cp 65 time 2222 nodes 453428 pv b8d7 a2a3 a8c8 c1b1 f8e8 g2g4 f6g4 h4h5 d7f6 h5g6 h7g6 d1g1 d8d7 d2g5

[twobeer]

Or another way to put it..

Why is
Crystal,
CFish
Fisherov
RaubFish
Eman
Sugar AI
Corchess

etc.. exluded on the list..

[dkappe]

Or another way to put it..

Why is
Crystal,
CFish
Fisherov
RaubFish
Eman
Sugar AI
Corchess

etc.. exluded on the list..

No idea. You’d have to ask CCRL. But you’ve strayed away from chess again and into asking questions about rating lists.

[dkappe]

I challenge you to write an engine yourself to use these weight files that would beat std stockfish even without using any nets at all for eval.. You and this A. Silver guy cannot of course do that, so what do you do? You rip of SF and then try to make a grab the money and run scheme before people realise the emperor has no clothes...

I have written a few engines. Right now I’m exploring hybrid mcts/ab with a0lite, which incorporates mcts/nn and ab/nnue. I doubt it will ever catch stockfish, however. I’m having a hard enough time catching xiphos. It is encouraging that the mcts+ab hybrid is now stronger than each search technique on its own.

And you do realize that almost anyone, using the training cookbook for SF could train a NNUE-net in 24h that together with SF Dev would beat your A0 Lite running the best Network out there?. right?

And Still claim the Network is more important than the Engine ... pffft.

My engine is bigger than your engine? I myself have trained some pretty strong nets. I’m pretty sure a0lite can’t beat Dragon either. Does that mean I’ve pwnd myself?

[Modern Times]

Why not just upgrade the old Fat Fritz 1.0 engine with this network .. Or bringa a truly unique engine to the table as Fat Fritz, instead of just ripping of the best Open Source one...

They certainly had Fritz 17 which I assume they could have modified and enhanced to use their new CPU based NNUE.