Stoofvlees II d2 is Killing Stockfish in TCEC

[Viz]

But what [larger board] has to do with "rewarding knowledge"?

Roughly speaking, the size of the game tree rises exponentially with the size of the board (this is discussed, with evidence, in the "Fruit fly races on steroids?" thread - link). In the kind of position in which you're used to getting, say, 10 ply full width and 30 ply selective search, you will now be lucky to get 3 ply full width and 10 ply selective search.

So... if you rely on search for most of your information about the position, then... oh dear - it's gone! Even having the shiniest graphics card on offer won't resolve that.

You're going to be relying on knowledge to enable you to play a good game!

...making a good NN via selfplay is mostly about how much hardware you can pour in it...

If you think that NNs are the only way to encode knowledge, then today's cool kid will be tomorrow's dinosaur!

The thing is that the bigger playing field is the more you benefit from larger NN.
More or less proven, in 3x3 tic tac toe you need to NN, in chess NNUE kinda beats deep net, in go deep nets are insane.
So yes, NNs are the only way to encode knowledge.
The only times where handcrafted stuff beats NNs are in things that can be solved fully or in search - but definitely not in evaluation of position, period.
So enlarging playing field is more or less making NNs even more important than they are in chess.
Not to mention that you make some random game instead of chess, and this is forum about you know, actual chess.

[towforce]

But what [larger board] has to do with "rewarding knowledge"?

Roughly speaking, the size of the game tree rises exponentially with the size of the board (this is discussed, with evidence, in the "Fruit fly races on steroids?" thread - link). In the kind of position in which you're used to getting, say, 10 ply full width and 30 ply selective search, you will now be lucky to get 3 ply full width and 10 ply selective search.

So... if you rely on search for most of your information about the position, then... oh dear - it's gone! Even having the shiniest graphics card on offer won't resolve that.

You're going to be relying on knowledge to enable you to play a good game!

...making a good NN via selfplay is mostly about how much hardware you can pour in it...

If you think that NNs are the only way to encode knowledge, then today's cool kid will be tomorrow's dinosaur!

The thing is that the bigger playing field is the more you benefit from larger NN.
More or less proven, in 3x3 tic tac toe you need to NN, in chess NNUE kinda beats deep net, in go deep nets are insane.
So yes, NNs are the only way to encode knowledge.
The only times where handcrafted stuff beats NNs are in things that can be solved fully or in search - but definitely not in evaluation of position, period.
So enlarging playing field is more or less making NNs even more important than they are in chess.
Not to mention that you make some random game instead of chess, and this is forum about you know, actual chess.

Yes - very good points!

I am alone in this quest (like the grand computer chess men of old in the times of mainframe computers and 8-bit CPUs), but I'm hunting for deeper patterns in chess. This has been discussed in depth, but briefly:

* I believe that NNs are encoding a large number of shallow (simple) chess patterns, and very few deep (complex) ones. The strongest evidence for this was that when LC0 first came out, she was playing beautiful chess in the middle game, but was relatively weaker in the endgame

* Unlike, say SAT (the Boolean satisfiability problem), there are a lot of symmetries in chess, and a lot of dependency between the variables (in the sense that chess variables are not independent from each other)

* Therefore, there are going to be some important deep patterns

In human chess, we start by learning simple patterns. Over time, we unconsciously learn deeper patterns (I have seen, in person, GMs look at a chess position and understand it in a heartbeat). Engine NNs learn by looking at billions of positions and their evaluations (many more positions that a human will see in a lifetime). Humans learn by picking a small number of positions (in comparison to the engine NNs) and then analysing these positions in depth: a different process.

For me, the probability that GMs have encoded the best and most important patterns in their brains is close to zero. If we can find these patterns, I see the following benefits:

1. We'll be able to create chess engines that play very well and very quickly on cheap computers (or in a web browser)

2. We might be able to reverse engineer it, come up with a way to teach it, and thus enable "ordinary people" to quickly and easily learn how to play good chess

3. Once we can do that for chess, we can do it in lots of other areas as well. We can have a whole world full of great people!

[Jouni]

I didn't remember, that this Stoofvlees is current world champion: Stoofvlees wins ICGA World Computer Chess Championship 2023 . Stockfish has never won . But Is Stoofvlees Lc0 clone?

[Uri Blass]

But what [larger board] has to do with "rewarding knowledge"?

Roughly speaking, the size of the game tree rises exponentially with the size of the board (this is discussed, with evidence, in the "Fruit fly races on steroids?" thread - link). In the kind of position in which you're used to getting, say, 10 ply full width and 30 ply selective search, you will now be lucky to get 3 ply full width and 10 ply selective search.

So... if you rely on search for most of your information about the position, then... oh dear - it's gone! Even having the shiniest graphics card on offer won't resolve that.

You're going to be relying on knowledge to enable you to play a good game!

...making a good NN via selfplay is mostly about how much hardware you can pour in it...

If you think that NNs are the only way to encode knowledge, then today's cool kid will be tomorrow's dinosaur!

The thing is that the bigger playing field is the more you benefit from larger NN.
More or less proven, in 3x3 tic tac toe you need to NN, in chess NNUE kinda beats deep net, in go deep nets are insane.
So yes, NNs are the only way to encode knowledge.
The only times where handcrafted stuff beats NNs are in things that can be solved fully or in search - but definitely not in evaluation of position, period.
So enlarging playing field is more or less making NNs even more important than they are in chess.
Not to mention that you make some random game instead of chess, and this is forum about you know, actual chess.

Yes - very good points!

I am alone in this quest (like the grand computer chess men of old in the times of mainframe computers and 8-bit CPUs), but I'm hunting for deeper patterns in chess. This has been discussed in depth, but briefly:

* I believe that NNs are encoding a large number of shallow (simple) chess patterns, and very few deep (complex) ones. The strongest evidence for this was that when LC0 first came out, she was playing beautiful chess in the middle game, but was relatively weaker in the endgame

* Unlike, say SAT (the Boolean satisfiability problem), there are a lot of symmetries in chess, and a lot of dependency between the variables (in the sense that chess variables are not independent from each other)

* Therefore, there are going to be some important deep patterns

In human chess, we start by learning simple patterns. Over time, we unconsciously learn deeper patterns (I have seen, in person, GMs look at a chess position and understand it in a heartbeat). Engine NNs learn by looking at billions of positions and their evaluations (many more positions that a human will see in a lifetime). Humans learn by picking a small number of positions (in comparison to the engine NNs) and then analysing these positions in depth: a different process.

For me, the probability that GMs have encoded the best and most important patterns in their brains is close to zero. If we can find these patterns, I see the following benefits:

1. We'll be able to create chess engines that play very well and very quickly on cheap computers (or in a web browser)

2. We might be able to reverse engineer it, come up with a way to teach it, and thus enable "ordinary people" to quickly and easily learn how to play good chess

3. Once we can do that for chess, we can do it in lots of other areas as well. We can have a whole world full of great people!

I see no reason to believe that NN encoded many simple patterns and it seems to me that they encoded in the beginning mainly complex patterns.

The simpler patterns are in the endgame and not in the middlegame so
the fact that LC0 was playing beautiful chess in the middle game, but was relatively weaker in the endgame only support my theory that she was type of genius that understood complex things in the middle game that even top GM did not understand but did not understand simple things about the endgame that even weak humans player understand.

Of course now when NN became better they learned also a lot of simple stuff that human also understand and are superior relative to humans in all patterns.

[dkappe]

I didn't remember, that this Stoofvlees is current world champion: Stoofvlees wins ICGA World Computer Chess Championship 2023 . Stockfish has never won . But Is Stoofvlees Lc0 clone?

Are you a ChessQuack clone? Do a little reading about the author and the history of the engine before spouting such uninformed speculation.

P.S. The author Gian-Carlo Pascutto is also the author of Leela, you know, the original go program?

[Dann Corbit]

I am pretty sure he was joking.

BTW, they do test the openings before the contest (or at least, I know some book-writers who had openings in TCEC and carefully tested them before-hand). The problem is analysis effort. The TCEC hardware is very high end and the time control is fairly slow. It takes months to run the contest on this high-end stuff. So there is no way to test all the combinations at the same time control and hardware strength.

In the test contests, the openings were not busted. But when you give the strong hardware a long time to think, it can find defects and break the opening beyond repair. IOW, the openings are broken by the actual tournament play.

Broken openings obviously inflate the strength of weak programs because they will get a win when normally they would not be able to. But the percentage of broken openings is low, so the contest is not affected very much. It can also soothe the psyche of the weaker engine authors, by giving them wins against SF or LC0.

[towforce]

I see no reason to believe that NN encoded many simple patterns and it seems to me that they encoded in the beginning mainly complex patterns.

The simpler patterns are in the endgame and not in the middlegame so
the fact that LC0 was playing beautiful chess in the middle game, but was relatively weaker in the endgame only support my theory that she was type of genius that understood complex things in the middle game that even top GM did not understand but did not understand simple things about the endgame that even weak humans player understand.

Of course now when NN became better they learned also a lot of simple stuff that human also understand and are superior relative to humans in all patterns.

1. The patterns in the endgame are deep, but the positions are simple enough for humans to not get confused by interacting patterns

2. Given the way NNs are trained, how are they going to find and encode deep patterns, other than "only occasionally" and "by accident"?

3. If (1) and (2) haven't convinced you, then here's the killer evidence: everyone had assumed that Go engines are now so good that, going forward, no human would ever challenge them. We were all wrong: it turns out that they cannot see things that are simple and obvious to humans - they didn't understand the concept of a group of stones - and it doesn't get any more basic than that in Go! Suddenly, it became clear why it's such a struggle to get computers to drive cars well - something that many stupid humans have no problem doing. They have terrible blind spots because they're encoding a large number of simple patterns, but missing most of the deeper ones. Armed with the knowledge of Go engine blind spots, an amateur player was able to consistently beat a Go program that had been thought to be out of reach of any human:

https://www.vice.com/en/article/v7v5xb/ ... mple-trick

I use NNs on a daily basis (I use LLMs multiple times every day). They are phenomenal, they are very useful, and they are a lot better than anything else we have today!

But they're stochastic parrots, and achieve their effects (and, in my life, their value) by encoding large numbers of surface patterns in the knowledge area: they're missing a lot of important deeper patterns, which could add a lot of value if they could be found, encoded, and included.

[Uri Blass]

I see no reason to believe that NN encoded many simple patterns and it seems to me that they encoded in the beginning mainly complex patterns.

The simpler patterns are in the endgame and not in the middlegame so
the fact that LC0 was playing beautiful chess in the middle game, but was relatively weaker in the endgame only support my theory that she was type of genius that understood complex things in the middle game that even top GM did not understand but did not understand simple things about the endgame that even weak humans player understand.

Of course now when NN became better they learned also a lot of simple stuff that human also understand and are superior relative to humans in all patterns.

1. The patterns in the endgame are deep, but the positions are simple enough for humans to not get confused by interacting patterns

2. Given the way NNs are trained, how are they going to find and encode deep patterns, other than "only occasionally" and "by accident"?

3. If (1) and (2) haven't convinced you, then here's the killer evidence: everyone had assumed that Go engines are now so good that, going forward, no human would ever challenge them. We were all wrong: it turns out that they cannot see things that are simple and obvious to humans - they didn't understand the concept of a group of stones - and it doesn't get any more basic than that in Go! Suddenly, it became clear why it's such a struggle to get computers to drive cars well - something that many stupid humans have no problem doing. They have terrible blind spots because they're encoding a large number of simple patterns, but missing most of the deeper ones. Armed with the knowledge of Go engine blind spots, an amateur player was able to consistently beat a Go program that had been thought to be out of reach of any human:

https://www.vice.com/en/article/v7v5xb/ ... mple-trick

I use NNs on a daily basis (I use LLMs multiple times every day). They are phenomenal, they are very useful, and they are a lot better than anything else we have today!

But they're stochastic parrots, and achieve their effects (and, in my life, their value) by encoding large numbers of surface patterns in the knowledge area: they're missing a lot of important deeper patterns, which could add a lot of value if they could be found, encoded, and included.

I find that the NN simply miss simple tactical patterns also in the middle game.
White is probably a super GM level engine but it did not help it to avoid 14.Qd2

The tactics with 14...Rh1+ is something that I expect most humans players with fide rating 2000 to notice immediately because the mate is based on a known pattern.

White is already losing at that point but at least it could play 14.Re1 that does not get a forced mate when black has a decisive material advantage after 14...Rh1+ 15.Kxh1 Nf2+

With clear hash berserk13 needs less nodes to see Rh1+ but it still need many thousands of nodes.

[pgn][Event "Computer event"]
[Site "Somewhere on Earth"]
[Date "2024.05.01"]
[Time "05:02:33"]
[Round "12"]
[Board "21"]
[White "Berserk 13 65536 nodes"]
[Black "Stockfish 16.1"]
[Result "0-1"]
[Termination "mate"]
[ECO "C45"]
[Opening "Scotch"]
[TimeControl "nodes: 65536;60+1"]
[PlyCount "34"]

1.e4 {+0.5/13 39 44664 220/749/31} e5 {-0.3/33 8415 23258381} 2.Nf3 {+0.5/13 41 60329 206/764/30} Nc6 {-0.3/27 763 2503541}
3.d4 {+0.4/13 39 35169 183/787/30} exd4 {-0.1/21 1080 2871738} 4.Nxd4 {+0.4/13 44 35950 182/791/27} Nf6 {-0.1/22 931 2557183}
5.Nxc6 {+0.7/13 43 53028 269/717/14; C45: Scotch, Schmidt variation} bxc6 {-0.1/22 1178 3210276} 6.Bd3 {+0.2/12 42 53076 96/863/41} d5 {+0.0/23 1001 2842382}
7.e5 {+0.2/12 42 25414 99/867/34} Ng4 {+0.4/23 999 2787568} 8.O-O {+0.5/14 41 43197 197/790/13} Bc5 {+0.4/23 1875 5331561}
9.Bf4 {+0.0/11 38 61869 45/898/57} g5 {+0.4/25 2016 5786227} 10.Bg3 {-0.1/12 39 23717 40/906/54} h5 {+1.1/22 1268 3295191}
11.b4 {-0.6/12 39 27316 7/770/223} h4 {+2.5/21 1024 2678885} 12.bxc5 {-0.7/14 41 29632 5/741/254} hxg3 {+2.9/21 1297 3423031}
13.hxg3 {-0.7/15 40 54647 4/746/250} Ke7 {+3.9/22 1097 3934982} 14.Qd2 {+0.5/10 39 15915 170/824/6} Rh1+ {M+4/245 282 5450494}
15.Kxh1 {M-3/69 24 65402 0/0/1000} Qh8+ {M+3/245 35 62957} 16.Bh7 {M-2/200 37 22083 0/0/1000} Qxh7+ {M+2/245 22 31493}
17.Kg1 {M-1/200 33 601 0/0/1000} Qh2# {M+1/245 77 35310} 0-1

[/pgn]

[towforce]

I find that the NN simply miss simple tactical patterns also in the middle game.
White is probably a super GM level engine but it did not help it to avoid 14.Qd2

The tactics with 14...Rh1+ is something that I expect most humans players with fide rating 2000 to notice immediately because the mate is based on a known pattern.

White is already losing at that point but at least it could play 14.Re1 that does not get a forced mate when black has a decisive material advantage after 14...Rh1+ 15.Kxh1 Nf2+

With clear hash berserk13 needs less nodes to see Rh1+ but it still need many thousands of nodes.

[pgn][Event "Computer event"]
[Site "Somewhere on Earth"]
[Date "2024.05.01"]
[Time "05:02:33"]
[Round "12"]
[Board "21"]
[White "Berserk 13 65536 nodes"]
[Black "Stockfish 16.1"]
[Result "0-1"]
[Termination "mate"]
[ECO "C45"]
[Opening "Scotch"]
[TimeControl "nodes: 65536;60+1"]
[PlyCount "34"]

1.e4 {+0.5/13 39 44664 220/749/31} e5 {-0.3/33 8415 23258381} 2.Nf3 {+0.5/13 41 60329 206/764/30} Nc6 {-0.3/27 763 2503541}
3.d4 {+0.4/13 39 35169 183/787/30} exd4 {-0.1/21 1080 2871738} 4.Nxd4 {+0.4/13 44 35950 182/791/27} Nf6 {-0.1/22 931 2557183}
5.Nxc6 {+0.7/13 43 53028 269/717/14; C45: Scotch, Schmidt variation} bxc6 {-0.1/22 1178 3210276} 6.Bd3 {+0.2/12 42 53076 96/863/41} d5 {+0.0/23 1001 2842382}
7.e5 {+0.2/12 42 25414 99/867/34} Ng4 {+0.4/23 999 2787568} 8.O-O {+0.5/14 41 43197 197/790/13} Bc5 {+0.4/23 1875 5331561}
9.Bf4 {+0.0/11 38 61869 45/898/57} g5 {+0.4/25 2016 5786227} 10.Bg3 {-0.1/12 39 23717 40/906/54} h5 {+1.1/22 1268 3295191}
11.b4 {-0.6/12 39 27316 7/770/223} h4 {+2.5/21 1024 2678885} 12.bxc5 {-0.7/14 41 29632 5/741/254} hxg3 {+2.9/21 1297 3423031}
13.hxg3 {-0.7/15 40 54647 4/746/250} Ke7 {+3.9/22 1097 3934982} 14.Qd2 {+0.5/10 39 15915 170/824/6} Rh1+ {M+4/245 282 5450494}
15.Kxh1 {M-3/69 24 65402 0/0/1000} Qh8+ {M+3/245 35 62957} 16.Bh7 {M-2/200 37 22083 0/0/1000} Qxh7+ {M+2/245 22 31493}
17.Kg1 {M-1/200 33 601 0/0/1000} Qh2# {M+1/245 77 35310} 0-1

[/pgn]

Very nice example - thank you for that!

[Ras]

2. To find out which chess engines are the best. For this, you really need bigger boards (12x12 might be a good starting point).

Translation: in order to find the best chess engine, we have to play some game other than chess. Doesn't make sense.