Commercial Release of Ethereal 13.00 (NNUE) for AVX2 Systems

[CMCanavessi]

Well, 1st 2 games with the avx binary instead of avx2, 1 loss on time and 1 "stalled connection"

[pgn][Event "FRC Testing"]
[Site "CCLS"]
[Date "2021.06.18"]
[Round "6"]
[White "Houdini 6.03 Pro x64"]
[Black "Ethereal 13.00 x64"]
[Result "1-0"]
[FEN "rnbbqnkr/pppppppp/8/8/8/8/PPPPPPPP/RNBBQNKR w KQkq - 0 1"]
[GameDuration "00:06:06"]
[GameEndTime "2021-06-18T13:11:32.327 Hora estándar de Argentina"]
[GameStartTime "2021-06-18T13:05:25.715 Hora estándar de Argentina"]
[PlyCount "39"]
[SetUp "1"]
[Termination "time forfeit"]
[TimeControl "180+2"]
[Variant "fischerandom"]

1. d4 {+0.23/21 11.60s} d5 {-0.18/22 6.92s} 2. e4 {+0.37/19 3.09s}
dxe4 {+0.06/22 7.33s} 3. Qxe4 {+0.30/20 2.91s} Nbd7 {-0.04/22 3.11s}
4. Qe2 {+0.27/21 11.68s} e6 {+0.04/23 12.90s} 5. c3 {+0.20/21 28.33s}
c5 {-0.10/23 10.46s} 6. Bc2 {+0.16/21 1.12s} Ng6 {+0.01/23 11.65s}
7. Ng3 {+0.31/19 4.25s} cxd4 {+0.35/24 9.27s} 8. cxd4 {+0.44/19 6.46s}
Nb6 {-0.03/24 15.99s} 9. Nc3 {+0.14/21 7.71s} Bd7 {+0.15/22 4.04s}
10. O-O {+0.04/20 7.74s} O-O {-0.11/24 11.69s} 11. Re1 {+0.09/20 4.41s}
Bc6 {+0.27/21 3.74s} 12. Nge4 {+0.18/20 2.24s} Qd7 {+0.20/22 4.46s}
13. g3 {+0.03/19 11.31s} Qxd4 {+0.38/24 8.09s} 14. Be3 {+0.24/21 4.75s}
Qe5 {+0.66/23 0.81s} 15. Rad1 {-0.01/20 10.58s} Be7 {+0.84/26 1.09s}
16. Bd4 {-0.12/20 14.60s} Qc7 {-0.01/27 9.81s} 17. h4 {-0.22/21 12.55s}
Rad8 {+0.67/30 5.50s} 18. a4 {-0.22/22 0.90s} h6 {+0.57/24 3.91s}
19. Nb5 {-0.22/22 0s} Bxb5 {+0.57/28 8.46s}
20. axb5 {-0.21/24 0s, Black loses on time} 1-0

[Event "FRC Testing"]
[Site "CCLS"]
[Date "2021.06.18"]
[Round "6"]
[White "Ethereal 13.00 x64"]
[Black "Houdini 6.03 Pro x64"]
[Result "0-1"]
[FEN "rnbbqnkr/pppppppp/8/8/8/8/PPPPPPPP/RNBBQNKR w KQkq - 0 1"]
[GameDuration "00:01:37"]
[GameEndTime "2021-06-18T13:13:09.748 Hora estándar de Argentina"]
[GameStartTime "2021-06-18T13:11:32.337 Hora estándar de Argentina"]
[PlyCount "12"]
[SetUp "1"]
[Termination "stalled connection"]
[TimeControl "180+2"]
[Variant "fischerandom"]

1. d4 {+0.21/22 8.66s} e5 {-0.10/22 11.52s} 2. dxe5 {+0.37/22 6.56s}
Qxe5 {-0.23/23 5.92s} 3. c4 {+0.42/23 2.09s} Ne6 {-0.15/18 6.96s}
4. Nc3 {+0.71/23 9.40s} h5 {-0.32/20 7.16s} 5. Nd2 {+0.58/21 4.80s}
g5 {-0.07/18 3.00s} 6. Nd5 {+0.58/23 16.09s}
c6 {-0.37/23 0s, White's connection stalls} 0-1[/pgn]

I really don't have a clue of what's going on... Ethereal has always been one of the best engines, stability-wise in my system.

[AndrewGrant]

SCOOP:
Ethereal 13.00 NNUE github works with Fat Fritz 2.0 NET, so we can test it deeply

Hmm, no. The NNUE code is not even compiled in the Github versions. You may have set a network file, but it does not run.
For the final time: No one has networks that can run in Ethereal except me.

[AndrewGrant]

Hey Andrew, I got 2 loses on time in 8 games (so far), using Cute Chess, FRC, 3 min + 2 sec, 1 thread, ponder on, 1024 hash, non-pext avx2 binary. Tournament is still ongoing in my stream if you wanna take a look. Ethereal was comfortably winning both games against Shredder 13 and ended up losing on time. After that, it didn't happen again.

Also, a different thing: i was benchmarking avx vs avx2 vs ssse3 (all non-pext) with the go infinite command, all 3 at the same time and looking at the output. All 3 were showing the same ammount of nodes for every depth it was printing, until around depth 30 where avx2 started doing things different. avx and ssse3 kept printing the same node count, but avx2 was different. Is this behaviour normal?

Another loss on time, 3rd in 10 games, this time Vs. SlowChessBlitz 2.5
I have now replaced the avx2 binary with the avx one. We'll see what happens and if that fixes it.

Nodes will change at a high enough bench. There are floating point operations, and eventually the paths will differ due to tiny differences in error buildups from floating operations.

I'm not sure why you are seeing issues. You are the first person to have any issues. You are pondering, so I guess I can look at that quickly, but there are no issues at TCEC, Rating Lists, or the hundreds of thousands of games just played on OpenBench. So I'm not sure where to go from there.

I'm also assuming you are using the ones from Github, which are not even NNUE so the avx2/avx/ssse3 stuff hardly matters. I still have a smaller NN that runs in all versions, but its not vectorized by hand.

I guess the questions would be: What GUI are you using? Do you have any logs of the time losses? Do you see the issue when pondering is disabled?

You can also always increase the MoveOverhead option. IF your machine is slow, or has high latency due to other workloads, this may help. Right now its 300ms which is extremely large, even enough for Syzygy on the HDD, but at tournaments I use like 2500ms.

[CMCanavessi]

I guess the questions would be: What GUI are you using? Do you have any logs of the time losses? Do you see the issue when pondering is disabled?

You can also always increase the MoveOverhead option. IF your machine is slow, or has high latency due to other workloads, this may help. Right now its 300ms which is extremely large, even enough for Syzygy on the HDD, but at tournaments I use like 2500ms.

Using CuteChess GUI at the moment, will try later with Arena, but that will be with normal chess, not FRC. As this happened last night and today, I still have no logs (I had them disabled). I'll see in a couple of days if I can get some for you, I'll run only Ethereal matches. Will try with and without ponder. No TBs also, so that's out of the equation.

[AndrewGrant]

I guess the questions would be: What GUI are you using? Do you have any logs of the time losses? Do you see the issue when pondering is disabled?

You can also always increase the MoveOverhead option. IF your machine is slow, or has high latency due to other workloads, this may help. Right now its 300ms which is extremely large, even enough for Syzygy on the HDD, but at tournaments I use like 2500ms.

Using CuteChess GUI at the moment, will try later with Arena, but that will be with normal chess, not FRC. As this happened last night and today, I still have no logs (I had them disabled). I'll see in a couple of days if I can get some for you, I'll run only Ethereal matches. Will try with and without ponder. No TBs also, so that's out of the equation.

Okay, I'll be ready when you have more info. I'm sure we can figure something out, as Ethereal is not known for having these sorts of issues.

[CMCanavessi]

I started a tournament with Ethereal and Shredder only, and lowered the time control to 1m + 1s so it goes faster... 2nd game, Ethereal with black pieces, lost on time. Got the log, don't see anything wrong there... it may be a cutechess issue?

Log here: https://file.io/RgQLAIOBefT5

[AndrewGrant]

I started a tournament with Ethereal and Shredder only, and lowered the time control to 1m + 1s so it goes faster... 2nd game, Ethereal with black pieces, lost on time. Got the log, don't see anything wrong there... it may be a cutechess issue?

Log here: https://file.io/RgQLAIOBefT5

Can you start by increasing the MoveOverhead setting to 1000? That log has Ethereal going over the timer by about 200ms. That really surprises me, as Ethereal checks the clock every 1024 nodes, and you have resonable NPS.

Also, what exactly is your setup here? Is this going to be a case of CCC season 1? How many cores/threads does your machine have, and how many are you giving to either engine.

[AndrewGrant]

I started a tournament with Ethereal and Shredder only, and lowered the time control to 1m + 1s so it goes faster... 2nd game, Ethereal with black pieces, lost on time. Got the log, don't see anything wrong there... it may be a cutechess issue?

Log here: https://file.io/RgQLAIOBefT5

Can you start by increasing the MoveOverhead setting to 1000? That log has Ethereal going over the timer by about 200ms. That really surprises me, as Ethereal checks the clock every 1024 nodes, and you have resonable NPS.

Also, what exactly is your setup here? Is this going to be a case of CCC season 1? How many cores/threads does your machine have, and how many are you giving to either engine.

No need -- problem identified and solved. Pushing a fix to the Github now, and will update the binaries in just a few minutes.
Ethereal would eat a "ponderhit" message if it came while Ethereal was still preparing the search after getting a go command.

EDIT 1: Updated: https://github.com/AndyGrant/Ethereal/r ... tag/v13.00

EDIT 2: If you saw this thread the instant I posted it ... you downloaded a version that prints 'test' sometimes. Oops, fixed now though, but hopefully you are not sitting here ctrl+r'ing this page.

[CMCanavessi]

I started a tournament with Ethereal and Shredder only, and lowered the time control to 1m + 1s so it goes faster... 2nd game, Ethereal with black pieces, lost on time. Got the log, don't see anything wrong there... it may be a cutechess issue?

Log here: https://file.io/RgQLAIOBefT5

Can you start by increasing the MoveOverhead setting to 1000? That log has Ethereal going over the timer by about 200ms. That really surprises me, as Ethereal checks the clock every 1024 nodes, and you have resonable NPS.

Also, what exactly is your setup here? Is this going to be a case of CCC season 1? How many cores/threads does your machine have, and how many are you giving to either engine.

No need -- problem identified and solved. Pushing a fix to the Github now, and will update the binaries in just a few minutes.
Ethereal would eat a "ponderhit" message if it came while Ethereal was still preparing the search after getting a go command.

EDIT 1: Updated: https://github.com/AndyGrant/Ethereal/r ... tag/v13.00

EDIT 2: If you saw this thread the instant I posted it ... you downloaded a version that prints 'test' sometimes. Oops, fixed now though, but hopefully you are not sitting here ctrl+r'ing this page.

Great! Glad I could be of help to solve an issue. What's weird though, is that after using CuteChess and experiencing the problem, I went to Banksia 0.45 and started a tournament with the exact same configuration as in CuteChess... so far, 6 played games and not a single issue. Maybe Banksia and CuteChess handle ponder differently?

With Banksia:

Code: Select all

1) Ethereal 13.00 (POPCNT) vs Shredder 13 x64, #129, 1-0 (mate)
2) Shredder 13 x64 vs Ethereal 13.00 (POPCNT), #164, 0-1 (mate)
3) Ethereal 13.00 (POPCNT) vs Shredder 13 x64, #87, 1/2-1/2 (repetition)
4) Shredder 13 x64 vs Ethereal 13.00 (POPCNT), #140, 0-1 (mate)
5) Ethereal 13.00 (POPCNT) vs Shredder 13 x64, #166, 0-1 (mate)
6) Shredder 13 x64 vs Ethereal 13.00 (POPCNT), #158, 1/2-1/2 (repetition)

BTW my setup is a Ryzen 7 1700 (8 cores, 16 threads) with 32 gigs of Ram. But I'm using only 1 thread and only 1 gb of hash, so it should never be a problem of resources or latency. Only crappy GUIs

[AndrewGrant]

I started a tournament with Ethereal and Shredder only, and lowered the time control to 1m + 1s so it goes faster... 2nd game, Ethereal with black pieces, lost on time. Got the log, don't see anything wrong there... it may be a cutechess issue?

Log here: https://file.io/RgQLAIOBefT5

Can you start by increasing the MoveOverhead setting to 1000? That log has Ethereal going over the timer by about 200ms. That really surprises me, as Ethereal checks the clock every 1024 nodes, and you have resonable NPS.

Also, what exactly is your setup here? Is this going to be a case of CCC season 1? How many cores/threads does your machine have, and how many are you giving to either engine.

No need -- problem identified and solved. Pushing a fix to the Github now, and will update the binaries in just a few minutes.
Ethereal would eat a "ponderhit" message if it came while Ethereal was still preparing the search after getting a go command.

EDIT 1: Updated: https://github.com/AndyGrant/Ethereal/r ... tag/v13.00

EDIT 2: If you saw this thread the instant I posted it ... you downloaded a version that prints 'test' sometimes. Oops, fixed now though, but hopefully you are not sitting here ctrl+r'ing this page.

Great! Glad I could be of help to solve an issue. What's weird though, is that after using CuteChess and experiencing the problem, I went to Banksia 0.45 and started a tournament with the exact same configuration as in CuteChess... so far, 6 played games and not a single issue. Maybe Banksia and CuteChess handle ponder differently?

With Banksia:

Code: Select all

1) Ethereal 13.00 (POPCNT) vs Shredder 13 x64, #129, 1-0 (mate)
2) Shredder 13 x64 vs Ethereal 13.00 (POPCNT), #164, 0-1 (mate)
3) Ethereal 13.00 (POPCNT) vs Shredder 13 x64, #87, 1/2-1/2 (repetition)
4) Shredder 13 x64 vs Ethereal 13.00 (POPCNT), #140, 0-1 (mate)
5) Ethereal 13.00 (POPCNT) vs Shredder 13 x64, #166, 0-1 (mate)
6) Shredder 13 x64 vs Ethereal 13.00 (POPCNT), #158, 1/2-1/2 (repetition)

BTW my setup is a Ryzen 7 1700 (8 cores, 16 threads) with 32 gigs of Ram. But I'm using only 1 thread and only 1 gb of hash, so it should never be a problem of resources or latency. Only crappy GUIs

It really comes down to how fast the ponderhit is sent. How long does it take Ethereal to start the search once getting go? microseconds? Yeah, its a tight window and I'm actually surprised that I could reproduce it. I don't fault Cutechess here (although they do have broken FRC in older versions, and maybe still not updated!) I accept responsibility when needed