Introducing Igel chess engine

[pohl4711]

Your sample size is too small You will need to run around 16k long time control games to get something precise, here is the run from updated regression:

http://chess.grantnet.us/test/9069/

Code: Select all

ELO   | 23.78 +- 3.56 (95%)
SPRT  | 60.0+0.6s Threads=1 Hash=64MB
LLR   | 0.00 (-2.94, 2.94) [0.00, 0.00]
Games | N: 16112 W: 4102 L: 3001 D: 9009

AB-testrun of Igel 2.9.0 finished (7000 games)

https://www.sp-cc.de

(Perhaps you have to clear your browsercache or reload the website)

[voffka]

AB-testrun of Igel 2.9.0 finished (7000 games)

https://www.sp-cc.de

(Perhaps you have to clear your browsercache or reload the website)

Great! Thank you for the test pohl4711!

[voffka]

I finished my regression testing between Igel and top 10 engines, the results are quite interesting. This time I used a short time control (20s and 0.2s increment) with many games to make sure elo estimate is accurate.

In this time control the elo gain of Igel 2.9.0 is around 95 elo over 2.8.0 which is mind blowing:

Igel 2.8.0 BMI2 AVX2 vs top 10 engines in tc=all/20+0.2:

Code: Select all

Rank Name                          Elo     +/-   Games   Score   Draws
   0 Igel 2.8.0 64 BMI2 AVX2      -179       7    6000   26.3%   38.1%
   1 Stockfish 12 64-bit           458      30    1000   93.3%   13.0%
   2 Komodo 14 64-bit              208      17    1000   76.8%   39.7%
   3 Ethereal 12.75 64-bit         173      16    1000   73.0%   42.7%
   4 Fire 7.1 64-bit               130      17    1000   67.8%   41.1%
   5 Xiphos 0.6 64-bit             121      16    1000   66.7%   47.6%
   6 Komodo 14 MCTS 64-bit         103      16    1000   64.3%   44.3%

Igel 2.9.0 BMI2 AVX2 vs top 10 engines in tc=all/20+0.2:

Code: Select all

Rank Name                          Elo     +/-   Games   Score   Draws
   0 Igel 2.9.0 64 BMI2 AVX2       -84       7    6000   38.2%   40.8%
   1 Stockfish 12 64-bit           387      25    1000   90.3%   18.3%
   2 Komodo 14 64-bit              122      16    1000   66.9%   46.4%
   3 Ethereal 12.75 64-bit          83      16    1000   61.8%   46.9%
   4 Fire 7.1 64-bit                21      16    1000   53.0%   46.3%
   5 Xiphos 0.6 64-bit               5      16    1000   50.7%   45.0%
   6 Komodo 14 MCTS 64-bit         -13      16    1000   48.2%   41.6%

I attribute this improvement to the use of the IGN network which was trained on Igel own search and eval.

[voffka]

Igel 3.0.0 at https://github.com/vshcherbyna/igel/releases/tag/3.0.0 (official executable binaries for Windows and IGN net).

This release brings significant improvements in search, move ordering, time management and evaluation. Strength increase due to search/move ordering improvements is around 40 elo, strength increase due to new network is 20 elo more.

The new network ign-1-139b702b is trained using nnue-pytorch on 10.5 billions of positions using data from Igel 2.6.0 (HCE) and Igel 2.9.0 (NNUE).

Enormous amount of computational power has been used to test this build on OpenBench, I would like to thank Andrew Grant and Bojun Guo (noobpwnftw) for supporting Igel project!

I would like to also thank authors of nnue-pytorch project for helping to make this release possible and more or less 'hassle free' training experience!

P.S. I decided to remove 'EvalFile' option from uci parameters and embed network file into executable by default: there were too many users on Igel 2.9.0 having all sort of problems with that option and being confused if network is embedded or not. I hope it will be simpler as of Igel 3.0.0.

What's new:

- Train a new network using Igel 2.6.0 (HCE) and Igel 2.9.0 (NNUE) data: ign-1-139b702b
- Remove 'EvalFile' option - as of Igel 3.0.0 the network file is always embedded in the executable
- Fix in check extensions
- Use incbin on Linux to inject network file into the binary
- Tune null move prunning conditions
- Tune probcut prunning conditions
- Tune quiets prunning
- Fix bogus condition for history calculation
- Implement more aggressive null move prunning
- Stop extension of captures
- Prevent explosion of history extensions
- Implement better time management for sudden death time control
- Count not played quiets for LMP prunning
- Remove dummy 100 multiplier for history ordering calculation
- Fix compilation issues with gcc 10 and Android NDK's Clang
- Remove unused counter move table
- Fix out of boundary access for history array
- Fix issue with singular extensions - reported by ChizhovVadim (author of CounterGo chess engine)
- Fix crash when 'ucinewgame' command is not issued - reported by Ipmanchess
- Fix crash when thread position is not initialized - reported by Ipmanchess

Regression run against Igel 2.9.0:

Short Time Control No Increment

Code: Select all

ELO   | 115.05 +- 3.88 (95%)
SPRT  | 10.0+0.0s Threads=1 Hash=8MB
LLR   | 0.00 (-2.94, 2.94) [0.00, 0.00]
Games | N: 20652 W: 9913 L: 3314 D: 7425

Short Time Control With Increment

Code: Select all

ELO   | 75.88 +- 3.53 (95%)
SPRT  | 10.0+0.1s Threads=1 Hash=8MB
LLR   | 0.00 (-2.94, 2.94) [0.00, 0.00]
Games | N: 20904 W: 8061 L: 3567 D: 9276

Long Time Control No Increment

Code: Select all

ELO   | 83.16 +- 4.04 (95%)
SPRT  | 60.0+0.0s Threads=1 Hash=64MB
LLR   | 0.00 (-2.94, 2.94) [0.00, 0.00]
Games | N: 13508 W: 4814 L: 1641 D: 7053

Long Time Control With Increment

Code: Select all

ELO   | 58.22 +- 3.81 (95%)
SPRT  | 60.0+0.6s Threads=1 Hash=64MB
LLR   | 0.00 (-2.94, 2.94) [0.00, 0.00]
Games | N: 13540 W: 4023 L: 1775 D: 7742

[bastiball]

Hello mr. Igel

Congrats for the new release! and Goodluck!
Can you provide silvermont binaries? Thanks!




Regards.....

[Guenther]

Igel 3.0.0 at https://github.com/vshcherbyna/igel/releases/tag/3.0.0 (official executable binaries for Windows and IGN net).

This release brings significant improvements in search, move ordering, time management and evaluation. Strength increase due to search/move ordering improvements is around 40 elo, strength increase due to new network is 20 elo more.

The new network ign-1-139b702b is trained using nnue-pytorch on 10.5 billions of positions using data from Igel 2.6.0 (HCE) and Igel 2.9.0 (NNUE).

...

Just a little note, the makefile still points to the old network file.

Edit:
And I just cannot believe my eyes, 3.0.0 seems three! times faster now for old hardware? (ok before it was really slow with nn)

This one is just 2 commits after official 2.9.0 (compiled in 2021-01-04 - nn.bin is just renamed ign-0-9b1937cc)

Code: Select all

Igel 3.0-dev-0 64 by V. Medvedev, V. Shcherbyna
uci
id name Igel 3.0-dev-0 64
..
setoption name EvalFile value eval/nn.bin
Using EvalFile eval/nn.bin
go infinite
...
info depth 15 seldepth 28 score cp 40 time 7469 nodes 858496 tbhits 0 nps 114869 pv g1f3 d7d5 d2d4 c7c6 c2c4 g8f6 c4d5 c6d5 c1f4 g7g6 b1c3 f8g7 a1c1 e8g8 e2e3 b8c6 f1b5
info depth 16 seldepth 33 score cp 48 time 11051 nodes 1273026 tbhits 0 nps 114833 pv g1f3 d7d5 d2d4 g8f6 c2c4 e7e6 b1c3 c7c6 e2e3 f8d6 b2b3 e8g8 f1d3 b8d7 d1c2 a7a6 e1g1
info depth 17 seldepth 33 score cp 38 time 14313 nodes 1661963 tbhits 0 nps 115122 pv g1f3 d7d5 d2d4 g8f6 c2c4 e7e6 b1c3 d5c4 e2e3 c7c5 f1c4 b8c6 e1g1 c5d4 f3d4 c6d4 e3d4 f8e7 d4d5 e8g8 d5e6 c8e6
info depth 18 seldepth 33 score cp 40 time 18567 nodes 2156037 tbhits 0 nps 116034 pv g1f3 d7d5 d2d4 g8f6 c2c4 e7e6 b1c3 d5c4 e2e3 c7c5 f1c4 f8e7 e1g1 c5d4 e3d4 e8g8 c1f4 b8c6 d1d2 f8e8
info depth 19 seldepth 33 score cp 44 time 28316 nodes 3286963 tbhits 0 nps 116065 pv d2d4 g8f6 g1f3 d7d5 c2c4 e7e6 b1c3 f8e7 c4d5 e6d5 c1f4 e8g8 e2e3 c7c5 d4c5 e7c5 f1e2 b8c6 e1g1 f8e8 c3b5
info depth 20 seldepth 33 score cp 44 time 39551 nodes 4554611 tbhits 0 nps 116040 pv d2d4 g8f6 g1f3 d7d5 c2c4 e7e6 c1g5 f8e7 c4d5 e6d5 e2e3 e8g8 f1d3 c7c6 d1c2 b8d7 b1c3 a7a6 g5h4 f8e8 e1g1
info depth 21 seldepth 34 score cp 38 time 55818 nodes 6279974 tbhits 0 nps 115131 pv d2d4 g8f6 g1f3 d7d5 c2c4 e7e6 c1g5 f8e7 c4d5 e6d5 e2e3 e8g8 b1c3 h7h6 g5h4 f8e8 f1d3 c7c6 e1g1 c8g4 h4g3 b8d7 f1e1 d7b6

compiled 3.0.0 half an hour ago

Code: Select all

Igel 3.0.0 64 NOPOPCNT by V. Medvedev, V. Shcherbyna
uci
id name Igel 3.0.0 64 NOPOPCNT
...
info depth 20 seldepth 38 score cp 66 time 8208 nodes 2674374 tbhits 0 nps 320265 pv e2e4 c7c5 g1f3 e7e6 b1c3 a7a6 d2d4 c5d4 f3d4 b7b5 f1d3 c8b7 e1g1 b8c6 d4c6 b7c6 d1e2 b5b4 c3d1 d8c7 d1e3
info depth 21 seldepth 38 score cp 45 time 11259 nodes 3690222 tbhits 0 nps 325349 pv e2e4 e7e5 g1f3 b8c6 d2d4 e5d4 f3d4 g8f6 d4c6 d7c6 d1d8 e8d8 b1c3 c8e6 c1d2 a7a5 a2a4 f8b4 e1c1 d8e7 f2f3 h8d8 f1e2 b4c5 d2f4 d8d1 c3d1 c5d6 f4d6 c7d6 d1e3
info depth 22 seldepth 38 score cp 51 time 12222 nodes 4019569 tbhits 0 nps 327292 pv e2e4 e7e5 g1f3 b8c6 d2d4 e5d4 f3d4 g8f6 d4c6 d7c6 d1d8 e8d8 b1c3 c8e6 c1d2 a7a5 a2a4 f8b4 f2f3 d8e7 e1c1 h8d8 f1e2 b4c5 d2f4
info depth 23 seldepth 38 score cp 55 time 13950 nodes 4606225 tbhits 0 nps 328530 pv e2e4 e7e5 g1f3 b8c6 d2d4 e5d4 f3d4 g8f6 d4c6 d7c6 d1d8 e8d8 b1c3 c8e6 c1d2 a7a5 a2a4 f8b4 f2f3 d8e7 e1c1 a8d8 h2h4 f6d7 d2f4 b4c3 b2c3 d8c8 h4h5
info depth 24 seldepth 41 score cp 47 time 20015 nodes 6684555 tbhits 0 nps 329840 pv e2e4 e7e5 g1f3 b8c6 d2d4 e5d4 f3d4 g8f6 d4c6 d7c6 d1d8 e8d8 b1c3 c8e6 c1d2 a7a5 f2f3 f6d7 e1c1 d8c8 a2a4 b7b6 h2h4 f8d6 h4h5 h7h6 c3e2 d7c5 e2f4 c5a4 f4e6
info depth 25 seldepth 41 score cp 61 time 24764 nodes 8315645 tbhits 0 nps 333710 pv e2e4 e7e5 g1f3 b8c6 d2d4 e5d4 f3d4 g8f6 d4c6 d7c6 d1d8 e8d8 b1c3 c8e6 f2f3 a7a5 c1e3 f6d7 a2a4 d8c8 f3f4 f7f6 f1e2 f8b4 e1c1 b7b6 f4f5 e6f7 g2g4
info depth 26 seldepth 41 score cp 58 time 39528 nodes 13452253 tbhits 0 nps 335592 pv e2e4 e7e5 g1f3 b8c6 d2d4 e5d4 f3d4 g8f6 d4c6 d7c6 d1d8 e8d8 b1c3 c8e6 f2f3 a7a5 c1e3 f6d7 e1c1 d8c8 f3f4 f7f6 f1e2 f8b4 e3d2 b4c5 a2a4 b7b6 h2h4 h7h5 f4f5 e6f7
info depth 27 seldepth 46 score cp 70 time 49208 nodes 16803269 tbhits 0 nps 340175 pv e2e4 e7e5 g1f3 b8c6 d2d4 e5d4 f3d4 g8f6 d4c6 d7c6 d1d8 e8d8 b1c3 c8e6 f2f3 a7a5 c1e3 f6d7 e1c1 d8c8 f3f4 f7f6 f1e2 f8b4 e3d2 b4c5 a2a4 h7h5 c1b1 e6f7 e2f3 h5h4 d2e1 c5d6

[Damir]

Thanks Volodymyr. you did a really good job with your engine. Another +100 elo or more in next release and you will overtake Stockfish as number 1 Engine.

[voffka]

Thanks Volodymyr. you did a really good job with your engine. Another +100 elo or more in next release and you will overtake Stockfish as number 1 Engine.

Thanks Damir, I personally doubt it since Stockfish is a moving target

[voffka]

Hello Guenther,

And I just cannot believe my eyes, 3.0.0 seems three! times faster now for old hardware? (ok before it was really slow with nn)

I doubt Igel became significantly faster. It is true that I removed some extensions (no capture extensions, less history extensions, etc) and Igel should be able to reach bigger depth faster, but this would not be a huge difference compared to 2.9.0 or dev-0.

I think you need to have a bigger sample size of positions to see more results

[voffka]

Hello bastiball,

Hello mr. Igel

Congrats for the new release! and Goodluck!
Can you provide silvermont binaries? Thanks!

Regards.....

Thanks! During last releases it became significantly easier to compile Igel using gcc with the existing makefile. Please give it and try and if it does not work - I will help you to compile it.

This way you will not depend on me and will be able to compile future of versions of Igel