Chessprogams with the most chessknowing

[Henryval]

Which Chessprogram(s) have the most implemented chessknowing. And are there more than a few proofs out there in the chesscommunity?

I wonder why we don't have any ranking about this issue.

[Laskos]

Which Chessprogram(s) have the most implemented chessknowing. And are there more than a few proofs out there in the chesscommunity?

I wonder why we don't have any ranking about this issue.

I am not sure how to define this. Fixed depth play at depth=1 shows Komodo as the most knowledgeable engine. Generally, modern engines seem to have better knowledge than older ones.

[Ferdy]

Which Chessprogram(s) have the most implemented chessknowing. And are there more than a few proofs out there in the chesscommunity?

I wonder why we don't have any ranking about this issue.

I have been trying to implement some knowledge in my engine, and one of these is the ff. Black to move and drive the score to near zero. No egtb of course. H4 and Hiarcs 14 are good on this.
[d]6k1/8/6PP/3B1K2/8/2b5/8/8 b - - 0 1

Code: Select all

Deuterium v14.4.35.13 64bit POPCNT:
[...]
 34/40	00:03	 11,803k	3,356k	-2.33	Kf8 Ke6 Bd4 Bc4 Bh8 Kd5 Bb2 Ke4 Bh8 Ba2 Bb2 Bb3 Ba1 Bf7 Bc3 Kd3 Bf6 Bd5 Bh8 Bc4 Ba1 Ba2 Bf6 Kc4 Ba1 Kc5 Bh8 Bf7 Bc3 Kc4 Bb2 Bd5 Bh8 g7+ Bxg7 h7 Bh8
 35/43	00:04	 16,071k	3,465k	-2.32	Kf8 Ke6 Bd4 Bc4 Bh8 Kd5 Bb2 Ke4 Bh8 Ba2 Bb2 Bb3 Ba1 Bf7 Bc3 Kd3 Bf6 Bd5 Bh8 Bc4 Ba1 Kc2 Bd4 Kb3 Bh8 Kb4 Bb2 Bd3 Bh8 Kb5 Kg8 Kc6 Kf8 Kd5 Bb2 g7+ Bxg7 h7 Bh8
 36/43	00:06	 24,563k	3,527k	-2.31	Kf8 Ke6 Bd4 Bc4 Bh8 Kd5 Bb2 Ke4 Bh8 Ba2 Bb2 Bb3 Ba1 Bf7 Bc3 Kd3 Bf6 Bd5 Bh8 Bc4 Ba1 Be6 Bh8 Kc4 Ba1 Bd5 Bh8 Kb5 Bc3 Be6 Bh8 Bc4 Bb2 Kc6 Bd4 h7 Bh8 Be2 Bc3 h8Q+ Bxh8

[Ferdy]

Which Chessprogram(s) have the most implemented chessknowing. And are there more than a few proofs out there in the chesscommunity?

I wonder why we don't have any ranking about this issue.

Another one to implement is this, white should know that this is a draw.
[d]8/8/r5kP/6P1/1R3K2/8/8/8 w - - 0 1

[Ferdy]

Which Chessprogram(s) have the most implemented chessknowing. And are there more than a few proofs out there in the chesscommunity?

I wonder why we don't have any ranking about this issue.

Firing up the script to get some uci engines to work. Single thread, 64mb hash, time is 1 sec. Hannibal is the best here. Score is in cp.
[d]6k1/8/6PP/3B1K2/8/2b5/8/8 b - - 0 1

Code: Select all

Engine id name Andscacs 0.71
score -387

Engine id name Arasan 17.4
score -394

Engine id name cheng4 0.36c
score -204

Engine id name Deuterium v14.3.34.130
score -561

Engine id name Deuterium v14.4.35.13 64bit POPCNT
score -229

Engine id name Equinox 3.30 x64mp
score -183

Engine id name Fire 4 x64
score -150

Engine id name Fruit reloaded 2.1
score -255

Engine id name Gaviota v1.0
score -438

Engine id name GNU Chess 5.60-64
score -344

Engine id name Gull 3 x64
score -230

Engine id name Hannibal 1.4x64
score +0

Engine id name HIARCS 14 WCSC
score -188

Engine id name Houdini 4 x64
score -16

Engine id name iCE 2.0 v2240 x64/popcnt
score -370

Engine id name Maverick 0.51 x64
score -645

Engine id name Nebula 2.0
score -458

Engine id name Nemo SP64o 1.0.1 Beta 
score -437

Engine id name Quazar 0.4 x64
score -539

Engine id name Senpai 1.0
score -217

Engine id name Stockfish 131214 64 POPCNT
score -224

Engine id name Texel 1.04 64-bit
score -307

Engine id name Vajolet2 1.45
score -282

 1 id name Hannibal 1.4x64 (score 0)
 2 id name Houdini 4 x64 (score -16)
 3 id name Fire 4 x64 (score -150)
 4 id name Equinox 3.30 x64mp (score -183)
 5 id name HIARCS 14 WCSC (score -188)
 6 id name cheng4 0.36c (score -204)
 7 id name Senpai 1.0 (score -217)
 8 id name Stockfish 131214 64 POPCNT (score -224)
 9 id name Deuterium v14.4.35.13 64bit POPCNT (score -229)
10 id name Gull 3 x64 (score -230)
11 id name Fruit reloaded 2.1 (score -255)
12 id name Vajolet2 1.45 (score -282)
13 id name Texel 1.04 64-bit (score -307)
14 id name GNU Chess 5.60-64 (score -344)
15 id name iCE 2.0 v2240 x64/popcnt (score -370)
16 id name Andscacs 0.71 (score -387)
17 id name Arasan 17.4 (score -394)
18 id name Nemo SP64o 1.0.1 Beta  (score -437)
19 id name Gaviota v1.0 (score -438)
20 id name Nebula 2.0 (score -458)
21 id name Quazar 0.4 x64 (score -539)
22 id name Deuterium v14.3.34.130 (score -561)
23 id name Maverick 0.51 x64 (score -645)

[Ferdy]

Which Chessprogram(s) have the most implemented chessknowing. And are there more than a few proofs out there in the chesscommunity?

I wonder why we don't have any ranking about this issue.

White to play and score should be close to zero. Single thread, 64mb hash, 1 sec thinking time.
[d]8/8/r5kP/6P1/1R3K2/8/8/8 w - - 0 1

Code: Select all

 1 id name Fire 4 x64 (score 92)
 2 id name Gull 3 x64 (score 93)
 3 id name Stockfish 131214 64 POPCNT (score 187)
 4 id name Houdini 4 x64 (score 199)
 5 id name Komodo 6 64-bit  (score 225)
 6 id name Hannibal 1.4x64 (score 247)
 7 id name HIARCS 14 WCSC (score 253)
 8 id name Equinox 3.30 x64mp (score 261)
 9 id name Nebula 2.0 (score 286)
10 id name Nemo SP64o 1.0.1 Beta  (score 301)
11 id name Andscacs 0.71 (score 316)
12 id name Senpai 1.0 (score 320)
13 id name GNU Chess 5.60-64 (score 334)
14 id name cheng4 0.36c (score 338)
15 id name Arasan 17.4 (score 340)
16 id name Fruit reloaded 2.1 (score 344)
17 id name Vajolet2 1.45 (score 355)
18 id name Gaviota v1.0 (score 378)
19 id name Texel 1.04 64-bit (score 378)
20 id name Quazar 0.4 x64 (score 381)
21 id name Deuterium v14.3.34.130 (score 384)
22 id name Maverick 0.51 x64 (score 505)
23 id name iCE 2.0 v2240 x64/popcnt (score 517)

[Laskos]

Which Chessprogram(s) have the most implemented chessknowing. And are there more than a few proofs out there in the chesscommunity?

I wonder why we don't have any ranking about this issue.

I am not sure how to define this. Fixed depth play at depth=1 shows Komodo as the most knowledgeable engine. Generally, modern engines seem to have better knowledge than older ones.

I did some fast testing with fixed depth=1 for several top modern engines and engines reputed to have much knowledge. I included the ancient Shredder 6PB (year 2002 IIRC), which also claimed to have lots of knowledge at that time.
RR in cutechess-cli:

Code: Select all

Rank Name                        ELO   Games   Score   Draws
   1 Komodo 8                     92    1000     63%     18%
   2 Houdini 4                    78    1000     61%     26%
   3 Hannibal 1.4                 56    1000     58%     23%
   4 SF 14122014                  43    1000     56%     20%
   5 Hiarcs 14                   -22    1000     47%     16%
   6 Shredder 6PB (2002)        -302    1000     15%     14%
Finished match

Modern engines seem to prevail, especially those whose eval passed at some time through Larry's hands.

[hgm]

I would expect slow searchers like The Baron to have far more knowledge.

[Adam Hair]

Which Chessprogram(s) have the most implemented chessknowing. And are there more than a few proofs out there in the chesscommunity?

I wonder why we don't have any ranking about this issue.

I am not sure how to define this. Fixed depth play at depth=1 shows Komodo as the most knowledgeable engine. Generally, modern engines seem to have better knowledge than older ones.

I did some fast testing with fixed depth=1 for several top modern engines and engines reputed to have much knowledge. I included the ancient Shredder 6PB (year 2002 IIRC), which also claimed to have lots of knowledge at that time.
RR in cutechess-cli:

Code: Select all

Rank Name                        ELO   Games   Score   Draws
   1 Komodo 8                     92    1000     63%     18%
   2 Houdini 4                    78    1000     61%     26%
   3 Hannibal 1.4                 56    1000     58%     23%
   4 SF 14122014                  43    1000     56%     20%
   5 Hiarcs 14                   -22    1000     47%     16%
   6 Shredder 6PB (2002)        -302    1000     15%     14%
Finished match

Modern engines seem to prevail, especially those whose eval passed at some time through Larry's hands.

I think that at least part of the difference is due to Shredder not searching as many nodes as Komodo and Houdini.

I tried the same experiment this afternoon, primarily because I have been thinking about how to measure which engines handle various material imbalances better. Here are my results:

Code: Select all

   # PLAYER              : RATING    POINTS  PLAYED    (%)
   1 Houdini 4           :   78.0    1865.5    3029   61.6%
   2 Equinox 3.3         :   76.7    1862.0    3040   61.2%
   3 Komodo 8            :   52.1    1727.0    3000   57.6%
   4 Critter 1.6a        :   37.9    1671.0    3000   55.7%
   5 Gull 3              :   18.9    1589.5    3040   52.3%
   6 Stockfish 141213    :   14.8    1564.0    3000   52.1%
   7 Gaviota 1.0         :  -89.3    1093.0    3000   36.4%
   8 Fire 4              : -189.1     697.0    3029   23.0%

Then I examined what Houdini and Fire do when made to search 1 ply. Houdini searches many more nodes on average. Here is an example:

Houdini

Code: Select all

1419023568.383 GUI->Adapter: position startpos moves e2e4 e7e5 g1f3 g8f6 d2d4 f6e4 f1d3 d7d5 f3e5 b8d7 e5d7 c8d7 e1g1 f8d6 c2c4 c7c6 c4d5 c6d5 b1c3 e4c3 b2c3 e8g8 d1h5 f7f5 h5f3 g8h8 c1d2 d8h4 g2g3 h4g4 g1g2 a8c8 a1b1 b7b6 f1e1 f5f4 f3g4 d7g4 a2a4 g7g6 g2g1 g4d7 d3a6 c8b8 a6b5 b8d8 b5d7 d8d7 b1b5 d6c7 d2f4 c7f4 g3f4 f8f4 e1e5
1419023568.383 Adapter->Engine: position startpos moves e2e4 e7e5 g1f3 g8f6 d2d4 f6e4 f1d3 d7d5 f3e5 b8d7 e5d7 c8d7 e1g1 f8d6 c2c4 c7c6 c4d5 c6d5 b1c3 e4c3 b2c3 e8g8 d1h5 f7f5 h5f3 g8h8 c1d2 d8h4 g2g3 h4g4 g1g2 a8c8 a1b1 b7b6 f1e1 f5f4 f3g4 d7g4 a2a4 g7g6 g2g1 g4d7 d3a6 c8b8 a6b5 b8d8 b5d7 d8d7 b1b5 d6c7 d2f4 c7f4 g3f4 f8f4 e1e5
1419023568.383 GUI->Adapter: go depth 1
1419023568.383 Adapter->Engine: go depth 1
1419023568.393 Engine->Adapter: info multipv 1 depth 1 seldepth 8 score cp 8 time 0 nodes 565 nps 0 tbhits 0 hashfull 0 pv d7c7
1419023568.393 Adapter->GUI: info multipv 1 depth 1 seldepth 8 score cp 8 time 0 nodes 565 nps 0 tbhits 0 hashfull 0 pv d7c7
1419023568.393 Engine->Adapter: bestmove d7c7 ponder 0000
1419023568.393 Adapter->GUI: bestmove d7c7 ponder 0000

Fire

Code: Select all

1419023696.997 GUI->Adapter: position startpos moves e2e4 e7e5 g1f3 g8f6 d2d4 f6e4 f1d3 d7d5 f3e5 b8d7 e5d7 c8d7 e1g1 f8d6 c2c4 c7c6 c4d5 c6d5 b1c3 e4c3 b2c3 e8g8 d1h5 f7f5 h5f3 g8h8 c1d2 d8h4 g2g3 h4g4 g1g2 a8c8 a1b1 b7b6 f1e1 f5f4 f3g4 d7g4 a2a4 g7g6 g2g1 g4d7 d3a6 c8b8 a6b5 b8d8 b5d7 d8d7 b1b5 d6c7 d2f4 c7f4 g3f4 f8f4 e1e5
1419023696.997 Adapter->Engine: position startpos moves e2e4 e7e5 g1f3 g8f6 d2d4 f6e4 f1d3 d7d5 f3e5 b8d7 e5d7 c8d7 e1g1 f8d6 c2c4 c7c6 c4d5 c6d5 b1c3 e4c3 b2c3 e8g8 d1h5 f7f5 h5f3 g8h8 c1d2 d8h4 g2g3 h4g4 g1g2 a8c8 a1b1 b7b6 f1e1 f5f4 f3g4 d7g4 a2a4 g7g6 g2g1 g4d7 d3a6 c8b8 a6b5 b8d8 b5d7 d8d7 b1b5 d6c7 d2f4 c7f4 g3f4 f8f4 e1e5
1419023696.997 GUI->Adapter: go depth 1
1419023696.997 Adapter->Engine: go depth 1
1419023696.997 Engine->Adapter: info time 1 nodes 0 nps 0 tbhits 0 depth 1 score cp 0 pv h8g8
1419023696.997 Adapter->GUI: info time 1 nodes 0 nps 0 tbhits 0 depth 1 score cp 0 pv h8g8
1419023696.997 Engine->Adapter: info time 1 nodes 0 nps 0 tbhits 0 depth 1 score cp 4 pv h8g7
1419023696.997 Adapter->GUI: info time 1 nodes 0 nps 0 tbhits 0 depth 1 score cp 4 pv h8g7
1419023696.998 Engine->Adapter: info time 1 nodes 41 nps 41000 tbhits 0 depth 1 score cp 8 pv f4f3
1419023696.998 Adapter->GUI: info time 1 nodes 41 nps 41000 tbhits 0 depth 1 score cp 8 pv f4f3
1419023696.998 Engine->Adapter: bestmove f4f3
1419023696.998 Adapter->GUI: bestmove f4f3

I suspect that part of the reason that Shredder is scoring much worse in your test is because it is not searching as many nodes.

[bob]

Which Chessprogram(s) have the most implemented chessknowing. And are there more than a few proofs out there in the chesscommunity?

I wonder why we don't have any ranking about this issue.

I am not sure how to define this. Fixed depth play at depth=1 shows Komodo as the most knowledgeable engine. Generally, modern engines seem to have better knowledge than older ones.

I did some fast testing with fixed depth=1 for several top modern engines and engines reputed to have much knowledge. I included the ancient Shredder 6PB (year 2002 IIRC), which also claimed to have lots of knowledge at that time.
RR in cutechess-cli:

Code: Select all

Rank Name                        ELO   Games   Score   Draws
   1 Komodo 8                     92    1000     63%     18%
   2 Houdini 4                    78    1000     61%     26%
   3 Hannibal 1.4                 56    1000     58%     23%
   4 SF 14122014                  43    1000     56%     20%
   5 Hiarcs 14                   -22    1000     47%     16%
   6 Shredder 6PB (2002)        -302    1000     15%     14%
Finished match

Modern engines seem to prevail, especially those whose eval passed at some time through Larry's hands.

This doesn't work very well. depth=1 does NOT mean the same thing to everyone. To some it is simply a 1 ply search followed by q-search. Some allow check extensions before going to q-search. Some allow check extensions AFTER q-search. Some have a couple of plies of threats between basic search and start of q-search. Some extend at the root if in check, some don't. Some extend if giving check at the root, some don't. Etc. I played a match Crafty vs Stockfish a while back to compare evaluations. It took quite a bit of work to make them search the same basic tree space.

I ended up having to modify code in both programs to reach a consistent meaning for "set depth = 1". After all the work was done, the net conclusion from playing 100K games was "no statistical difference".