Komodo - Stockfish (1.000 games) ... FEOBOS v5.0 Test-Set

Frank Quisinsky · Post by **Frank Quisinsky** » Sat Jun 17, 2017 5:49 pm

Hi there,

the same with B00-B99 codes ...

Download (same link):
http://www.amateurschach.de/download/wa ... komodo.zip (1.19 Mb).

Code: Select all

Games        &#58;    200 &#40;finished&#41;

White Wins   &#58;     47 &#40;23.5 %)
Black Wins   &#58;     19 ( 9.5 %)
Draws        &#58;    134 &#40;67.0 %)
Unfinished   &#58;      0

White Perf.  &#58; 57.0 %
Black Perf.  &#58; 43.0 %

ECO A =      0 Games ( 0.0 %)
ECO B =    200 Games &#40;100.0 %)
ECO C =      0 Games ( 0.0 %)
ECO D =      0 Games ( 0.0 %)
ECO E =      0 Games ( 0.0 %)

Code: Select all

Individual statistics&#58;

1 ASMFishW 20170522 BMI2 x64&#58; 3239  200 (+ 55,=134,- 11&#41;, 61.0 %

Komodo 11.01 x64              &#58; 200 (+ 55,=134,- 11&#41;, 61.0 %

2 Komodo 11.01 x64          &#58; 3161  200 (+ 11,=134,- 55&#41;, 39.0 %

ASMFishW 20170522 BMI2 x64    &#58; 200 (+ 11,=134,- 55&#41;, 39.0 %

Code: Select all

Games   = 200   ( no result = 0,  FEN tags = 0 )
Players = 2   ( clusters = 1 )
Date Range&#58; 2017.06.16 - 2017.06.17

Games with&#58;  WhiteElo = 0   BlackElo = 0   BothElos = 0

White Wins = 47 ( 23.5 % )
Draws      = 134 ( 67.0 % )
Black Wins = 19 ( 9.5 % )
White Pct = 57.0 %
Black Pct = 43.0 %

ECO&#58;  Total = 200  A&#58; 0  B&#58; 200  C&#58; 0  D&#58; 0  E&#58; 0
PlyCount&#58;  Total = 200  Range&#58; 34-514  Average = 166.22  StdDev = 67.5

finished&#58; be sure to rename/copy outSummary

And A00-B99 ...

Code: Select all

Games        &#58;    400 &#40;finished&#41;

White Wins   &#58;     93 &#40;23.2 %)
Black Wins   &#58;     46 &#40;11.5 %)
Draws        &#58;    261 &#40;65.2 %)
Unfinished   &#58;      0

White Perf.  &#58; 55.9 %
Black Perf.  &#58; 44.1 %

ECO A =    200 Games &#40;50.0 %)
ECO B =    200 Games &#40;50.0 %)
ECO C =      0 Games ( 0.0 %)
ECO D =      0 Games ( 0.0 %)
ECO E =      0 Games ( 0.0 %)

Code: Select all

Individual statistics&#58;

1 ASMFishW 20170522 BMI2 x64&#58; 3239  400 (+114,=261,- 25&#41;, 61.1 %

Komodo 11.01 x64              &#58; 400 (+114,=261,- 25&#41;, 61.1 %

2 Komodo 11.01 x64          &#58; 3161  400 (+ 25,=261,-114&#41;, 38.9 %

ASMFishW 20170522 BMI2 x64    &#58; 400 (+ 25,=261,-114&#41;, 38.9 %

Code: Select all

Games   = 400   ( no result = 0,  FEN tags = 0 )
Players = 2   ( clusters = 1 )
Date Range&#58; 2017.06.14 - 2017.06.17

Games with&#58;  WhiteElo = 0   BlackElo = 0   BothElos = 0

White Wins = 93 ( 23.25 % )
Draws      = 261 ( 65.25 % )
Black Wins = 46 ( 11.5 % )
White Pct = 55.88 %
Black Pct = 44.13 %

ECO&#58;  Total = 400  A&#58; 200  B&#58; 200  C&#58; 0  D&#58; 0  E&#58; 0
PlyCount&#58;  Total = 400  Range&#58; 34-590  Average = 168.61  StdDev = 70.78

finished&#58; be sure to rename/copy outSummary

Best
Frank

Frank Quisinsky · Post by **Frank Quisinsky** » Sat Jun 17, 2017 6:41 pm

Range: 34-590 ... a draw game after 17 moves

So, all isn't perfect after 5 engine analysis in FEOBOS only. The position is on TOP-300 of over 26.000 in Ranking and produced a short draw game. I am sure that this position never, never is on TOP-5000 if all engines analyszed the database with the possibilities we have in Excel file.

But not important ... it's only an Alpha test for testing the ranking and test-set options from FEOBOS.

But very nice is the not high draw quote.
So the system works generally.

We are on the way!

Best
Frank

Frank Quisinsky · Post by **Frank Quisinsky** » Mon Jun 19, 2017 9:28 am

Hi there,

interesting is the draw quote:

A00 - A99 = 63,5%, Stockfish made 122,5 : 77,5 points
B00 - B99 = 67,0%, Stockfish made 122,0 : 78,0 points
C00 - C99 = 68,5%, Stockfish made 120,5 : 79,5 Points
D00 - D99 = still running
E00 - E99 = follow at last

Summary = 66,3%, Stockfish made 365,0 : 235,0 Points
66,3% is fully normely with two engines where the difference is around 70-80 Elo.

Download stats, games, current FEOBOS Test-Set v5.04:
http://www.amateurschach.de/download/wa ... komodo.zip

= Stockfish vs. Komodo 166+ / 398= / -36 = 60,83% = 76 Elo difference!

More Information can be found in the readme to my "Test of the Test-Set".

###

Here are the results:

Code: Select all

Games        &#58;    200 &#40;finished&#41;

White Wins   &#58;     36 &#40;18.0 %)
Black Wins   &#58;     27 &#40;13.5 %)
Draws        &#58;    137 &#40;68.5 %)
Unfinished   &#58;      0

White Perf.  &#58; 52.2 %
Black Perf.  &#58; 47.8 %

ECO A =      0 Games ( 0.0 %)
ECO B =      0 Games ( 0.0 %)
ECO C =    200 Games &#40;100.0 %)
ECO D =      0 Games ( 0.0 %)
ECO E =      0 Games ( 0.0 %)

Code: Select all

Individual statistics&#58;

1 ASMFishW 20170522 BMI2 x64&#58; 3236  200 (+ 52,=137,- 11&#41;, 60.2 %

Komodo 11.01 x64              &#58; 200 (+ 52,=137,- 11&#41;, 60.2 %

2 Komodo 11.01 x64          &#58; 3164  200 (+ 11,=137,- 52&#41;, 39.8 %

ASMFishW 20170522 BMI2 x64    &#58; 200 (+ 11,=137,- 52&#41;, 39.8 %

Code: Select all

Games   = 200   ( no result = 0,  FEN tags = 0 )
Players = 2   ( clusters = 1 )
Date Range&#58; 2017.06.17 - 2017.06.19

Games with&#58;  WhiteElo = 0   BlackElo = 0   BothElos = 0

White Wins = 36 ( 18.0 % )
Draws      = 137 ( 68.5 % )
Black Wins = 27 ( 13.5 % )
White Pct = 52.25 %
Black Pct = 47.75 %

ECO&#58;  Total = 200  A&#58; 0  B&#58; 0  C&#58; 200  D&#58; 0  E&#58; 0
PlyCount&#58;  Total = 200  Range&#58; 32-381  Average = 169.9  StdDev = 70.05

finished&#58; be sure to rename/copy outSummary

And the results A00-C99 ...

Code: Select all

Games        &#58;    600 &#40;finished&#41;

White Wins   &#58;    129 &#40;21.5 %)
Black Wins   &#58;     73 &#40;12.2 %)
Draws        &#58;    398 &#40;66.3 %)
Unfinished   &#58;      0

White Perf.  &#58; 54.7 %
Black Perf.  &#58; 45.3 %

ECO A =    200 Games &#40;33.3 %)
ECO B =    200 Games &#40;33.3 %)
ECO C =    200 Games &#40;33.3 %)
ECO D =      0 Games ( 0.0 %)
ECO E =      0 Games ( 0.0 %)

Code: Select all

Individual statistics&#58;

1 ASMFishW 20170522 BMI2 x64&#58; 3238  600 (+166,=398,- 36&#41;, 60.8 %

Komodo 11.01 x64              &#58; 600 (+166,=398,- 36&#41;, 60.8 %

2 Komodo 11.01 x64          &#58; 3162  600 (+ 36,=398,-166&#41;, 39.2 %

ASMFishW 20170522 BMI2 x64    &#58; 600 (+ 36,=398,-166&#41;, 39.2 %

Code: Select all

Games   = 600   ( no result = 0,  FEN tags = 0 )
Players = 2   ( clusters = 1 )
Date Range&#58; 2017.06.14 - 2017.06.19

Games with&#58;  WhiteElo = 0   BlackElo = 0   BothElos = 0

White Wins = 129 ( 21.5 % )
Draws      = 398 ( 66.33 % )
Black Wins = 73 ( 12.17 % )
White Pct = 54.67 %
Black Pct = 45.33 %

ECO&#58;  Total = 600  A&#58; 200  B&#58; 200  C&#58; 200  D&#58; 0  E&#58; 0
PlyCount&#58;  Total = 600  Range&#58; 32-590  Average = 169.04  StdDev = 70.54

finished&#58; be sure to rename/copy outSummary

Best
Frank

Frank Quisinsky · Post by **Frank Quisinsky** » Mon Jun 19, 2017 9:45 am

Here my corrections I do so far:
In most of cases, not easy to compare Chessbase and Shredder GUI. Often Chessbase displayed a wrong ECO Code and that ECO codes I have in FEOBOS database.

So I must search in FEOBOS 26.146 database for ECO codes more clear and made during the test-set the corrections. So, FEOBOS Test-Set isn't ready because I am sure that in D00-E99 different codes are wrong too.

But end of the Test the first and good "FEOBOS v5 Test-Set" is ready. To each ECO Code 1 variants (one of the with the highest ranking) ... engine can play with White and black pieces = 2x500 = 1.000 games.

End of FEOBOS development with more engine analysis a much better test-set can be created, but for the moment the test-set v5 is very strong!

Code: Select all

---
Update 1&#58; June 15th, 2017
Version FEOBOS v5.01 Test-Set

- it seems that different of best lines goes in move transposition later.
- Shredder GUI indicate often an other ECO code in comparing to ChessBase GUI.
  ECO code must be more clear!

A02 replaced with Pos 00234
A05 replaced with Pos 00389
A16 replaced with Pos 01220
A21 replaced with Pos 01486
A22 replaced with Pos 01569
A24 replaced with Pos 01616
A27 replaced with Pos 01805
A31 replaced with Pos 02130
A40 replaced with Pos 02864
A53 replaced with Pos 03920
A74 replaced with Pos 04471
C09 replaced with Pos 12481
C14 replaced with Pos 12693
D54 replaced with Pos 19255
E47 replaced with Pos 23690
E57 replaced with Pos 24197
E77 replaced with Pos 25259
E82 replaced with Pos 25336
E89 replaced with Pos 25491
E98 replaced with Pos 26032

- no queens on board
  more interesting to have queens on board, fixed!
  
B29 replaced with Pos 07728
B33 replaced with Pos 08235
B39 replaced with Pos 08461
C60 replaced with Pos 14418
C69 replaced with Pos 14907
D26 replaced with Pos 17872
D41 replaced with Pos 18782
D84 replaced with Pos 20234
E34 replaced with Pos 23177
E80 replaced with Pos 25290
E90 replaced with Pos 25549
E92 replaced with Pos 25644
E94 replaced with Pos 25854

- mistake in *.epd file to A00 fixed

###

---
Update 2&#58; June 16th, 2017
Version FEOBOS v5.02 Test-Set

A31 replaced with Pos 02113
B06 replaced with Pos 05977

Note&#58;
We have found a big bug in our Excel file. Quantity of different first moves
aren't corret. Best positions found after 5 analyszed engines changed. This
test-set isn't optimal but very strong.

For a better test-set we need more engine analysis and I am sure to the end
of the project we can make it better. But for the moment I optimate what I have.

###

---
Update 3&#58; June 17th, 2017
Version FEOBOS v5.03 Test-Set

B47 replaced with Pos 09332

###

---
Update 4&#58; June 18th, 2017
Version FEOBOS v5.04 Test-Set

C66 replaced with Pos 14801
C79 replaced with Pos 15246
C84 replaced with Pos 15415
C90 replaced with Pos 15813
D04 replaced with Pos 16716

###

Frank Quisinsky · Post by **Frank Quisinsky** » Tue Jun 20, 2017 5:17 pm

Hi there,

interesting is the draw quote:

A00 - A99 = 63,5%, Stockfish made 122,5 : 77,5 points
B00 - B99 = 67,0%, Stockfish made 122,5 : 77,5 points
C00 - C99 = 68,5%, Stockfish made 120,5 : 79,5 points
D00 - D99 = 79,0%, Stockfish made 115,0 : 85,0 points
E00 - E99 = still running

Summary
69,4% draw quote, Stockfish made 480,5 : 319,5 points
Stockfish vs. Komodo +203 / =555 / -42 = 60,06% / 71 Elo difference!

Download stats, games, current FEOBOS Test-Set v5.05 can be found on FEOBOS detail page under:
http://www.amateurschach.de/main/_new-opening-book.htm

For C67 and B94 I made updates for my Test-Set.
So games are replayed and stats corrected.

More information can be found in readme.txt to FEOBOS v5.05 Test-Set. Now the Test-Set is complete and corrected undo D99 ... now the last 200 games. I am sure I have to make again an update to make the last E00-E99 codes perfect. In 2 days the Test-Set will be ready for Take-Off.

Best
Frank

With two codes I have problems (C37 / C38). I have two balanced positions only in database. In all 4 cases to each of the codes one game ended with draw after 16 moves. Looking more in detail ... if Stockfish is working with contempt factor ... this problem is solved. So after all corrections to each of the first 400 Codes two games (black / white). And only two fast draw games, 1x C37, 1x C38.

Code: Select all

Games        &#58;    800 &#40;finished&#41;

White Wins   &#58;    153 &#40;19.1 %)
Black Wins   &#58;     92 &#40;11.5 %)
Draws        &#58;    555 &#40;69.4 %)
Unfinished   &#58;      0

White Perf.  &#58; 53.8 %
Black Perf.  &#58; 46.2 %

ECO A =    200 Games &#40;25.0 %)
ECO B =    200 Games &#40;25.0 %)
ECO C =    200 Games &#40;25.0 %)
ECO D =    200 Games &#40;25.0 %)
ECO E =      0 Games ( 0.0 %)

Code: Select all

Individual statistics&#58;

1 ASMFishW 20170522 BMI2 x64&#58; 3235  800 (+203,=555,- 42&#41;, 60.1 %

Komodo 11.01 x64              &#58; 800 (+203,=555,- 42&#41;, 60.1 %

2 Komodo 11.01 x64          &#58; 3165  800 (+ 42,=555,-203&#41;, 39.9 %

ASMFishW 20170522 BMI2 x64    &#58; 800 (+ 42,=555,-203&#41;, 39.9 %

Code: Select all

Games   = 800   ( no result = 0,  FEN tags = 0 )
Players = 2   ( clusters = 1 )
Date Range&#58; 2017.06.14 - 2017.06.20

Games with&#58;  WhiteElo = 0   BlackElo = 0   BothElos = 0

White Wins = 153 ( 19.13 % )
Draws      = 555 ( 69.38 % )
Black Wins = 92 ( 11.5 % )
White Pct = 53.81 %
Black Pct = 46.19 %

ECO&#58;  Total = 800  A&#58; 200  B&#58; 200  C&#58; 200  D&#58; 200  E&#58; 0
PlyCount&#58;  Total = 800  Range&#58; 32-590  Average = 166.35  StdDev = 69.08

finished&#58; be sure to rename/copy outSummary

Frank Quisinsky · Post by **Frank Quisinsky** » Thu Jun 22, 2017 1:11 pm

Hi there,

interesting is the draw quote:

A00 - A99 = 63,5%, Stockfish made 122,5 : 77,5 points
B00 - B99 = 67,0%, Stockfish made 122,5 : 77,5 points
C00 - C99 = 68,5%, Stockfish made 120,5 : 79,5 points
D00 - D99 = 79,0%, Stockfish made 115,0 : 85,0 points
E00 - E99 = 63,5%, Stockfish made 121.5 : 78,5 points

Summary
68,2% draw quote, Stockfish made 602,0 : 398,0 points
Stockfish vs. Komodo +261 / =682 / -57 = 60,20% / 72 Elo difference!

Download stats, games, current FEOBOS Test-Set v5.06 can be found on FEOBOS detail page under:
http://www.amateurschach.de/main/_new-opening-book.htm

Games are included in FEOBOS 5.06 Test-Set link.

Best
Frank

FEOBOS systems works great. Engine give us the information what they will have for openings (FEOBOS). Much more interesting as humans give that information engines with more 500 - 1500 lesser in Elo performance. And with high ranking positions in FEOBOS database a very nice test-set was born and is ready now.

Have fun with it!

Frank Quisinsky · Post by **Frank Quisinsky** » Thu Jun 22, 2017 1:14 pm

Hi there,

a lot of material for the programmer teams.
Many lost games for Komodo and Stockfish.

To copy all the lost game in database and replaying is very interesting. Often the same errors for lost games. But I have no time to do that in detail. Need a break after the last weeks FEOBOS and this work with the new test-set.

Best
Frank

Frank Quisinsky · Post by **Frank Quisinsky** » Thu Jun 22, 2017 1:23 pm

[pgn][Event "FEOBOS, 40/10, p=off, ht512, i7 4GHz"]
[Site "Trier"]
[Date "2017.06.22"]
[Round "214.1"]
[White "Komodo 11.01 x64"]
[Black "ASMFishW 20170522 BMI2 x64"]
[Result "0-1"]
[ECO "E90"]
[PlyCount "76"]
[EventDate "2017.??.??"]

1. c4 Nf6 2. Nf3 g6 3. Nc3 Bg7 4. e4 d6 5. d4 O-O 6. h3 e5 7. d5 Nh5 8. Nh2 Qe8
9. a3 {[%eval 19,22] [%emt 0:00:37]} f5 {[%eval 16,25] [%emt 0:00:24] (b6)} 10.
Be3 {[%eval 20,23] [%emt 0:00:19] (exf5)} a5 {[%eval 0,23] [%emt 0:00:07]} 11.
c5 {[%eval 9,23] [%emt 0:00:49] (Sf3)} Na6 {[%eval -12,26] [%emt 0:00:26] (f4)}
12. cxd6 {[%eval 28,23] [%emt 0:00:21] (Lb5)} f4 {[%eval 0,28] [%emt 0:00:22]
(cxd6)} 13. Bd2 {[%eval 18,28] [%emt 0:00:28]} cxd6 {[%eval 0,32] [%emt 0:00:
29]} 14. Rb1 {[%eval 8,27] [%emt 0:00:24] (b4)} Bd7 {[%eval 2,31] [%emt 0:00:
50]} 15. b4 {[%eval 13,27] [%emt 0:00:18]} Bf6 {[%eval -37,28] [%emt 0:00:32]}
16. Bd3 {[%eval 0,28] [%emt 0:00:35] (Le2)} Bd8 {[%eval -54,27] [%emt 0:00:20]}
17. O-O {[%eval 3,26] [%emt 0:00:08] (Db3)} axb4 {[%eval -51,28] [%emt 0:00:21]
} 18. axb4 {[%eval 8,25] [%emt 0:00:05]} Bb6 {[%eval -61,25] [%emt 0:00:01]}
19. Kh1 {[%eval -8,24] [%emt 0:00:15]} Nf6 {[%eval -63,27] [%emt 0:00:12]} 20.
Nf3 {[%eval -16,25] [%emt 0:00:31] (De2)} h6 {[%eval -78,26] [%emt 0:00:06]}
21. Nh4 {[%eval -22,23] [%emt 0:00:13] (De2)} Kg7 {[%eval -73,25] [%emt 0:00:
21] (Kh7)} 22. Qe2 {[%eval -27,21] [%emt 0:00:12]} Nc7 {[%eval -84,24] [%emt 0:
00:14]} 23. Rfd1 {[%eval -39,22] [%emt 0:00:23] (Kg1)} Qe7 {[%eval -116,24]
[%emt 0:00:08] (Df7)} 24. Rf1 {[%eval -53,24] [%emt 0:00:22] (Sf3)} Nh7 {
[%eval -156,26] [%emt 0:00:09] (g5)} 25. Nf3 {[%eval -82,21] [%emt 0:00:07]}
Ng5 {[%eval -174,27] [%emt 0:00:02]} 26. Kg1 {[%eval -81,24] [%emt 0:00:12]
(Kh2)} Nxf3+ {[%eval -278,26] [%emt 0:00:13]} 27. Qxf3 {[%eval -76,24] [%emt 0:
00:04]} h5 {[%eval -320,27] [%emt 0:00:07] (Dh4)} 28. Rbc1 {[%eval -207,21]
[%emt 0:00:20] (Le2)} g5 {[%eval -375,27] [%emt 0:00:09]} 29. Qxh5 {[%eval
-329,24] [%emt 0:00:23] (g4)} g4 {[%eval -633,27] [%emt 0:00:10] (Th8)} 30. h4
{[%eval -504,25] [%emt 0:00:34]} Rh8 {[%eval -677,27] [%emt 0:00:04]} 31. Qg5+
{[%eval -525,25] [%emt 0:00:05]} Qxg5 {[%eval -705,29] [%emt 0:00:12]} 32. hxg5
{[%eval -544,25] [%emt 0:00:04]} Rh5 {[%eval -32753,50] [%emt 0:00:09] (g3)}
33. Rfe1 {[%eval -32755,19] [%emt 0:00:01]} Rh1+ {[%eval -32755,58] [%emt 0:00:
03]} 34. Kxh1 {[%eval -32757,5] [%emt 0:00:00]} Bxf2 {[%eval -32757,78] [%emt
0:00:17]} 35. Re3 {[%eval -32759,79] [%emt 0:00:01]} g3 {[%eval -32759,80]
[%emt 0:00:03]} 36. Rxg3 {[%eval -32761,99] [%emt 0:00:00]} fxg3 {[%eval
-32761,97] [%emt 0:00:20]} 37. Be2 {[%eval -32763,99] [%emt 0:00:00]} Rh8+ {
[%eval -32763,127] [%emt 0:00:00]} 38. Bh5 {[%eval -32765,5] [%emt 0:00:00]}
Rxh5# {[%eval -32765,127] [%emt 0:00:00]} 0-1
[/pgn]

Komodo - Stockfish (1.000 games) ... FEOBOS v5.0 Test-Set

Re: After 400 games, B00-B99 ... stats!

Re: An enquiring eye can see that ...

Re: After 600 games, C00-C99 ... stats!

Re: Part of the readme / changes I have to do ...

Re: After 800 games, D00-D99 ... stats / games!

Re: FINAL: SF won with +261,=682,- 57, 68.2% draws only!

Re: A lot of material for the programmer teams!

Re: Here a nice game ... SF gave mate in 38 moves!