Modular opening book SF analysed 87417 pos., beta-1

[Frank Quisinsky]

Hi Ferdinand,

if I set Komodo to 0.30 I can delete more as 25% from the 26.619 positions I have.

Note:
The positions I have are "only" three moves after ECO Code was formed and min. 4-6 moves. Often the lines ... 6 moves length only have higher evals but are absolutely balanced.

The topic is really not easy.
If I reject now with 0.30 I have to different ECO codes no lines.

So 0.50 and -0.30 is Standard for Komodo and Stockfish. On this Standard we have to find out the settings for the others. Means if Komodo and Stockfish have 0.50 ... Houdini must have 0.60 or 0.65 but very often the eval is much higher. Or from Fizbo 0.7 or 0.75 or to many of really good positions are to reject.

Again ... this Topic is really not easy.

Example:
You can see it in the *.epd from Stockfish with analysed positions.

with 0.40 and -0.25 ... around 18.500 / 87417 I have to reject.
with 0.50 and -0.30 ... 9.441 ...

And with 0.30 ... to many!

I think 0.50 and -0.30 is absolutely OK as Standard!

[Ferdy]

I combined K10, k10.1 and k10.2 and select opp with close in strength.

Code: Select all

Results from file k012.pgn:

No. Name                       Win Draw Loss Unf.  Score Games       %
----------------------------------------------------------------------
  1 Komodo 10.0.1.2 x64        +37 =236  -77   *0  155.0   350   44.3%
  2 SF 18Sep2016 BMI2 x64      +36 =102  -12   *0   87.0   150   58.0%
  3 SF 18Sep2016 BMI2 x64 C10  +35  =98  -17   *0   84.0   150   56.0%
  4 SF 24Apr2016 BMI2 x64 C10   +6  =36   -8   *0   24.0    50   48.0%

Total Games:     350
White Wins:       69 (19.7%)
Black Wins:       45 (12.9%)
Draws:           236 (67.4%)

Results:

Code: Select all

Summary:
                 players,      min,      max,     Pcnt,     Wcnt,     Lcnt,     Dcnt,  perf(%),
     Komodo 10.0.1.2 x64,    -1.00,    -0.91,        4,        0,        4,        0,    0.00,
     Komodo 10.0.1.2 x64,    -0.90,    -0.81,        7,        0,        5,        2,   14.29,
     Komodo 10.0.1.2 x64,    -0.80,    -0.71,        6,        0,        5,        1,    8.33,
     Komodo 10.0.1.2 x64,    -0.70,    -0.61,        1,        0,        0,        1,   50.00,
     Komodo 10.0.1.2 x64,    -0.60,    -0.51,       17,        0,       12,        5,   14.71,
     Komodo 10.0.1.2 x64,    -0.50,    -0.41,       40,        0,       29,       11,   13.75,
     Komodo 10.0.1.2 x64,    -0.40,    -0.31,       75,        0,       38,       37,   24.67,
     Komodo 10.0.1.2 x64,    -0.30,    -0.21,      177,       16,       69,       92,   35.03,
     Komodo 10.0.1.2 x64,    -0.20,    -0.11,      283,       29,      104,      150,   36.75,
     Komodo 10.0.1.2 x64,    -0.10,    -0.01,      288,       24,       87,      177,   39.06,
     Komodo 10.0.1.2 x64,    -0.00,     0.09,      993,       56,      215,      722,   41.99,
     Komodo 10.0.1.2 x64,     0.10,     0.19,      501,       19,       68,      414,   45.11,
     Komodo 10.0.1.2 x64,     0.20,     0.29,      439,       26,       82,      331,   43.62,
     Komodo 10.0.1.2 x64,     0.30,     0.39,      348,       46,       39,      263,   51.01,
     Komodo 10.0.1.2 x64,     0.40,     0.49,      274,       51,       39,      184,   52.19,
     Komodo 10.0.1.2 x64,     0.50,     0.59,      189,       48,       22,      119,   56.88,
     Komodo 10.0.1.2 x64,     0.60,     0.69,       95,       36,        3,       56,   67.37,
     Komodo 10.0.1.2 x64,     0.70,     0.79,       42,       19,        8,       15,   63.10,
     Komodo 10.0.1.2 x64,     0.80,     0.89,       27,       17,        0,       10,   81.48,
     Komodo 10.0.1.2 x64,     0.90,     1.00,       11,        3,        0,        8,   63.64,

Total real games: 350
Eval window: -1.00 to 1.00
Move range: 13 to 24
Elpased time: 0.88m

Taking a perf of around 45% to 55% we got a score range of around 0.10 to 0.60.

Code: Select all

                 players,      min,      max,     Pcnt,     Wcnt,     Lcnt,     Dcnt,  perf(%),
     Komodo 10.0.1.2 x64,     0.10,     0.19,      501,       19,       68,      414,   45.11,
     Komodo 10.0.1.2 x64,     0.20,     0.29,      439,       26,       82,      331,   43.62,
     Komodo 10.0.1.2 x64,     0.30,     0.39,      348,       46,       39,      263,   51.01,
     Komodo 10.0.1.2 x64,     0.40,     0.49,      274,       51,       39,      184,   52.19,
     Komodo 10.0.1.2 x64,     0.50,     0.59,      189,       48,       22,      119,   56.88,

[Ferdy]

The topic is really not easy.
If I reject now with 0.30 I have to different ECO codes no lines.

The solution to this for engine vs engine testing is to play the position twice. Since you are not doing this, it is better to reject those lines as it would favor the other engine.

So 0.50 and -0.30 is Standard for Komodo and Stockfish. On this Standard we have to find out the settings for the others. Means if Komodo and Stockfish have 0.50 ... Houdini must have 0.60 or 0.65 but very often the eval is much higher. Or from Fizbo 0.7 or 0.75 or to many of really good positions are to reject.

With 0.50 even Komodo can score well already vs Stockfish. Score up to 0.49 with 52.19% perf.

Code: Select all

Komodo 10.0.1.2 x64,     0.40,     0.49,      274,       51,       39,      184,   52.19,

What more can other engines do against these top engines when facing with that starting position on an inferior side?
It is 91.14% in favor of the top engine.

Code: Select all

Komodo 10.2 x64 C15,     0.40,     0.49,     2331,     1934,       16,      381,   91.14,

[Frank Quisinsky]

Hi Ferdinand,

can understand it but ...

Thinking on:

1. With a white move the games started. White have the advantage from move 1 for the first moves. Around 0.20 I think.

0.50 - 0.20 = Advantage in reality is 0.30 ... with more and more moves ordinary a bit lesser.

2. The high perf Komodo have ...
Komodo will have the same high Perf if each game started with 0.0 vs. weaker opponents, believe me! Stockfish and Komodo ... and Houdini 5 too ... to strong for the others. Absolutely clear if Komodo started with 0.5 vs. weaker opponents that most of games are earlier over. Very hard but fact.

3. This topic was a main topic before I start the analysis. But highest prioritiy for myself is that all 500 ECO codes should be on the board and many of all the good known lines inside the ECO codes. If others used later the database we have much more games to "critical openings" for better opening stats.

4. 0.50 should be OK. Also weaker engines can start vs. stronger engines with such a database with 0.5 advantage, pur random. I think very important is not to reject good known mainlines ... or believe me ... I have a problem with many strong players ... if they analysed the database.

Frank, why you have deactivated this main line ... is one of the main lines to the ECO code X and computer chess people will say us ... main line is bad? And the bad call computer chess people often have (GM view) ... will be not better. And with 0.30 ... 25% of main lines can be found in nirvana.

Generally I am thinking today that to many openings the eval from engines can't be correct. Good example is E99 ... very hard opening but 40% of all good known main lines engines will give clearly more as 0.50. In written opening books often grandmaster have much other opinions to the E99 positions. And to many other ECO codes the same.

To try to generate such a database is more as a challange. So I must looking on the right site, on the left site.

I am sure that opinions about this topic can be quiet differently.

Best
Frank

Of course with the basics in my download files others can reject with 0.4 or 0.3 if they like such a database for engine testing. I think with 0.50 I am perfect underway.

I think engine programmers have to give engines a Booster for own calculating opening book moves ... for the first 15 moves ... if engine is out of book in move 8 ... booster from move 8-15 ... a lot of engines need more time ... if I am looking on all the fast lost games on FCP Rating List ... Loud thinking! OK, OK now my call is very bad for the group of engine programmers. I know I know ... but in reality I am very harmless.

[Frank Quisinsky]

Hi Ferdinand,

thinking the long night about your words.

Main work for this project is to reject the games and to set the priorities in the opening books later.

Main work because I must do that by hand.
No tool for it.

If I have an *.epd file with the evals after the analysis and the same game numbers in *.pgn it would be great to have such a tool I can more easy reject all the "bad" lines ...

I give the tool the Information:
Eval higher as x (for an example: 0.30 / -0.20, ce in EPD) ...
please reject the games from PGN and give me two new PGNs. A *.pgn with the games rejected and a *.pgn with "good" games.

If we have that its very esay to make experiements with the database. Again, *.epd and *.pgn have the same game numbers.

So I can reject with 0.50 or 0.40 or 0.30 ... what I like. I think 0.50 is perfect but if others are thinking better is 0.40 or 0.30 we can do it to.

Many hours I need for ...
The first beta ... to reject over 9.441 games in a 87.417 (Setting: 0.50 / -0.30) database. I do it in Chessbase game by game during the anlysis in the last 30 days are still running.

To sort the analysis is very easy with Norm's "epdorder" from his epd tool collection. But if we used "epdorder" the game numbers are Logical others as in *.pgn file.

0.30 is very hard ...
Maybe 0.40 is better ... with such a tool we can looking how many games to the ECO codes we have after.

Best
Frank

[Ferdy]

If I have an *.epd file with the evals after the analysis and the same game numbers in *.pgn it would be great to have such a tool I can more easy reject all the "bad" lines ...

I give the tool the Information:
Eval higher as x (for an example: 0.30 / -0.20, ce in EPD) ...
please reject the games from PGN and give me two new PGNs. A *.pgn with the games rejected and a *.pgn with "good" games

Most of the work for this job can be done by pgn-extract. What is needed is the epd file with eval and your big file. The output can be what you have wished.

[Frank Quisinsky]

Hi Ferdinand,

in both donwload files (big or small) on detail page you can find it.

Best
Frank

[Ferdy]

Hi Ferdinand,

in both donwload files (big or small) on detail page you can find it.

Best
Frank

I downloaded the small file.
In the reject dir, there is an epd file with ce op code like this,

Code: Select all

rnbqk2r/pp2ppbp/2pp1np1/8/P2PP3/2N5/1PP1BPPP/R1BQK1NR w KQkq - id "21865"; ce 53; acd 29; acs 30; acn 290052015; pv Sf3 ;

As I see it, this is the epd file outside of your score window, a score of 53 cp for example. So this is not a complete epd file with analysis.

[Frank Quisinsky]

Ferdinand,

right!

In the Stockfish analysis directory you can find the epd with all analysis.
Included / inside in the big download file.

Best
Frank

[Frank Quisinsky]

PS: I added the file now in the small download file too (now 15Mb).
Stockfish / analysis ...

file: 00001-87417_analysis.epd (16.519Mb)