HERT - brand new openings-set by Thomas Zipproth

[pohl4711]

My new gamebase for Stockfish-testruns is now complete. And all is new: +30% faster CPU (i7-6700HQ, Intel Skylake, 2.6GHz, longer thinkingtime (180“+1000ms (= 3'+1“) instead of 70“+700ms, 4x bigger Hash (512 MB per engine). And a complete new openings test-set is used. Thomas Zipproth created a new 500 positions openings set: The HERT chess engine test set (“H“uman and “E“ngine “R“elevant “T“heory.). Because of his stellar work on the Cerebellum-Library (Brainfish (www.zipproth.de)), he is definitly one of the best experts for enginechess openings on the planet.
It contains openings, played in high-level engine (online-)chess and by human GMs. So, it will not contain all ECO-codes (like FEOBOS-project for example), but only those openings, strong humans and engines in tournaments really play on the chessboard. So, this testset has probably the highest practical relevance an openings-set ever had and it has a very low draw-rate (higher, than SALC-openings, of course, but measureable lower, than other "normal" openings-sets).

Because of this, it is strongly recommended to use it for engine testing and for engine development.


You can download the HERT set on my website:

http://spcc.beepworld.de/downloads--links.htm


Here further information about the HERT testset, taken out of the ReadMe-File:

500 test positions selected from the most played variations in Engine and Human tournaments.

What properties should a good test set have?

1.) It should test all aspects of an engine, so it must contain openings leading to different positional and tactical problems.
2.) For a good analysis of humans games, some variations mostly played by humans like special gambits should also be contained.
3.) The positions should not be too drawish with todays best engines
4.) The games should not contain too much transitions, so that most games are unique.
5.) The positions should reflect what is actually played most in Engine and Human Tournaments.

The last point is the most important and unique point of Hert. Many test sets contain positions which appeared less than 5 times or even never in any Engine or Human tournament.

The idea of Hert is that a position which was played several thousand times in Engine or Humans tournaments, should have some desirable properties by default.
For example it is unlikely that such a  position is extremely drawish or nearly lost, otherwise it would not be played so often.
Of course some exceptions where engines tend to make a fast draw always have to be excluded.

Additionally the Hert set tries to reproduce the importance of all openings up to some degree. That means, when a special opening like Giuoco Piano appeared over 300.000 times in Engine games, it was split up into several variations with positions which appeared around 5000 - 20000 times.

So finally the Hert Set tests the ability of Engines to play all kinds of variations which are commonly appearing in Engine and Human Tournaments.

The openings were mixed (not sorted by ECO-code), so it is possible to use only a part of the Hert Set without distorted results. Using the full Hert Set (500 positions) means 1000 games in an engine head-to-head competiton...But keep in mind, that the lower the number of played games is, the higher the Errorbar of testing-results gets!

No line of the Hert Set includes an En-Passant move. So it is possible to use the Hert.PGN in the LittleBlitzerGUI (which has an En-Passant Bug (the captured pawn is not removed from the chessboard, when an En-Passant move is in the opening-line
PGN)).

All work on the Hert Set was done by Thomas Zipproth.
Tests and documentation: Thomas Zipproth & Stefan Pohl

[Gusev]

Be careful, Stefan! A sudden -28 Elo regression in SF dev builds is reported at https://nextchessmove.com/dev-builds since 2017-08-13.

[pohl4711]

Be careful, Stefan! A sudden -28 Elo regression in SF dev builds is reported at https://nextchessmove.com/dev-builds since 2017-08-13.

Thanx for the hint. But I am testing Stockfish 170810, which is a version before the regression.
And after 1000 games played (from 5000), there is no measureable regression in my testrun, compared to Stockfish 170526.

Regards - Stefan (SPCC)

[Leo]

Thomas, on a different subject, when Cerebellum comes out in its commercial form, could I plug it into Fritz GUI and build a book while using Fritz?

[jefk]

Using the full Hert Set (500 positions) means 1000 games in an engine head-to-head competiton...
All work on the Hert Set was done by Thomas Zipproth.
Tests and documentation: Thomas Zipproth & Stefan Pohl

ok i did a quick analysis on this pgn with K10.4, mainly out of curiosity,
and it appears that 1.c4 gives a slight advantage in this set (o.25 or so).
Whereby i think this isn't correct. A better defense (also with 1...e5)
for Black against 1.c4 certainly is possible !

[Dann Corbit]

Excellent.
Question everything. That is the only way forward.

[Dann Corbit]

Analysis of all the book nodes for the Hert.pgn file:
http://rybkaforum.net/cgi-bin/rybkaforu ... pid=574757

(No attachments here...)

[Dann Corbit]

Side to move should win:
[d]rnbqk2r/ppp2pbp/3p1np1/4p3/2PPP3/2N2N2/PP3PPP/R1BQKB1R w KQkq - acd 37; acs 297; bm dxe5; cce 131; ce 118; pm Be2 {57} dxe5 {22} d5 {4}; pv dxe5 dxe5 Qxd8+ Kxd8 Nxe5 Be6 Nf3 Nc6 Ng5 Ke8 Bf4 Nd4 O-O-O Ng4 Bg3 h5 Rd2 f6 Nb5 fxg5 Nxd4 Bxd4 Rxd4 Rf8 f3 Ne3 b3 Rd8 Rxd8+ Kxd8 h3 Nxf1 Rxf1 Kd7 Kd2 Kc6 Be5 Rf7 Kc3 g4 hxg4 Bxg4 Kd2 Be6 Rh1 Rd7+ Ke3 Kb6 f4 c5; white_wins 41; black_wins 18; draws 19;

Side to move is in some trouble:
[d]r1bq1rk1/ppp2pbp/2np1np1/3Pp3/2P1P3/2N2N2/PP2BPPP/R1BQ1RK1 b - - acd 37; acs 2738; bm Nd4; c3 "Ne7"; cce -70; ce -88; pm Ne7 {32465} Nb8 {73} Nb4 {66} Nd4 {12} Nd7 {2} Kh8 {1} Na5 {1}; pv Nd4 Nxd4 exd4 Qxd4 Nd7 Qd1 Nc5 Re1 Re8 Bf3 Bd7 Qc2 Be5 Be3 b6 g3 Qf6 Bg2 Qg7 f3 f5 Rab1 fxe4 Nxe4 Nxe4 fxe4 Rf8 Qd2 Qh8 Kh1 a5 Bh6 Bg7 Bxg7 Qxg7 Rf1 Bg4 Kg1 Rae8; white_wins 13890; black_wins 8980; draws 8470; Opening ECO:E97d; King's Indian: Mar del Plata, 8.d5; 1.d4 Nf6 2.c4 g6 3.Nc3 Bg7 4.e4 d6 5.Nf3 O-O 6.Be2 e5 7.O-O Nc6 8.d5 *;