18 engines I find out ... for test-set experiments!

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Dann Corbit, Harvey Williamson

Frank Quisinsky
Posts: 6808
Joined: Wed Nov 18, 2009 7:16 pm
Location: Gutweiler, Germany
Full name: Frank Quisinsky

18 engines I find out ... for test-set experiments!

Post by Frank Quisinsky »

Hi there,

I am searching attackers generate a lesser move average (without contempt) and lost not to many games.
Defenders like to play a safe style with a lesser move average are OK (Seer).

I try out now all of the TOP-engines more times.

At last again Berserk 9.0 NN.
But without contempt the move average with 94 moves is clearly to high = 9 over average!

Have a look in Andscacs 0.1 NNSf stats.
Also not OK, move average from white / draw games.
Must cancel the engine in the field or my stats later will be bad.

The basic tourney is important for a comparasion with a test-set of first 50 positions.
Stats are generated with my best opening lines (opening book, most comes from FEOBOS).

So three engines I will delete in the group:

1. Wasp 5.50 NN (2x Wasp in the field).
2. Berserk 9.0 NN (move average is to high).
3. Andscacs 0.1 NNSF

So I think I have to work with 18 programs = 1.700 basis games per engines.

Again, all others in TOP-50 produced different problems:

1. Move average is to high in most of cases "no contempt parameter" (Rebel, Stockfish, Koivisto, Fritz 18, Minic, Clover, Berserk)
2. To many fast lost games, made no sense to use this engines for find positions for a test-set (Halogen, Nemorino, Fritz 18, Minic, Shredder, Zahak, Lc0 CPU, Combusken, Clover)
3. Engines are not strong enough in the field (Examples: Hiarcs, Weiss, Fizbo, Defenchess, DanaSah, Laser, Chiron and a lot of others). With around 50 Elo more Weiss or Hiarcs can be interesting. Also Danasah, Laser and Defenchess can be very interesting.

Best
Frank

OK, field of 18 engines is clear to start.
Opening book vs. Test-Set!
Can be start soon ... each opening position will get later an own ranking!

Code: Select all

***************************************************************
***              Short Games Analyzer Tool V3.1             ***
***           (C) 2022, Stefan Pohl, www.sp-cc.de           ***
***************************************************************
*** Evaluated file: games.pgn 
***************************************************************

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
*** Rank 1: "Velvet 4.0.1 NN"  All games: 2000 (192 wins / 1041 draws / 767 losses)
*** Overall score: 35.63%     Overall draw-rate: 52.05% 
*** Average game length: All: 85 moves (wins: 74 moves / draws: 85 moves / losses: 88 moves) 
*** Avg. eng play white: All: 84 moves (wins: 72 moves / draws: 84 moves / losses: 91 moves) 
*** Avg. eng play black: All: 86 moves (wins: 78 moves / draws: 85 moves / losses: 87 moves) 
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
*** Rank 2: "Stockfish 11"  All games: 2000 (540 wins / 1305 draws / 155 losses)
*** Overall score: 59.63%     Overall draw-rate: 65.25% 
*** Average game length: All: 89 moves (wins: 80 moves / draws: 93 moves / losses: 93 moves) 
*** Avg. eng play white: All: 87 moves (wins: 79 moves / draws: 90 moves / losses: 99 moves) 
*** Avg. eng play black: All: 92 moves (wins: 81 moves / draws: 95 moves / losses: 91 moves) 
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
*** Rank 3: "Booot 6.5"  All games: 1998 (147 wins / 1024 draws / 827 losses)
*** Overall score: 32.98%     Overall draw-rate: 51.25% 
*** Average game length: All: 87 moves (wins: 82 moves / draws: 86 moves / losses: 88 moves) 
*** Avg. eng play white: All: 86 moves (wins: 80 moves / draws: 84 moves / losses: 90 moves) 
*** Avg. eng play black: All: 88 moves (wins: 86 moves / draws: 89 moves / losses: 87 moves) 
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
*** Rank 4: "Xiphos 0.6"  All games: 2000 (146 wins / 1005 draws / 849 losses)
*** Overall score: 32.43%     Overall draw-rate: 50.25% 
*** Average game length: All: 85 moves (wins: 85 moves / draws: 86 moves / losses: 83 moves) 
*** Avg. eng play white: All: 85 moves (wins: 83 moves / draws: 85 moves / losses: 86 moves) 
*** Avg. eng play black: All: 84 moves (wins: 89 moves / draws: 87 moves / losses: 81 moves) 
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
*** Rank 5: "Dragon 3 NN (Komodo)"  All games: 2000 (1050 wins / 949 draws / 1 losses)
*** Overall score: 76.23%     Overall draw-rate: 47.45% 
*** Average game length: All: 82 moves (wins: 80 moves / draws: 83 moves / losses: 103 moves) 
*** Avg. eng play white: All: 82 moves (wins: 80 moves / draws: 88 moves / losses: 0 moves) 
*** Avg. eng play black: All: 82 moves (wins: 82 moves / draws: 81 moves / losses: 103 moves) 
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
*** Rank 6: "Andscacs 0.1 NNSf"  All games: 2000 (514 wins / 1328 draws / 158 losses)
*** Overall score: 58.90%     Overall draw-rate: 66.40% 
*** Average game length: All: 88 moves (wins: 82 moves / draws: 90 moves / losses: 86 moves) 
*** Avg. eng play white: All: 101 moves (wins: 81 moves / draws: 115 moves / losses: 87 moves) 
*** Avg. eng play black: All: 74 moves (wins: 85 moves / draws: 70 moves / losses: 85 moves) 
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
*** Rank 7: "Tucano 10.00 NN"  All games: 2000 (202 wins / 1227 draws / 571 losses)
*** Overall score: 40.78%     Overall draw-rate: 61.35% 
*** Average game length: All: 82 moves (wins: 91 moves / draws: 80 moves / losses: 83 moves) 
*** Avg. eng play white: All: 81 moves (wins: 90 moves / draws: 77 moves / losses: 87 moves) 
*** Avg. eng play black: All: 83 moves (wins: 94 moves / draws: 82 moves / losses: 81 moves) 
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
*** Rank 8: "Wasp 5.50 NN"  All games: 2000 (169 wins / 1093 draws / 738 losses)
*** Overall score: 35.78%     Overall draw-rate: 54.65% 
*** Average game length: All: 81 moves (wins: 85 moves / draws: 79 moves / losses: 84 moves) 
*** Avg. eng play white: All: 82 moves (wins: 84 moves / draws: 79 moves / losses: 87 moves) 
*** Avg. eng play black: All: 81 moves (wins: 87 moves / draws: 79 moves / losses: 82 moves) 
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
*** Rank 9: "SlowChess Blitz 2.9 NN"  All games: 2000 (605 wins / 1301 draws / 94 losses)
*** Overall score: 62.78%     Overall draw-rate: 65.05% 
*** Average game length: All: 90 moves (wins: 84 moves / draws: 93 moves / losses: 91 moves) 
*** Avg. eng play white: All: 86 moves (wins: 83 moves / draws: 87 moves / losses: 92 moves) 
*** Avg. eng play black: All: 94 moves (wins: 86 moves / draws: 97 moves / losses: 91 moves) 
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
*** Rank 10: "Wasp 5.58 NN dev"  All games: 2000 (247 wins / 1239 draws / 514 losses)
*** Overall score: 43.33%     Overall draw-rate: 61.95% 
*** Average game length: All: 81 moves (wins: 86 moves / draws: 78 moves / losses: 86 moves) 
*** Avg. eng play white: All: 81 moves (wins: 85 moves / draws: 77 moves / losses: 90 moves) 
*** Avg. eng play black: All: 82 moves (wins: 88 moves / draws: 78 moves / losses: 85 moves) 
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
*** Rank 11: "Revenge 3.0 NN exp"  All games: 2000 (727 wins / 1209 draws / 64 losses)
*** Overall score: 66.58%     Overall draw-rate: 60.45% 
*** Average game length: All: 85 moves (wins: 82 moves / draws: 87 moves / losses: 90 moves) 
*** Avg. eng play white: All: 85 moves (wins: 82 moves / draws: 89 moves / losses: 87 moves) 
*** Avg. eng play black: All: 85 moves (wins: 83 moves / draws: 85 moves / losses: 90 moves) 
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
*** Rank 12: "rofChade 3.0 NN"  All games: 1998 (549 wins / 1301 draws / 148 losses)
*** Overall score: 60.03%     Overall draw-rate: 65.12% 
*** Average game length: All: 90 moves (wins: 87 moves / draws: 91 moves / losses: 84 moves) 
*** Avg. eng play white: All: 89 moves (wins: 86 moves / draws: 91 moves / losses: 86 moves) 
*** Avg. eng play black: All: 90 moves (wins: 90 moves / draws: 91 moves / losses: 83 moves) 
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
*** Rank 13: "Rodent 1.0 NNSf"  All games: 2000 (212 wins / 1256 draws / 532 losses)
*** Overall score: 42.00%     Overall draw-rate: 62.80% 
*** Average game length: All: 85 moves (wins: 88 moves / draws: 85 moves / losses: 83 moves) 
*** Avg. eng play white: All: 84 moves (wins: 89 moves / draws: 83 moves / losses: 87 moves) 
*** Avg. eng play black: All: 85 moves (wins: 87 moves / draws: 87 moves / losses: 81 moves) 
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
*** Rank 14: "Fire 8.2"  All games: 2000 (290 wins / 1182 draws / 528 losses)
*** Overall score: 44.05%     Overall draw-rate: 59.10% 
*** Average game length: All: 87 moves (wins: 83 moves / draws: 88 moves / losses: 88 moves) 
*** Avg. eng play white: All: 87 moves (wins: 81 moves / draws: 89 moves / losses: 89 moves) 
*** Avg. eng play black: All: 87 moves (wins: 88 moves / draws: 87 moves / losses: 87 moves) 
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
*** Rank 15: "RubiChess 20220223 NN"  All games: 2000 (539 wins / 1276 draws / 185 losses)
*** Overall score: 58.85%     Overall draw-rate: 63.80% 
*** Average game length: All: 86 moves (wins: 84 moves / draws: 86 moves / losses: 91 moves) 
*** Avg. eng play white: All: 84 moves (wins: 82 moves / draws: 85 moves / losses: 89 moves) 
*** Avg. eng play black: All: 87 moves (wins: 88 moves / draws: 86 moves / losses: 92 moves) 
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
*** Rank 16: "Arasan 23.3 NN"  All games: 2000 (367 wins / 1261 draws / 372 losses)
*** Overall score: 49.88%     Overall draw-rate: 63.05% 
*** Average game length: All: 84 moves (wins: 89 moves / draws: 84 moves / losses: 79 moves) 
*** Avg. eng play white: All: 84 moves (wins: 88 moves / draws: 83 moves / losses: 81 moves) 
*** Avg. eng play black: All: 85 moves (wins: 91 moves / draws: 86 moves / losses: 78 moves) 
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
*** Rank 17: "Igel 3.1.0 NN"  All games: 2000 (299 wins / 1329 draws / 372 losses)
*** Overall score: 48.18%     Overall draw-rate: 66.45% 
*** Average game length: All: 81 moves (wins: 89 moves / draws: 80 moves / losses: 76 moves) 
*** Avg. eng play white: All: 77 moves (wins: 87 moves / draws: 73 moves / losses: 76 moves) 
*** Avg. eng play black: All: 85 moves (wins: 94 moves / draws: 86 moves / losses: 76 moves) 
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
*** Rank 18: "Uralochka 3.37c NN"  All games: 2000 (313 wins / 1256 draws / 431 losses)
*** Overall score: 47.05%     Overall draw-rate: 62.80% 
*** Average game length: All: 91 moves (wins: 86 moves / draws: 93 moves / losses: 88 moves) 
*** Avg. eng play white: All: 90 moves (wins: 85 moves / draws: 93 moves / losses: 87 moves) 
*** Avg. eng play black: All: 92 moves (wins: 88 moves / draws: 95 moves / losses: 89 moves) 
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
*** Rank 19: "Marvin 6.0.0 NN"  All games: 2000 (122 wins / 966 draws / 912 losses)
*** Overall score: 30.25%     Overall draw-rate: 48.30% 
*** Average game length: All: 82 moves (wins: 86 moves / draws: 79 moves / losses: 84 moves) 
*** Avg. eng play white: All: 81 moves (wins: 85 moves / draws: 79 moves / losses: 86 moves) 
*** Avg. eng play black: All: 82 moves (wins: 88 moves / draws: 81 moves / losses: 83 moves) 
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
*** Rank 20: "Berserk 9 NN"  All games: 1996 (760 wins / 1198 draws / 38 losses)
*** Overall score: 68.09%     Overall draw-rate: 60.02% 
*** Average game length: All: 94 moves (wins: 91 moves / draws: 97 moves / losses: 88 moves) 
*** Avg. eng play white: All: 95 moves (wins: 90 moves / draws: 101 moves / losses: 84 moves) 
*** Avg. eng play black: All: 93 moves (wins: 92 moves / draws: 94 moves / losses: 88 moves) 
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
*** Rank 21: "Seer 2.5.0 NN"  All games: 2000 (496 wins / 1274 draws / 230 losses)
*** Overall score: 56.65%     Overall draw-rate: 63.70% 
*** Average game length: All: 85 moves (wins: 94 moves / draws: 83 moves / losses: 80 moves) 
*** Avg. eng play white: All: 87 moves (wins: 93 moves / draws: 85 moves / losses: 81 moves) 
*** Avg. eng play black: All: 83 moves (wins: 96 moves / draws: 81 moves / losses: 80 moves) 
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Ranking is with short win games (I delete the information on Stefan's stat file because 20 engines have good stats for short wins), not Seer but to have a defender with a small move average is super for the field. What you can see here are the strongest attackers with a lesser move average. I will not lost to many test-time with engines have a "Horror move-average".
Frank Quisinsky
Posts: 6808
Joined: Wed Nov 18, 2009 7:16 pm
Location: Gutweiler, Germany
Full name: Frank Quisinsky

Re: 18 engines I find out ... for test-set experiments!

Post by Frank Quisinsky »

Code: Select all

     Name                       Games     Win     Draw     Lose       Pts         S-B         %

Dragon 3 NN (Komodo)         :  2000  : 1050+  :  949=  :    1-  :  1524.5  : 1442587.00  :  76.23%
Berserk 9 NN                 :  2000  :  761+  : 1201=  :   38-  :  1361.5  : 1277172.00  :  68.08%
Revenge 3.0 NN exp           :  2000  :  727+  : 1209=  :   64-  :  1331.5  : 1246751.75  :  66.58%
SlowChess Blitz 2.9 NN       :  2000  :  605+  : 1301=  :   94-  :  1255.5  : 1178802.75  :  62.78%
rofChade 3.0 NN              :  2000  :  549+  : 1302=  :  149-  :  1200.0  : 1117931.50  :  60.00%
Stockfish 11                 :  2000  :  540+  : 1305=  :  155-  :  1192.5  : 1119246.25  :  59.63%
Andscacs 0.1 NNSf            :  2000  :  514+  : 1328=  :  158-  :  1178.0  : 1105562.50  :  58.90%
RubiChess 20220223 NN        :  2000  :  539+  : 1276=  :  185-  :  1177.0  : 1094093.75  :  58.85%
Seer 2.5.0 NN                :  2000  :  496+  : 1274=  :  230-  :  1133.0  : 1053792.00  :  56.65%
Arasan 23.3 NN               :  2000  :  367+  : 1261=  :  372-  :   997.5  :  928311.00  :  49.88%
Igel 3.1.0 NN                :  2000  :  299+  : 1329=  :  372-  :   963.5  :  899108.00  :  48.18%
Uralochka 3.37c NN           :  2000  :  313+  : 1256=  :  431-  :   941.0  :  875890.75  :  47.05%
Fire 8.2                     :  2000  :  290+  : 1182=  :  528-  :   881.0  :  812957.25  :  44.05%
Wasp 5.58 NN dev             :  2000  :  247+  : 1239=  :  514-  :   866.5  :  807382.25  :  43.33%
Rodent 1.0 NNSf              :  2000  :  212+  : 1256=  :  532-  :   840.0  :  786560.75  :  42.00%
Tucano 10.00 NN              :  2000  :  202+  : 1227=  :  571-  :   815.5  :  758122.00  :  40.78%
Wasp 5.50 NN                 :  2000  :  169+  : 1093=  :  738-  :   715.5  :  659652.25  :  35.78%
Velvet 4.0.1 NN              :  2000  :  192+  : 1041=  :  767-  :   712.5  :  662247.50  :  35.63%
Booot 6.5                    :  2000  :  147+  : 1026=  :  827-  :   660.0  :  612653.25  :  33.00%
Xiphos 0.6                   :  2000  :  146+  : 1005=  :  849-  :   648.5  :  604002.75  :  32.43%
Marvin 6.0.0 NN              :  2000  :  122+  :  966=  :  912-  :   605.0  :  563242.75  :  30.25%
Ineresting is Stockfish 11.
Lost 155 games but no games below 60 moves (same quantity of fast won games as Stockfish 15).
So we can see how Neural Network is working. NN Elo gain comes to 75% from the very late mid-games - endgames ...
Can be see in many stats I made here.

I will give Komodo 14.1 MCTS (last version without NN) and Pedone 3.1 NN (last Pedone version a chance).
Maybe I can hold the field of 21 engines, 2000 games for each engine is more easy for statistics later with test-set positions in comparing to this basic tournament (each one vs. each other 100 games).

Not important here to have 2x Wasp or Revenge / Pedone or Dragon / Komodo in the field.
It's not an Elo list.

Best
Frank
Frank Quisinsky
Posts: 6808
Joined: Wed Nov 18, 2009 7:16 pm
Location: Gutweiler, Germany
Full name: Frank Quisinsky

Re: 18 engines I find out ... for test-set experiments!

Post by Frank Quisinsky »

Code: Select all

Games   = 21000   ( no result = 0,  FEN tags = 0 )
Players = 21   ( clusters = 1 )
Date Range: 2022.07.22 - 2022.08.10

Games with:  WhiteElo = 0   BlackElo = 0   BothElos = 0

White Wins = 5716 ( 27.22 % )
Draws      = 12513 ( 59.59 % )
Black Wins = 2771 ( 13.2 % )
White Pct = 57.01 %
Black Pct = 42.99 %

ECO:  Total = 21000  A: 3380  B: 5163  C: 3835  D: 4126  E: 4496
PlyCount:  Total = 21000  Range: 39-594  Average = 171.89  StdDev = 73.36

finished: be sure to rename/copy outSummary
40 moves in 8 minutes
Ponder = off
512Mb for hash
5-pieces endgames bases
without resign mode

Test-set have to beat:


1. Move average from 171.89 halfmoves
2. draw quote 59,59%


But at first I have to replace Berserk 9.0 NN and Andscacs 0.1 NNSf (wrote that before).
Test-field isn't optimal.

Best
Frank
Frank Quisinsky
Posts: 6808
Joined: Wed Nov 18, 2009 7:16 pm
Location: Gutweiler, Germany
Full name: Frank Quisinsky

Re: 18 engines I find out ... for test-set experiments!

Post by Frank Quisinsky »

OK, I changed that ...
Hope of a lesser move-average and many short wins.

1. Komodo 14.1 MCTS Agg in the field ... never I try out "MCTS" before (Last Komodo without Neural Network is playing with Contempt = 0).
Important for stats is later to have different engines in the field without Neural Network (now 5/21 with Komodo 14.1).

2. Pedone 3.1 NN in the field (here I am very sure it will be a fight for place 1 ... with Velvet 4.0.1 NN for short wins.
Can be see later with the tool from Stefan very easy!

https://www.amateurschach.de/fling/fcp_ ... ecret.html

And the experiment can start later with the first 50 strong opening positions I found out.
And we can compare with the basic tournament the draw quote and the move average with test-positions.
That ist the goal.

So, the 21 engines in the field are more or less team-worker for building a great test-set.
Sure, it need time!

If I offer such a test-set (have promised it to a few people) it must be perfect or the others invest time and waste electricity. Maybe the ready test-set can be interesting for the next 3-4 years. I don't know how strong are the engines in 3-4 years.

Best
Frank