Release of my Engines Aggressiveness Statistic Tools

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Dann Corbit, Harvey Williamson

User avatar
pohl4711
Posts: 2388
Joined: Sat Sep 03, 2011 7:25 am
Location: Berlin, Germany
Full name: Stefan Pohl

Release of my Engines Aggressiveness Statistic Tools

Post by pohl4711 »

I proudly release my 2 Engine Aggressiveness Statistics Tools (EAS-Tools).
With these tools, the aggressiveness of engines (or human players) can be measured. How do the tools work?
The tools are searching for sacrifices in games (from 1 pawnunit-sacs up to 5+ pawnunit-sacs) and for won games, which are very short (split into games up to 40/45/50/55/60 moves), but only, if the short game ended before the endgame (a low material check is done!). Because a weaker player can be playing aggressive, too, the EAS-Score (= Engine Aggressivenes Score, see explanation below) and all other statistics are build on percents from the won games of an engine/player. So, if an engine has won more games, it must win more short games or win games with sacrifices. A weaker engine, which has won less games, need less wins of short games or win games with sacrifices.

What is the EAS-Score?
EAS-Score is: (percent*10) of the percent-values of the sacs (1-5+ pawnunits) calculated out of the won games by the engine, only. Higher pawnunits-sacs give bonus-points:
1 pawnsac = 3x points *** 2 pawnsac = 10x points *** 3 pawnsac = 25x points 4 pawnsac = 50x points *** 5+ pawnsac = 100x points
Additionally, very short won games (percent*10) with or without sacs give bonus-points: 60 moves or less= 3x points *** 55 moves or less= 10x points *** 50 moves or less= 25x points 45 moves or less= 50x points *** 40 moves or less= 100x points (but only, if the short game ended before endgame (low material check is done!))

Of course, the EAS-Score is not a fixed value. It highly depends on the strength of players and opponents. And in engine-tournaments it depends on thinking-time, opening-sets, PC-speed etc. But, IMO, the EAS-Score can be very helpful especially for engine-developers, which test their engine dev-versions (or new neural-nets) always in the same way and versus the same opponents. In this case, the EAS-Score quickly shows progress or regress in aggressiveness...
And in a RoundRobin-tournament or used on the gamebase of a ratinglist, the EAS-Score ratinglist (see the statistics_EAS_rating.txt file) can be very interesting.
The EAS-Score of humans is much higher, than the EAS-Score of engines, because humans win much more short games, because of blunders, played by the (human) opponent... So never compare EAS-Scores of humans and engines!!!
The EAS-tool for one engine/player writes 5 pgn-files with all found sacrifice games. All short games, which ended before endgame, are stored in short_midgame_wins.pgn. And a statistics.txt -file for the engine, with all values of interest, inccluding the EAS-Score.
The EAS-tool for all engines/players writes a statistics_EAS_rating textfile, where all engines/players are ranked by their EAS-Score. And a second ratinglist below, with all important stats: number of wins, sacrifices and short-wins (before endgame) by the engine. Additionally all interesting games (sac-games and all short games, which ended before endgame, are stored in shorts_and_sacs_games.pgn (without doubles and comments)

Download from my website:
https://www.sp-cc.de/files/engines_aggr ... cs_tool.7z
User avatar
pohl4711
Posts: 2388
Joined: Sat Sep 03, 2011 7:25 am
Location: Berlin, Germany
Full name: Stefan Pohl

Re: Release of my Engines Aggressiveness Statistic Tools

Post by pohl4711 »

Here the statistics calculated by the One_Engine_Aggressiveness_Stats-Tool of Stockfish 220205 (last SPCC-tested version before
introducing the new nnue-net architecture) and Stockfish 220210 (first SPCC-tested version with new net arch).
Mention: SF 220210 scored less points and played more draws than SF 220205, but the EAS-Tool reveals very clearly,
that SF 220210 with new net-arch plays much more aggressive! Number of short wins is higher, number of played sacrifices is higher
and the final EAS-Score is clearly higher (2146 to 3260) !!!

********************************************************************************************
*** Evaluated file: SF_220205.pgn ***
*********************************************************************************************
*** Number of games played by "Stockfish 220205 avx2": 7000
*********************************************************************************************
*** Number of draws played by "Stockfish 220205 avx2": 3691 (52.72%)
*********************************************************************************************
*** Number of won games (no doubles) played by "Stockfish 220205 avx2": 3284 (46.91%)
*********************************************************************************************
*** Overall scoring rate by "Stockfish 220205 avx2": 73.2%
*********************************************************************************************
*** Following percent-values calculated on the number of all won games, only (3284)
*** played by "Stockfish 220205 avx2"
*********************************************************************************************
*** Number of won games up to 40 moves: 13 (0.39%)
*** Number of won games up to 45 moves: 15 (0.45%)
*** Number of won games up to 50 moves: 27 (0.82%)
*** Number of won games up to 55 moves: 43 (1.30%)
*** Number of won games up to 60 moves: 24 (0.73%)
*** (all short games, counted here, ended before endgame (low material))
*** Number of all short wins (total) : 122 (3.71%)
*********************************************************************************************
*** Sacrifices found in the won games of "Stockfish 220205 avx2"
*** Number of 5+ pawnunits sacrifices: 8 (0.24%)
*** Number of 4 pawnunits sacrifices : 13 (0.39%)
*** Number of 3 pawnunits sacrifices : 30 (0.91%)
*** Number of 2 pawnunits sacrifices : 141 (4.29%)
*** Number of 1 pawnunit sacrifices : 331 (10.07%)
*** Number of all sacrifices : 523 (15.92%)
*********************************************************************************************
*** EAS-Tool Score points : 2146
*********************************************************************************************
*********************************************************************************************
*********************************************************************************************
*********************************************************************************************
*** Evaluated file: SF_220210.pgn ***
*********************************************************************************************
*** Number of games played by "Stockfish 220210 avx2": 7000
*********************************************************************************************
*** Number of draws played by "Stockfish 220210 avx2": 3754 (53.62%)
*********************************************************************************************
*** Number of won games (no doubles) played by "Stockfish 220210 avx2": 3231 (46.15%)
*********************************************************************************************
*** Overall scoring rate by "Stockfish 220210 avx2": 72.9%
*********************************************************************************************
*** Following percent-values calculated on the number of all won games, only (3231)
*** played by "Stockfish 220210 avx2"
*********************************************************************************************
*** Number of won games up to 40 moves: 22 (0.68%)
*** Number of won games up to 45 moves: 16 (0.49%)
*** Number of won games up to 50 moves: 31 (0.95%)
*** Number of won games up to 55 moves: 38 (1.17%)
*** Number of won games up to 60 moves: 34 (1.05%)
*** (all short games, counted here, ended before endgame (low material))
*** Number of all short wins (total) : 141 (4.36%)
*********************************************************************************************
*** Sacrifices found in the won games of "Stockfish 220210 avx2"
*** Number of 5+ pawnunits sacrifices: 11 (0.34%)
*** Number of 4 pawnunits sacrifices : 28 (0.86%)
*** Number of 3 pawnunits sacrifices : 62 (1.91%)
*** Number of 2 pawnunits sacrifices : 183 (5.66%)
*** Number of 1 pawnunit sacrifices : 390 (12.07%)
*** Number of all sacrifices : 674 (20.86%)
*********************************************************************************************
*** EAS-Tool Score points : 3260
*********************************************************************************************
User avatar
pohl4711
Posts: 2388
Joined: Sat Sep 03, 2011 7:25 am
Location: Berlin, Germany
Full name: Stefan Pohl

Re: Release of my Engines Aggressiveness Statistic Tools

Post by pohl4711 »

Here the EAS-ratinglist of my SPCC-ratinglist gamebase. IMO it is spectacular. Danasah 9 for example is on rank 2 in the EAS-ratinglist, but on rank 37 (of 38) in my regular SPCC-ratinglist. So, Danasah 9 plays not so strong, but extremly aggressive. And see, how aggressive Velevet 3.3.0 is! These 2 engines are clearly better than all others (even Wasp on place 3 plays way less aggressive (see the much weaker score!) - Wasp plays a lot of short games, but a small amount of sacs). But Velvet and Danasah won around 25% of their win-games with a sacrifice - thats amazing!
And see, how bad the EAS-score of the 3 strong engines Ethereal, Koivisto and Berserk is...


*****************************************************************************
*** Engine Aggressiveness Tool V1.7 Score points Ratinglist
*****************************************************************************
Score is: (percent*10) of the percent-values of the sacs (1-5+ pawnunits)
calculated out of the won games by the engine, only.
So, a weak engine (with a small number of won games) can get
a high EAS-scoring, too, when the percent of sac-games in the won games
is high (and the number of short wins). Higher pawnunits-sacs give bonus-points:
1 pawnsac = 3x points *** 2 pawnsac = 10x points *** 3 pawnsac = 25x points
4 pawnsac = 50x points *** 5+ pawnsac = 100x points
Additionally, very short won games (percent*10) give bonus-points, when they
ended before endgame (a check for low material is done):
60 moves= 3x points *** 55 moves= 10x points *** 50 moves= 25x points
45 moves= 50x points *** 40 moves= 100x points.
*****************************************************************************
*** Evaluated file: All_Stockfish.pgn ***
*****************************************************************************
Rank EAS-Score sacs shorts Engine/player
-----------------------------------------------------------------------------
001 009053 24.08% 12.06% "Velvet 3.3.0 avx2"
002 008480 26.24% 11.53% "Danasah 9.0 avx2"
003 005780 12.83% 11.31% "Wasp 5.20 avx"
004 005440 20.07% 05.37% "Revenge 2.0 avx2"
005 005311 17.89% 09.09% "Tucano 10.00 avx2"
006 004610 17.62% 06.71% "Arasan 23.3 avx2"
007 004458 16.35% 05.70% "Lc0 0.28.0 744706"
008 004074 12.52% 08.86% "Scorpio 3.0.14d cpu"
009 003952 23.77% 03.60% "Coiled 1.1 avx2"
010 003888 08.97% 08.44% "Fritz 18 x64"
011 003775 15.43% 07.40% "Stockfish final HCE"
012 003773 14.57% 07.94% "Igel 3.0.5 popavx2"
013 003694 20.67% 02.31% "Gogobello 3 avx2"
014 003599 12.33% 06.48% "Marvin 5.2 avx2"
015 003529 16.84% 06.55% "Velvet 3.2.0 avx2"
016 003503 10.04% 07.57% "Weiss 2.0 popc"
017 003389 21.24% 04.67% "Stockfish 220319 avx2"
018 003326 18.46% 02.75% "Toga 4 1.11 avx2"
019 003155 18.92% 04.40% "Stockfish 220308 avx2"
020 003133 17.70% 03.25% "Fire 8.NN MCTS avx2"
021 002982 10.37% 06.56% "Combusken 2.0.0 amd64"
022 002893 16.94% 05.43% "Stockfish 14 210702"
023 002817 09.39% 06.16% "Chiron 5 x64"
024 002388 15.26% 03.00% "Slow Chess 2.83 avx2"
025 002347 07.78% 05.02% "Minic 3.18 znver3"
026 002285 18.17% 03.52% "Stockfish 14.1 211028"
027 002202 19.06% 02.78% "KomodoDragon 2.6 MCTS"
028 002194 20.07% 03.63% "KomodoDragon 2.6 avx2"
029 002081 13.61% 02.13% "Fire 8.NN avx2"
030 001950 10.68% 04.12% "RubiChess 2021 avx2"
031 001899 12.17% 04.58% "Nemorino 6.00 avx2"
032 001759 05.17% 05.98% "Clover 3.0 avx2"
033 001726 07.44% 03.45% "Zahak 9.0 avx"
034 001703 07.63% 03.87% "Stash 32.0 popc"
035 001649 08.74% 02.45% "Berserk 8.5 avx2"
036 001206 13.93% 02.16% "Ethereal 13.50 nnue"
037 000773 08.33% 01.25% "Koivisto 8.0 avx2"
038 000767 09.06% 00.90% "Seer 2.4.0 avx2"
****************************************************
*** EAS-tool (C) 2022 Stefan Pohl (www.sp-cc.de) ***
****************************************************

(for this ratinglist, the tool had to evaluate more than 180000 games. That took around 40 minutes on a "normal" PC)
Frank Quisinsky
Posts: 6808
Joined: Wed Nov 18, 2009 7:16 pm
Location: Gutweiler, Germany
Full name: Frank Quisinsky

Re: Release of my Engines Aggressiveness Statistic Tools

Post by Frank Quisinsky »

Hi,

yes, can be see in short-game-stats from my FCP Tourneys since a long time.
Also in the Excel tables by Klaus Wlotzka.

Danasah is indeed very strong here !!

In my opinion the short game-stats should be view in combination with short-lost games.

A good example:
Spark for many years ... in relation to Elo Spark was many years on place 1.
Stronger as Stockfish ...

The aera before Spark ...
Glaurung / Gothmog, Aristarch, ETChess

The aera before all these engines:
Phalanx 22 (on place 1 in the World in short-won games in times Fritz, Hiarcs, Junior, Tiger, Shredder are on place 1).
Or Anaconda / Gromit ... the style from the engines Frank Schneider produced are all the time the same.

Hakka ... good to compare, but lost a lot of games very fast, not Spark, not Wasp.

Fizbo is the next good example:
Very speculative playing style, won a lot of fast games, but lost a lot.

In case of Fritz 18 ...
Fritz 18 is also very speculative with attacking moves.
Quantity of short-lost games is very high.

Such statistics are all the time very interesting.
For me since the beginning of computer chess.

The quantity of such games is much smaller with longer time controls I am using.
6-Minutes games by yourself on 3Ghz hardware to 80-minutes games by myself on 4.5Ghz hardware.
A big different!!

But the final results will be "around" the same.

Have a look on the short-lost games!!
For Danasah for one example only!

The secret ist ...
Many short-won-games without to produced short-lost games.
Optimal combination for new and interesting chess computer projects (the style most humanes like).

:-)

Best
Frank

In relation to playing strength (combination short-won / short-lost games).
Wasp is since a longer time one of the TOP-3 engines in the World.
Strong are Pedone / Revenge 1.0 ... Revenge 1.0 much stronger as Revenge 2.0.
Booot are all the time very strong.

Very important statistics for opening book projects with many engines.
But important is also to looking on short-lost games, not only on short-won games.


Code: Select all

March 10th, 2022
Short games statistic, run-2
After round 30 out of 30 =  1.200 games per engine, final results for run-2

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

  970 of 24.600 = 3,943% : Miniatures, mate below 60 moves
  943 of 24.600 = 3,833% : Fast draws below 40 moves

                                 won59 lost59 draw39        won59w won59b lost59w lost59b
01. ShashChess 20.2 NN             115      0     40            84     31       0       0
02. Dragon 2.6 NN (Komodo)         101      0     16            75     26       0       0
03. Stockfish 311221 NN dev         89      0     37            49     40       0       0
04. Revenge 2.0 NN                  50      2     57            35     15       1       1
05. SlowChess Blitz 2.83 NN         42      1     31            28     14       0       1
06. Fire 8 MC.3 NNSf                39      0     31            29     10       0       0
07. rofChade 2.317 NN dev           34      5     39            22     12       1       4
08. Igel 3.0.10 NN                  34      9     81            26      8       1       8
09. Berserk 9 dev                   32      3     35            21     11       1       2
10. Wasp 5.26 NN dev                28      3     55            13     15       0       3
11. Tucano 10.00 NN                 28     10     71            20      8       4       6
12. Arasan 23.2 NN                  27      4     41            20      7       1       3
13. RubiChess 2021 NN               23      3     55            16      7       0       3
14. Booot 7.0 NN dev                21      3     80            17      4       2       1
15. Rebel 14.1 NN                   19      8     47            18      1       1       7
16. DanaSah 9.0 NN                  19     34     49            17      2      14      20
17. Lc0 0.28.2 752187 CPU           19     46     65            15      4       8      38
18. Koivisto 7.13 NN                17      7     23            11      6       3       4
19. Nemorino 6.09 NN dev            17     20     41            10      7       3      17
20. Rodent 1.0 NNSf                 15      8     88            12      3       1       7
21. Schooner 2.2 XB                 15     18     42            11      4       1      17
22. Minic 3.18 NN                   15     26     53             9      6       7      19
23. Fritz 18 (Ginkgo)               14     32     33            11      3       5      27
24. Velvet 3.2.0 NN                 13     20     35            10      3       4      16
25. Andscacs 0.95.123 dev           13     40     36            10      3      12      28
26. Xiphos 0.6                      12     11     41             9      3       4       7
27. Weiss 2.0                       12     17     37             8      4       3      14
28. Coiled 1.1 NNSf                 12     21     47             7      5       7      14
29. Shredder 13                     12     32     42             8      4      10      22
30. Hiarcs 15                       12     37     64             8      4      15      22
31. Laser 1.7                       10     22     38             7      3       7      15
32. Defenchess 2.3 dev              10     24     45             6      4       4      20
33. Combusken 2.0.0                 10     30     38             5      5       6      24
34. Chiron 5                        10     48     31             7      3      17      31
35. Marvin 5.2.0 NN                  9     20     49             6      3       8      12
36. Fizbo 2.0                        8     29     37             5      3      12      17
37. Seer 2.4.0 NN                    4     12     56             3      1       2      11
38. Clover 3.0 NN                    4     45     44             2      2      20      25
39. Zahak 9.0                        4     84     47             4      0      38      46
40. Bit-Genie 9.19 dev               1     91     56             1      0      25      65
41. Black Marlin 4.0 NN              1    144     33             1      0      43     100

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

  313 of 24.600 = 1,272% : Miniatures, mate below 50 moves
  305 of 24.600 = 1,239% : Fast draws below 30 moves

                                 won49 lost49 draw29        won49w won49b lost49w lost49b
01. ShashChess 20.2 NN              43      0     15            36      7       0       0
02. Dragon 2.6 NN (Komodo)          36      0      5            25     11       0       0
03. Stockfish 311221 NN dev         31      0     19            17     14       0       0
04. SlowChess Blitz 2.83 NN         20      0      7            12      8       0       0
05. Revenge 2.0 NN                  15      0     17            11      4       0       0
06. Fire 8 MC.3 NNSf                13      0      8            11      2       0       0
07. rofChade 2.317 NN dev           12      2     18             7      5       0       2
08. Igel 3.0.10 NN                  12      3     25             9      3       1       2
09. Rebel 14.1 NN                    9      0     15             9      0       0       0
10. DanaSah 9.0 NN                   9     15     20             8      1       6       9
11. Arasan 23.2 NN                   8      1     11             5      3       1       0
12. Berserk 9 dev                    8      1     12             7      1       1       0
13. Wasp 5.26 NN dev                 8      1     17             4      4       0       1
14. Shredder 13                      7      6     14             4      3       0       6
15. Booot 7.0 NN dev                 6      0     21             3      3       0       0
16. RubiChess 2021 NN                6      1     12             6      0       0       1
17. Andscacs 0.95.123 dev            6      5     10             5      1       0       5
18. Defenchess 2.3 dev               5      3     22             2      3       1       2
19. Fritz 18 (Ginkgo)                5     11     11             3      2       3       8
20. Tucano 10.00 NN                  4      2     18             3      1       0       2
21. Coiled 1.1 NNSf                  4      3     14             3      1       1       2
22. Weiss 2.0                        4      4     17             3      1       0       4
23. Hiarcs 15                        4      7     22             2      2       2       5
24. Xiphos 0.6                       3      1     17             2      1       1       0
25. Rodent 1.0 NNSf                  3      1     29             2      1       0       1
26. Koivisto 7.13 NN                 3      3      9             1      2       1       2
27. Seer 2.4.0 NN                    3      3     16             2      1       1       2
28. Velvet 3.2.0 NN                  3      4     11             3      0       0       4
29. Laser 1.7                        3      4     12             2      1       0       4
30. Marvin 5.2.0 NN                  3      5     14             2      1       2       3
31. Minic 3.18 NN                    3      6     17             2      1       0       6
32. Combusken 2.0.0                  3     12     15             1      2       4       8
33. Lc0 0.28.2 752187 CPU            3     15     16             2      1       4      11
34. Schooner 2.2 XB                  2      2     11             0      2       0       2
35. Fizbo 2.0                        2      4     10             0      2       3       1
36. Chiron 5                         1      8     10             1      0       4       4
37. Nemorino 6.09 NN dev             1      9     15             1      0       2       7
38. Clover 3.0 NN                    1     22     13             1      0      12      10
39. Zahak 9.0                        1     36     14             1      0      17      19
40. Bit-Genie 9.19 dev               0     43     21             0      0       9      34
41. Black Marlin 4.0 NN              0     70     10             0      0      19      51
Frank Quisinsky
Posts: 6808
Joined: Wed Nov 18, 2009 7:16 pm
Location: Gutweiler, Germany
Full name: Frank Quisinsky

Re: Release of my Engines Aggressiveness Statistic Tools

Post by Frank Quisinsky »

With other words ...
DanaSah is today the Fizbo from the past.
Lost to many games very fast.
Most speculative engine today!

Have a look on the fast lost games from Wasp.
This is perfect in combination with all the fast won games.
Same stats Spark have in the past, or Phalanx have in the beginning in Winboard times.

But Wasp have much more aggressivess in pawn moves against the king position Spark had.
Later in the games the pawn positions are often the reason that Wasp lost games vs. stronger engines.

Engines by Tord and Frank Schneider are all the time very interesting.

It make sense to looking on short-lost games in your stats.
To build a combination from short-won / short-lost games for building the style of an engine for the most important "mid-games".

New for me only is your information to:
Velvet 3.3.0

I will looking on Velvet games during my "run-4" is still running!

:-)
Frank Quisinsky
Posts: 6808
Joined: Wed Nov 18, 2009 7:16 pm
Location: Gutweiler, Germany
Full name: Frank Quisinsky

Re: Release of my Engines Aggressiveness Statistic Tools

Post by Frank Quisinsky »

Stefan,

you have to added Pedone (last free version) in your list.
Or Pedone without NN ... the version before.
Spark isn't strong enough in the group of engines today!

Maybe you have a new number 1 (if you looking not on short-lost games).

:-)

Best
Frank
User avatar
pohl4711
Posts: 2388
Joined: Sat Sep 03, 2011 7:25 am
Location: Berlin, Germany
Full name: Stefan Pohl

Re: Release of my Engines Aggressiveness Statistic Tools

Post by pohl4711 »

I added a new site to my website:

https://www.sp-cc.de/eas-ratinglist.htm

There, the EAS-ratinglist and the ranking of the engines in my SPCC-ratinglist can be compared directly. I think, this is helpful...