48,000 game RR with 16 engines 2m + 2s

Discussion of computer chess matches and engine tournaments.

Moderator: Ras

User avatar
MikeB
Posts: 4889
Joined: Thu Mar 09, 2006 6:34 am
Location: Pen Argyl, Pennsylvania

48,000 game RR with 16 engines 2m + 2s

Post by MikeB »

Games will start shortly, auto updates will be made after every round (240 games ) roughly every 2 hours and 45 mins - will let it run run along s I can - so first update will be around 6AM EST.

Results: https://www.dropbox.com/s/cx9vntj8n8myw ... a.txt?dl=1

PGN Files:https://www.dropbox.com/s/tmwepshusdy48h7/all.pgn?dl=1

Engines:
arasan.ini
crafty-25.1.Syzy-0611d - (current dev crafty with syzygy support)
shredder
hiarcs-14
senpai
kohai-1.0 (senpai derivative)
hakka322
texel
Mikhail-0.04 (Firenzina derivative)
Stockfish-DDi
critter
Gull-3.0
Hawkeye-1.0 (Gull derivative)
komodo-10
Stockfish-062816-x (current SF)
Stockfish-062816-xy (modified SF)


Using Bob Hyatt's automated script (modified)...

Code: Select all

set eng1 = arasan.ini
set eng2 = crafty-25.1.Syzy-0611d
set eng3 = shredder.ini
set eng4 = hiarcs-14.ini
set eng5 = senpai.ini
set eng6 = kohai-1.0.ini
set eng7 = hakka322.ini
set eng8 = texel.ini
set eng9 = Mikhail-0.04.ini
set eng10 = Stockfish-DD.ini
set eng11 = critter.ini
set eng12 = Gull-3.0.ini
set eng13 = Hawkeye-1.0.ini
set eng14 = komodo-10.ini
set eng15 = Stockfish-062816-x.ini
set eng16 = Stockfish-062816-xy.ini

set opponents1=( $eng1 $eng2 $eng3 $eng4 $eng5 $eng6 $eng7 $eng8 $eng9 $eng10 $eng11 $eng12 $eng13  $eng14 $eng15 $eng16 )
set opponents2=( $eng16 $eng15 $eng14 $eng13 $eng12 $eng11 $eng10 $eng9 $eng8 $eng7 $eng6 $eng5 $eng4 $eng3 $eng2 $eng1 )

set players1=( 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 )
set players2=( 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 )

set number_of_players1 = 16
set number_of_players2 = 16

set file = new2500.epd
#set file = tcec50.epd

@ positions  = 200
@ rounds  = $positions * 2
set time_odds = 1
set games_per_position = 1
set positions_per_match = 1

echo "using $positions positions from EPD file"
@ gamesPlanned =  ( $number_of_players1  )  * ( $number_of_players2 - 1 )   * $positions * $games_per_position
echo "gamesPlanned"
@ counterStart = ( $gamesPlanned  / $positions ) + 10
@ counterIncrement = ( $counterStart - 10 )

cd run
set logid1 = 101
set logid2 = 50101
rm -f commands
@ pairings = 0

@ position = 0 #you can change this to start @ different position in the file

set counter = 0
set gamesStarted = 0
loop:
#@ position = $position + 1 # start at the  1st position ,
@ position = `jot -r 1 1 2500` #for random selection
	foreach player1 ($players1)
		foreach player2 ($players2)
		if ($opponents1[$player1] == $opponents2[$player2] ) goto skip  #  do not want engines to play themselves
			echo "./match "match$player1.$position" $file $position $games_per_position $positions_per_match $time $inc  $opponents1[$player1] localhost $logid1 $opponents2[$player2] localhost $logid2 $time_odds" >> commands
			@ logid1 = $logid1 + $games_per_position * $positions_per_match
			@ logid2 = $logid2 + $games_per_position * $positions_per_match
			@ pairings = ( $pairings + $games_per_position )
			echo "$pairings"
			@ completed = ( $pairings - 10 )

		@ gamesStarted = $pairings - $counter
		if ($gamesStarted < $counterStart ) goto skip_1
			echo "echo " "" >> commands
			echo "echo "$completed of $gamesPlanned games completed..."" >> commands
			echo "echo "time control: $time+$inc"" >> commands
			echo "./geteloRR1" >> commands
			echo "echo " ">>~/Dropbox/Shared\ Folder/elota.txt" >> commands
			echo "echo " ">>~/Dropbox/Shared\ Folder/elotb.txt" >> commands
			echo "echo "$completed of $gamesPlanned games completed...">>~/Dropbox/Shared\ Folder/elota.txt" >> commands
			echo "echo "$completed of $gamesPlanned games completed...">>~/Dropbox/Shared\ Folder/elotb.txt" >> commands
			echo "echo "time control: $time+$inc">>~/Dropbox/Shared\ Folder/elota.txt" >> commands
			echo "echo "time control: $time+$inc">>~/Dropbox/Shared\ Folder/elotb.txt" >> commands
			echo "./geteloRR2>>~/Dropbox/Shared\ Folder/elota.txt" >> commands
			echo "./geteloRR3>>~/Dropbox/Shared\ Folder/elotb.txt" >> commands
			echo "cp pgn /Users/michaelbyrne/Dropbox/Shared\ Folder/all.pgn" >> commands
			@ counter = $counter + $counterIncrement
		skip_1:
		skip:
		end
	end  # round is completed with both colors scripted

if ($pairings < $gamesPlanned) goto loop
echo "echo " "" >> commands
echo "echo $completed games completed" >> commands
echo "time control: $time+$inc" >.commands
echo "./geteloRR1" >> commands

echo "EOF" >> commands
touch run.1
foreach file (run.*)
  rm $file
end
@ record = 0
foreach run ({0,1,2,3,4,5,6,7,8,0}{0,1,2,3,4,5,6,7,8,9}{0,1,2,3,4,5,6,7,8,9}{0,1,2,3,4,5,6,7,8,9}{0,1,2,3,4,5,6,7,8,9})
  if ($run == "00000") continue
  @ record = $record + 1
  set command = `cat commands | head -$record | tail -1`
  if ($command[1] == "EOF") break
  touch run.$run
echo "$command &" >> run.$run
echo "wait" >> run.$run
  chmod +x run.$run
  if ($command[1] == "EOF") break
end
if ($run == "99999") then
  echo "ERROR, more than 99999 commands, fix script and retry"
  exit
endif
echo "$record run scripts were prepared."
cd ..
User avatar
MikeB
Posts: 4889
Joined: Thu Mar 09, 2006 6:34 am
Location: Pen Argyl, Pennsylvania

Re: 48,000 game RR with 16 engines 2m + 2s

Post by MikeB »

looks like my estimate will be right on - about 2 hours and 45 minutes for every update ( 240 completed). We have another 40 games to go for the nest auto update.

Code: Select all

[Mac-Pro:~/cluster.mfb] michaelbyrne% cat match[0-9]*.* > pgn
[Mac-Pro:~/cluster.mfb] michaelbyrne% bay
version 0057, Copyright (C) 1997-2010 Remi Coulom.
compiled Dec 17 2015 00:39:54.
This program comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it
under the terms and conditions of the GNU General Public License.
See http://www.gnu.org/copyleft/gpl.html for details.
ResultSet>rp pgn
440 game(s) loaded, 0 game(s) with unknown result ignored.
ResultSet>elo
ResultSet-EloRating>mm 1 1
00:00:00,00
ResultSet-EloRating>offset 3200
3200
ResultSet-EloRating>ratings
Rank Name                     Elo    +    - games score oppo. draws 
   1 Komodo 10 64-bit        3502   90   90    49   87%  3183   22% 
   2 Stockfish 062816-xy     3459   93   93    43   86%  3170   23% 
   3 Stockfish 062816-x      3432   88   88    44   83%  3178   25% 
   4 Hawkeye 1.0 x64         3347   70   70    57   70%  3183   35% 
   5 Gull 3 x64 (syzygy)     3296   68   68    58   64%  3185   34% 
   6 Stockfish DD 64 SSE4.2  3282   67   67    58   62%  3186   41% 
   7 Critter 1.6a 64-bit     3213   66   66    58   53%  3191   38% 
   8 Mikhail 0.04 OSx 64     3179   69   69    58   48%  3193   28% 
   9 Senpai 1.0              3133   68   68    57   42%  3191   35% 
  10 Kōhai 1.0               3101   71   71    56   37%  3190   30% 
  11 Texel 1.05 64-bit       3092   71   71    57   37%  3194   21% 
  12 Hakkapeliitta TCEC v2   3073   70   70    57   36%  3195   30% 
  13 Crafty-25.1.Syzy-0611d  3042   72   72    57   31%  3197   26% 
  14 Deep Shredder 12 x64    3032   72   72    57   30%  3198   25% 
  15 Deep HIARCS 14 WCSC     3020   74   74    57   29%  3199   26% 
  16 Arasan 19.0             2998   75   75    57   26%  3200   21%
You can see by the results, that this will complete the round for this position ,where every engine has play all the engines on this one position with both colors w&b. 199 positions are left. All the positions were chosen at random by the script from a 2500 position database created by sifting through the CCRL 40/40 games, looking for games that were not draws and after every 8 moves, the static score of the position was between 40 to 65 cp after a brief search.
User avatar
MikeB
Posts: 4889
Joined: Thu Mar 09, 2006 6:34 am
Location: Pen Argyl, Pennsylvania

Re: 48,000 game RR with 16 engines 2m + 2s

Post by MikeB »

just for clarity, the run is completed for a position after every 240 games - all engines would have played from both sides of the position with with every opponent. I track the games started by a counter in the script, so through the math, I know when exactly 240 games are completed. Of course some games may be short, others may be long and that is why you not always see a perfect 30 games played by each engine after every round - but it will work its way out by the completion of the 48,000 games.
User avatar
Ozymandias
Posts: 1537
Joined: Sun Oct 25, 2009 2:30 am

Re: 48,000 game RR with 16 engines 2m + 2s

Post by Ozymandias »

Your Hawkeye started strong, let's see if it can keep a good chunk of that +50. If you got that just by cleaning a third of Gull's warnings, the end result can be really interesting.
User avatar
MikeB
Posts: 4889
Joined: Thu Mar 09, 2006 6:34 am
Location: Pen Argyl, Pennsylvania

Re: 48,000 game RR with 16 engines 2m + 2s

Post by MikeB »

Hawkeyes is still holding on ..next update will be about 2:45 PM EST - every 2:45 minutes or so another round of 240 games completes

the bayelo table below is what you get when you follow the links, time control is in centiseconds 2 min + 2 sec

Code: Select all

720 of 48000 games completed...
time control: 12000+200
Date: 07/02/16 : 11:59:17
Rank Name                     Elo    +    - games score oppo. draws 
   1 Komodo 10 64-bit        3505   64   64    90   85%  3180   23% 
   2 Stockfish 062816-xy     3502   63   63    89   84%  3181   25% 
   3 Stockfish 062816-x      3489   61   61    91   84%  3178   26% 
   4 Hawkeye 1.0 x64         3331   52   52    90   67%  3191   39% 
   5 Stockfish DD 64 SSE4.2  3297   53   53    90   63%  3194   34% 
   6 Gull 3 x64 (syzygy)     3294   52   52    90   62%  3194   33% 
   7 Critter 1.6a 64-bit     3190   52   52    90   49%  3201   32% 
   8 Mikhail 0.04 OSx 64     3157   52   52    90   46%  3203   36% 
   9 Senpai 1.0              3115   53   53    89   39%  3202   36% 
  10 Hakkapeliitta TCEC v2   3110   54   54    90   40%  3206   29% 
  11 Kōhai 1.0               3094   54   54    90   36%  3207   29% 
  12 Texel 1.05 64-bit       3078   56   56    90   36%  3208   18% 
  13 Deep HIARCS 14 WCSC     3042   56   56    90   31%  3211   29% 
  14 Crafty-25.1.Syzy-0611d  3031   56   56    90   29%  3211   23% 
  15 Deep Shredder 12 x64    2995   58   58    90   26%  3214   19% 
  16 Arasan 19.0             2969   60   60    91   23%  3218   18% 
For those that interested - here is the los:

Code: Select all

ResultSet-EloRating>los
                        Ko St St Ha St Gu Cr Mi Se Ha K? Te De Cr De Ar
Komodo 10 64-bit           52 64 99 99 99 99 99100100100100100100100100
Stockfish 062816-xy     47    61 99 99 99 99 99100100100100100100100100
Stockfish 062816-x      35 38    99 99 99 99 99100100100100100100100100
Hawkeye 1.0 x64          0  0  0    81 83 99 99 99 99 99 99 99 99 99100
Stockfish DD 64 SSE4.2   0  0  0 18    52 99 99 99 99 99 99 99 99 99 99
Gull 3 x64 (syzygy)      0  0  0 16 47    99 99 99 99 99 99 99 99 99 99
Critter 1.6a 64-bit      0  0  0  0  0  0    80 97 97 99 99 99 99 99 99
Mikhail 0.04 OSx 64      0  0  0  0  0  0 19    86 88 94 97 99 99 99 99
Senpai 1.0               0  0  0  0  0  0  2 13    54 70 81 96 98 99 99
Hakkapeliitta TCEC v2    0  0  0  0  0  0  2 11 45    66 78 95 97 99 99
Kōhai 1.0               0  0  0  0  0  0  0  5 29 33    64 89 93 99 99
Texel 1.05 64-bit        0  0  0  0  0  0  0  2 18 21 35    81 87 97 99
Deep HIARCS 14 WCSC      0  0  0  0  0  0  0  0  3  4 10 18    60 86 95
Crafty-25.1.Syzy-0611d   0  0  0  0  0  0  0  0  1  2  6 12 39    79 92
Deep Shredder 12 x64     0  0  0  0  0  0  0  0  0  0  0  2 13 20    73
Arasan 19.0              0  0  0  0  0  0  0  0  0  0  0  0  4  7 26  

and the details so far:

Code: Select all

ResultSet-EloRating>details
   1 Komodo 10 64-bit        3505  90.0 ( 76.5 :  13.5)
                                    6.0 (  4.0 :   2.0) Stockfish 062816-xy     3502
                                    6.0 (  2.5 :   3.5) Stockfish 062816-x      3489
                                    6.0 (  4.5 :   1.5) Hawkeye 1.0 x64         3331
                                    6.0 (  4.0 :   2.0) Stockfish DD 64 SSE4.2  3297
                                    6.0 (  5.0 :   1.0) Gull 3 x64 (syzygy)     3294
                                    6.0 (  5.5 :   0.5) Critter 1.6a 64-bit     3190
                                    6.0 (  5.5 :   0.5) Mikhail 0.04 OSx 64     3157
                                    6.0 (  5.0 :   1.0) Senpai 1.0              3115
                                    6.0 (  6.0 :   0.0) Hakkapeliitta TCEC v2   3110
                                    6.0 (  5.0 :   1.0) Kōhai 1.0              3094
                                    6.0 (  6.0 :   0.0) Texel 1.05 64-bit       3078
                                    6.0 (  6.0 :   0.0) Deep HIARCS 14 WCSC     3042
                                    6.0 (  5.5 :   0.5) Crafty-25.1.Syzy-0611d  3031
                                    6.0 (  6.0 :   0.0) Deep Shredder 12 x64    2995
                                    6.0 (  6.0 :   0.0) Arasan 19.0             2969
   2 Stockfish 062816-xy     3502  89.0 ( 75.0 :  14.0)
                                    6.0 (  2.0 :   4.0) Komodo 10 64-bit        3505
                                    6.0 (  3.5 :   2.5) Stockfish 062816-x      3489
                                    6.0 (  4.5 :   1.5) Hawkeye 1.0 x64         3331
                                    6.0 (  4.5 :   1.5) Stockfish DD 64 SSE4.2  3297
                                    6.0 (  4.5 :   1.5) Gull 3 x64 (syzygy)     3294
                                    6.0 (  6.0 :   0.0) Critter 1.6a 64-bit     3190
                                    6.0 (  5.5 :   0.5) Mikhail 0.04 OSx 64     3157
                                    5.0 (  4.5 :   0.5) Senpai 1.0              3115
                                    6.0 (  4.5 :   1.5) Hakkapeliitta TCEC v2   3110
                                    6.0 (  6.0 :   0.0) Kōhai 1.0              3094
                                    6.0 (  6.0 :   0.0) Texel 1.05 64-bit       3078
                                    6.0 (  6.0 :   0.0) Deep HIARCS 14 WCSC     3042
                                    6.0 (  6.0 :   0.0) Crafty-25.1.Syzy-0611d  3031
                                    6.0 (  6.0 :   0.0) Deep Shredder 12 x64    2995
                                    6.0 (  5.5 :   0.5) Arasan 19.0             2969
   3 Stockfish 062816-x      3489  91.0 ( 76.0 :  15.0)
                                    6.0 (  3.5 :   2.5) Komodo 10 64-bit        3505
                                    6.0 (  2.5 :   3.5) Stockfish 062816-xy     3502
                                    6.0 (  4.5 :   1.5) Hawkeye 1.0 x64         3331
                                    6.0 (  4.5 :   1.5) Stockfish DD 64 SSE4.2  3297
                                    6.0 (  5.0 :   1.0) Gull 3 x64 (syzygy)     3294
                                    6.0 (  5.5 :   0.5) Critter 1.6a 64-bit     3190
                                    6.0 (  5.5 :   0.5) Mikhail 0.04 OSx 64     3157
                                    6.0 (  5.5 :   0.5) Senpai 1.0              3115
                                    6.0 (  6.0 :   0.0) Hakkapeliitta TCEC v2   3110
                                    6.0 (  5.5 :   0.5) Kōhai 1.0              3094
                                    6.0 (  4.5 :   1.5) Texel 1.05 64-bit       3078
                                    6.0 (  5.0 :   1.0) Deep HIARCS 14 WCSC     3042
                                    6.0 (  6.0 :   0.0) Crafty-25.1.Syzy-0611d  3031
                                    6.0 (  6.0 :   0.0) Deep Shredder 12 x64    2995
                                    7.0 (  6.5 :   0.5) Arasan 19.0             2969
   4 Hawkeye 1.0 x64         3331  90.0 ( 60.5 :  29.5)
                                    6.0 (  1.5 :   4.5) Komodo 10 64-bit        3505
                                    6.0 (  1.5 :   4.5) Stockfish 062816-xy     3502
                                    6.0 (  1.5 :   4.5) Stockfish 062816-x      3489
                                    6.0 (  3.0 :   3.0) Stockfish DD 64 SSE4.2  3297
                                    6.0 (  3.5 :   2.5) Gull 3 x64 (syzygy)     3294
                                    6.0 (  4.0 :   2.0) Critter 1.6a 64-bit     3190
                                    6.0 (  4.5 :   1.5) Mikhail 0.04 OSx 64     3157
                                    6.0 (  5.5 :   0.5) Senpai 1.0              3115
                                    6.0 (  5.0 :   1.0) Hakkapeliitta TCEC v2   3110
                                    6.0 (  5.5 :   0.5) Kōhai 1.0              3094
                                    6.0 (  5.0 :   1.0) Texel 1.05 64-bit       3078
                                    6.0 (  5.0 :   1.0) Deep HIARCS 14 WCSC     3042
                                    6.0 (  5.0 :   1.0) Crafty-25.1.Syzy-0611d  3031
                                    6.0 (  5.5 :   0.5) Deep Shredder 12 x64    2995
                                    6.0 (  4.5 :   1.5) Arasan 19.0             2969
   5 Stockfish DD 64 SSE4.2  3297  90.0 ( 56.5 :  33.5)
                                    6.0 (  2.0 :   4.0) Komodo 10 64-bit        3505
                                    6.0 (  1.5 :   4.5) Stockfish 062816-xy     3502
                                    6.0 (  1.5 :   4.5) Stockfish 062816-x      3489
                                    6.0 (  3.0 :   3.0) Hawkeye 1.0 x64         3331
                                    6.0 (  4.5 :   1.5) Gull 3 x64 (syzygy)     3294
                                    6.0 (  4.5 :   1.5) Critter 1.6a 64-bit     3190
                                    6.0 (  2.5 :   3.5) Mikhail 0.04 OSx 64     3157
                                    6.0 (  4.0 :   2.0) Senpai 1.0              3115
                                    6.0 (  4.5 :   1.5) Hakkapeliitta TCEC v2   3110
                                    6.0 (  4.5 :   1.5) Kōhai 1.0              3094
                                    6.0 (  5.5 :   0.5) Texel 1.05 64-bit       3078
                                    6.0 (  4.5 :   1.5) Deep HIARCS 14 WCSC     3042
                                    6.0 (  4.5 :   1.5) Crafty-25.1.Syzy-0611d  3031
                                    6.0 (  4.5 :   1.5) Deep Shredder 12 x64    2995
                                    6.0 (  5.0 :   1.0) Arasan 19.0             2969
   6 Gull 3 x64 (syzygy)     3294  90.0 ( 56.0 :  34.0)
                                    6.0 (  1.0 :   5.0) Komodo 10 64-bit        3505
                                    6.0 (  1.5 :   4.5) Stockfish 062816-xy     3502
                                    6.0 (  1.0 :   5.0) Stockfish 062816-x      3489
                                    6.0 (  2.5 :   3.5) Hawkeye 1.0 x64         3331
                                    6.0 (  1.5 :   4.5) Stockfish DD 64 SSE4.2  3297
                                    6.0 (  4.5 :   1.5) Critter 1.6a 64-bit     3190
                                    6.0 (  5.0 :   1.0) Mikhail 0.04 OSx 64     3157
                                    6.0 (  3.5 :   2.5) Senpai 1.0              3115
                                    6.0 (  5.0 :   1.0) Hakkapeliitta TCEC v2   3110
                                    6.0 (  4.0 :   2.0) Kōhai 1.0              3094
                                    6.0 (  5.5 :   0.5) Texel 1.05 64-bit       3078
                                    6.0 (  4.5 :   1.5) Deep HIARCS 14 WCSC     3042
                                    6.0 (  5.5 :   0.5) Crafty-25.1.Syzy-0611d  3031
                                    6.0 (  5.0 :   1.0) Deep Shredder 12 x64    2995
                                    6.0 (  6.0 :   0.0) Arasan 19.0             2969
   7 Critter 1.6a 64-bit     3190  90.0 ( 44.5 :  45.5)
                                    6.0 (  0.5 :   5.5) Komodo 10 64-bit        3505
                                    6.0 (  0.0 :   6.0) Stockfish 062816-xy     3502
                                    6.0 (  0.5 :   5.5) Stockfish 062816-x      3489
                                    6.0 (  2.0 :   4.0) Hawkeye 1.0 x64         3331
                                    6.0 (  1.5 :   4.5) Stockfish DD 64 SSE4.2  3297
                                    6.0 (  1.5 :   4.5) Gull 3 x64 (syzygy)     3294
                                    6.0 (  3.5 :   2.5) Mikhail 0.04 OSx 64     3157
                                    6.0 (  3.5 :   2.5) Senpai 1.0              3115
                                    6.0 (  3.5 :   2.5) Hakkapeliitta TCEC v2   3110
                                    6.0 (  4.5 :   1.5) Kōhai 1.0              3094
                                    6.0 (  5.5 :   0.5) Texel 1.05 64-bit       3078
                                    6.0 (  4.5 :   1.5) Deep HIARCS 14 WCSC     3042
                                    6.0 (  5.0 :   1.0) Crafty-25.1.Syzy-0611d  3031
                                    6.0 (  4.0 :   2.0) Deep Shredder 12 x64    2995
                                    6.0 (  4.5 :   1.5) Arasan 19.0             2969
   8 Mikhail 0.04 OSx 64     3157  90.0 ( 41.0 :  49.0)
                                    6.0 (  0.5 :   5.5) Komodo 10 64-bit        3505
                                    6.0 (  0.5 :   5.5) Stockfish 062816-xy     3502
                                    6.0 (  0.5 :   5.5) Stockfish 062816-x      3489
                                    6.0 (  1.5 :   4.5) Hawkeye 1.0 x64         3331
                                    6.0 (  3.5 :   2.5) Stockfish DD 64 SSE4.2  3297
                                    6.0 (  1.0 :   5.0) Gull 3 x64 (syzygy)     3294
                                    6.0 (  2.5 :   3.5) Critter 1.6a 64-bit     3190
                                    6.0 (  3.0 :   3.0) Senpai 1.0              3115
                                    6.0 (  3.0 :   3.0) Hakkapeliitta TCEC v2   3110
                                    6.0 (  4.0 :   2.0) Kōhai 1.0              3094
                                    6.0 (  3.5 :   2.5) Texel 1.05 64-bit       3078
                                    6.0 (  5.0 :   1.0) Deep HIARCS 14 WCSC     3042
                                    6.0 (  4.0 :   2.0) Crafty-25.1.Syzy-0611d  3031
                                    6.0 (  3.5 :   2.5) Deep Shredder 12 x64    2995
                                    6.0 (  5.0 :   1.0) Arasan 19.0             2969
   9 Senpai 1.0              3115  89.0 ( 35.0 :  54.0)
                                    6.0 (  1.0 :   5.0) Komodo 10 64-bit        3505
                                    5.0 (  0.5 :   4.5) Stockfish 062816-xy     3502
                                    6.0 (  0.5 :   5.5) Stockfish 062816-x      3489
                                    6.0 (  0.5 :   5.5) Hawkeye 1.0 x64         3331
                                    6.0 (  2.0 :   4.0) Stockfish DD 64 SSE4.2  3297
                                    6.0 (  2.5 :   3.5) Gull 3 x64 (syzygy)     3294
                                    6.0 (  2.5 :   3.5) Critter 1.6a 64-bit     3190
                                    6.0 (  3.0 :   3.0) Mikhail 0.04 OSx 64     3157
                                    6.0 (  3.0 :   3.0) Hakkapeliitta TCEC v2   3110
                                    6.0 (  2.5 :   3.5) Kōhai 1.0              3094
                                    6.0 (  3.5 :   2.5) Texel 1.05 64-bit       3078
                                    6.0 (  3.0 :   3.0) Deep HIARCS 14 WCSC     3042
                                    6.0 (  3.5 :   2.5) Crafty-25.1.Syzy-0611d  3031
                                    6.0 (  3.0 :   3.0) Deep Shredder 12 x64    2995
                                    6.0 (  4.0 :   2.0) Arasan 19.0             2969
  10 Hakkapeliitta TCEC v2   3110  90.0 ( 36.0 :  54.0)
                                    6.0 (  0.0 :   6.0) Komodo 10 64-bit        3505
                                    6.0 (  1.5 :   4.5) Stockfish 062816-xy     3502
                                    6.0 (  0.0 :   6.0) Stockfish 062816-x      3489
                                    6.0 (  1.0 :   5.0) Hawkeye 1.0 x64         3331
                                    6.0 (  1.5 :   4.5) Stockfish DD 64 SSE4.2  3297
                                    6.0 (  1.0 :   5.0) Gull 3 x64 (syzygy)     3294
                                    6.0 (  2.5 :   3.5) Critter 1.6a 64-bit     3190
                                    6.0 (  3.0 :   3.0) Mikhail 0.04 OSx 64     3157
                                    6.0 (  3.0 :   3.0) Senpai 1.0              3115
                                    6.0 (  4.5 :   1.5) Kōhai 1.0              3094
                                    6.0 (  2.5 :   3.5) Texel 1.05 64-bit       3078
                                    6.0 (  3.5 :   2.5) Deep HIARCS 14 WCSC     3042
                                    6.0 (  4.0 :   2.0) Crafty-25.1.Syzy-0611d  3031
                                    6.0 (  5.0 :   1.0) Deep Shredder 12 x64    2995
                                    6.0 (  3.0 :   3.0) Arasan 19.0             2969
  11 Kōhai 1.0              3094  90.0 ( 32.0 :  58.0)
                                    6.0 (  1.0 :   5.0) Komodo 10 64-bit        3505
                                    6.0 (  0.0 :   6.0) Stockfish 062816-xy     3502
                                    6.0 (  0.5 :   5.5) Stockfish 062816-x      3489
                                    6.0 (  0.5 :   5.5) Hawkeye 1.0 x64         3331
                                    6.0 (  1.5 :   4.5) Stockfish DD 64 SSE4.2  3297
                                    6.0 (  2.0 :   4.0) Gull 3 x64 (syzygy)     3294
                                    6.0 (  1.5 :   4.5) Critter 1.6a 64-bit     3190
                                    6.0 (  2.0 :   4.0) Mikhail 0.04 OSx 64     3157
                                    6.0 (  3.5 :   2.5) Senpai 1.0              3115
                                    6.0 (  1.5 :   4.5) Hakkapeliitta TCEC v2   3110
                                    6.0 (  1.0 :   5.0) Texel 1.05 64-bit       3078
                                    6.0 (  4.0 :   2.0) Deep HIARCS 14 WCSC     3042
                                    6.0 (  2.5 :   3.5) Crafty-25.1.Syzy-0611d  3031
                                    6.0 (  5.0 :   1.0) Deep Shredder 12 x64    2995
                                    6.0 (  5.5 :   0.5) Arasan 19.0             2969
  12 Texel 1.05 64-bit       3078  90.0 ( 32.0 :  58.0)
                                    6.0 (  0.0 :   6.0) Komodo 10 64-bit        3505
                                    6.0 (  0.0 :   6.0) Stockfish 062816-xy     3502
                                    6.0 (  1.5 :   4.5) Stockfish 062816-x      3489
                                    6.0 (  1.0 :   5.0) Hawkeye 1.0 x64         3331
                                    6.0 (  0.5 :   5.5) Stockfish DD 64 SSE4.2  3297
                                    6.0 (  0.5 :   5.5) Gull 3 x64 (syzygy)     3294
                                    6.0 (  0.5 :   5.5) Critter 1.6a 64-bit     3190
                                    6.0 (  2.5 :   3.5) Mikhail 0.04 OSx 64     3157
                                    6.0 (  2.5 :   3.5) Senpai 1.0              3115
                                    6.0 (  3.5 :   2.5) Hakkapeliitta TCEC v2   3110
                                    6.0 (  5.0 :   1.0) Kōhai 1.0              3094
                                    6.0 (  3.5 :   2.5) Deep HIARCS 14 WCSC     3042
                                    6.0 (  5.0 :   1.0) Crafty-25.1.Syzy-0611d  3031
                                    6.0 (  2.0 :   4.0) Deep Shredder 12 x64    2995
                                    6.0 (  4.0 :   2.0) Arasan 19.0             2969
  13 Deep HIARCS 14 WCSC     3042  90.0 ( 28.0 :  62.0)
                                    6.0 (  0.0 :   6.0) Komodo 10 64-bit        3505
                                    6.0 (  0.0 :   6.0) Stockfish 062816-xy     3502
                                    6.0 (  1.0 :   5.0) Stockfish 062816-x      3489
                                    6.0 (  1.0 :   5.0) Hawkeye 1.0 x64         3331
                                    6.0 (  1.5 :   4.5) Stockfish DD 64 SSE4.2  3297
                                    6.0 (  1.5 :   4.5) Gull 3 x64 (syzygy)     3294
                                    6.0 (  1.5 :   4.5) Critter 1.6a 64-bit     3190
                                    6.0 (  1.0 :   5.0) Mikhail 0.04 OSx 64     3157
                                    6.0 (  3.0 :   3.0) Senpai 1.0              3115
                                    6.0 (  2.5 :   3.5) Hakkapeliitta TCEC v2   3110
                                    6.0 (  2.0 :   4.0) Kōhai 1.0              3094
                                    6.0 (  2.5 :   3.5) Texel 1.05 64-bit       3078
                                    6.0 (  2.5 :   3.5) Crafty-25.1.Syzy-0611d  3031
                                    6.0 (  3.5 :   2.5) Deep Shredder 12 x64    2995
                                    6.0 (  4.5 :   1.5) Arasan 19.0             2969
  14 Crafty-25.1.Syzy-0611d  3031  90.0 ( 26.5 :  63.5)
                                    6.0 (  0.5 :   5.5) Komodo 10 64-bit        3505
                                    6.0 (  0.0 :   6.0) Stockfish 062816-xy     3502
                                    6.0 (  0.0 :   6.0) Stockfish 062816-x      3489
                                    6.0 (  1.0 :   5.0) Hawkeye 1.0 x64         3331
                                    6.0 (  1.5 :   4.5) Stockfish DD 64 SSE4.2  3297
                                    6.0 (  0.5 :   5.5) Gull 3 x64 (syzygy)     3294
                                    6.0 (  1.0 :   5.0) Critter 1.6a 64-bit     3190
                                    6.0 (  2.0 :   4.0) Mikhail 0.04 OSx 64     3157
                                    6.0 (  2.5 :   3.5) Senpai 1.0              3115
                                    6.0 (  2.0 :   4.0) Hakkapeliitta TCEC v2   3110
                                    6.0 (  3.5 :   2.5) Kōhai 1.0              3094
                                    6.0 (  1.0 :   5.0) Texel 1.05 64-bit       3078
                                    6.0 (  3.5 :   2.5) Deep HIARCS 14 WCSC     3042
                                    6.0 (  4.0 :   2.0) Deep Shredder 12 x64    2995
                                    6.0 (  3.5 :   2.5) Arasan 19.0             2969
  15 Deep Shredder 12 x64    2995  90.0 ( 23.5 :  66.5)
                                    6.0 (  0.0 :   6.0) Komodo 10 64-bit        3505
                                    6.0 (  0.0 :   6.0) Stockfish 062816-xy     3502
                                    6.0 (  0.0 :   6.0) Stockfish 062816-x      3489
                                    6.0 (  0.5 :   5.5) Hawkeye 1.0 x64         3331
                                    6.0 (  1.5 :   4.5) Stockfish DD 64 SSE4.2  3297
                                    6.0 (  1.0 :   5.0) Gull 3 x64 (syzygy)     3294
                                    6.0 (  2.0 :   4.0) Critter 1.6a 64-bit     3190
                                    6.0 (  2.5 :   3.5) Mikhail 0.04 OSx 64     3157
                                    6.0 (  3.0 :   3.0) Senpai 1.0              3115
                                    6.0 (  1.0 :   5.0) Hakkapeliitta TCEC v2   3110
                                    6.0 (  1.0 :   5.0) Kōhai 1.0              3094
                                    6.0 (  4.0 :   2.0) Texel 1.05 64-bit       3078
                                    6.0 (  2.5 :   3.5) Deep HIARCS 14 WCSC     3042
                                    6.0 (  2.0 :   4.0) Crafty-25.1.Syzy-0611d  3031
                                    6.0 (  2.5 :   3.5) Arasan 19.0             2969
  16 Arasan 19.0             2969  91.0 ( 21.0 :  70.0)
                                    6.0 (  0.0 :   6.0) Komodo 10 64-bit        3505
                                    6.0 (  0.5 :   5.5) Stockfish 062816-xy     3502
                                    7.0 (  0.5 :   6.5) Stockfish 062816-x      3489
                                    6.0 (  1.5 :   4.5) Hawkeye 1.0 x64         3331
                                    6.0 (  1.0 :   5.0) Stockfish DD 64 SSE4.2  3297
                                    6.0 (  0.0 :   6.0) Gull 3 x64 (syzygy)     3294
                                    6.0 (  1.5 :   4.5) Critter 1.6a 64-bit     3190
                                    6.0 (  1.0 :   5.0) Mikhail 0.04 OSx 64     3157
                                    6.0 (  2.0 :   4.0) Senpai 1.0              3115
                                    6.0 (  3.0 :   3.0) Hakkapeliitta TCEC v2   3110
                                    6.0 (  0.5 :   5.5) Kōhai 1.0              3094
                                    6.0 (  2.0 :   4.0) Texel 1.05 64-bit       3078
                                    6.0 (  1.5 :   4.5) Deep HIARCS 14 WCSC     3042
                                    6.0 (  2.5 :   3.5) Crafty-25.1.Syzy-0611d  3031
                                    6.0 (  3.5 :   2.5) Deep Shredder 12 x64    2995
lkaufman
Posts: 6297
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA
Full name: Larry Kaufman

Re: 48,000 game RR with 16 engines 2m + 2s

Post by lkaufman »

MikeB wrote:Games will start shortly, auto updates will be made after every round (240 games ) roughly every 2 hours and 45 mins - will let it run run along s I can - so first update will be around 6AM EST.

Results: https://www.dropbox.com/s/cx9vntj8n8myw ... a.txt?dl=1

PGN Files:https://www.dropbox.com/s/tmwepshusdy48h7/all.pgn?dl=1

Engines:
arasan.ini
crafty-25.1.Syzy-0611d - (current dev crafty with syzygy support)
shredder
hiarcs-14
senpai
kohai-1.0 (senpai derivative)
hakka322
texel
Mikhail-0.04 (Firenzina derivative)
Stockfish-DDi
critter
Gull-3.0
Hawkeye-1.0 (Gull derivative)
komodo-10
Stockfish-062816-x (current SF)
Stockfish-062816-xy (modified SF)


Using Bob Hyatt's automated script (modified)...

Code: Select all

set eng1 = arasan.ini
set eng2 = crafty-25.1.Syzy-0611d
set eng3 = shredder.ini
set eng4 = hiarcs-14.ini
set eng5 = senpai.ini
set eng6 = kohai-1.0.ini
set eng7 = hakka322.ini
set eng8 = texel.ini
set eng9 = Mikhail-0.04.ini
set eng10 = Stockfish-DD.ini
set eng11 = critter.ini
set eng12 = Gull-3.0.ini
set eng13 = Hawkeye-1.0.ini
set eng14 = komodo-10.ini
set eng15 = Stockfish-062816-x.ini
set eng16 = Stockfish-062816-xy.ini

set opponents1=( $eng1 $eng2 $eng3 $eng4 $eng5 $eng6 $eng7 $eng8 $eng9 $eng10 $eng11 $eng12 $eng13  $eng14 $eng15 $eng16 )
set opponents2=( $eng16 $eng15 $eng14 $eng13 $eng12 $eng11 $eng10 $eng9 $eng8 $eng7 $eng6 $eng5 $eng4 $eng3 $eng2 $eng1 )

set players1=( 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 )
set players2=( 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 )

set number_of_players1 = 16
set number_of_players2 = 16

set file = new2500.epd
#set file = tcec50.epd

@ positions  = 200
@ rounds  = $positions * 2
set time_odds = 1
set games_per_position = 1
set positions_per_match = 1

echo "using $positions positions from EPD file"
@ gamesPlanned =  ( $number_of_players1  )  * ( $number_of_players2 - 1 )   * $positions * $games_per_position
echo "gamesPlanned"
@ counterStart = ( $gamesPlanned  / $positions ) + 10
@ counterIncrement = ( $counterStart - 10 )

cd run
set logid1 = 101
set logid2 = 50101
rm -f commands
@ pairings = 0

@ position = 0 #you can change this to start @ different position in the file

set counter = 0
set gamesStarted = 0
loop:
#@ position = $position + 1 # start at the  1st position ,
@ position = `jot -r 1 1 2500` #for random selection
	foreach player1 ($players1)
		foreach player2 ($players2)
		if ($opponents1[$player1] == $opponents2[$player2] ) goto skip  #  do not want engines to play themselves
			echo "./match "match$player1.$position" $file $position $games_per_position $positions_per_match $time $inc  $opponents1[$player1] localhost $logid1 $opponents2[$player2] localhost $logid2 $time_odds" >> commands
			@ logid1 = $logid1 + $games_per_position * $positions_per_match
			@ logid2 = $logid2 + $games_per_position * $positions_per_match
			@ pairings = ( $pairings + $games_per_position )
			echo "$pairings"
			@ completed = ( $pairings - 10 )

		@ gamesStarted = $pairings - $counter
		if ($gamesStarted < $counterStart ) goto skip_1
			echo "echo " "" >> commands
			echo "echo "$completed of $gamesPlanned games completed..."" >> commands
			echo "echo "time control: $time+$inc"" >> commands
			echo "./geteloRR1" >> commands
			echo "echo " ">>~/Dropbox/Shared\ Folder/elota.txt" >> commands
			echo "echo " ">>~/Dropbox/Shared\ Folder/elotb.txt" >> commands
			echo "echo "$completed of $gamesPlanned games completed...">>~/Dropbox/Shared\ Folder/elota.txt" >> commands
			echo "echo "$completed of $gamesPlanned games completed...">>~/Dropbox/Shared\ Folder/elotb.txt" >> commands
			echo "echo "time control: $time+$inc">>~/Dropbox/Shared\ Folder/elota.txt" >> commands
			echo "echo "time control: $time+$inc">>~/Dropbox/Shared\ Folder/elotb.txt" >> commands
			echo "./geteloRR2>>~/Dropbox/Shared\ Folder/elota.txt" >> commands
			echo "./geteloRR3>>~/Dropbox/Shared\ Folder/elotb.txt" >> commands
			echo "cp pgn /Users/michaelbyrne/Dropbox/Shared\ Folder/all.pgn" >> commands
			@ counter = $counter + $counterIncrement
		skip_1:
		skip:
		end
	end  # round is completed with both colors scripted

if ($pairings < $gamesPlanned) goto loop
echo "echo " "" >> commands
echo "echo $completed games completed" >> commands
echo "time control: $time+$inc" >.commands
echo "./geteloRR1" >> commands

echo "EOF" >> commands
touch run.1
foreach file (run.*)
  rm $file
end
@ record = 0
foreach run ({0,1,2,3,4,5,6,7,8,0}{0,1,2,3,4,5,6,7,8,9}{0,1,2,3,4,5,6,7,8,9}{0,1,2,3,4,5,6,7,8,9}{0,1,2,3,4,5,6,7,8,9})
  if ($run == "00000") continue
  @ record = $record + 1
  set command = `cat commands | head -$record | tail -1`
  if ($command[1] == "EOF") break
  touch run.$run
echo "$command &" >> run.$run
echo "wait" >> run.$run
  chmod +x run.$run
  if ($command[1] == "EOF") break
end
if ($run == "99999") then
  echo "ERROR, more than 99999 commands, fix script and retry"
  exit
endif
echo "$record run scripts were prepared."
cd ..

I suppose this is all single core and all using default settings?

Is SF xy modified by you or by someone else? Is it public? Is it substantially different from normal SF or just one or two mods?
Komodo rules!
User avatar
MikeB
Posts: 4889
Joined: Thu Mar 09, 2006 6:34 am
Location: Pen Argyl, Pennsylvania

Re: 48,000 game RR with 16 engines 2m + 2s

Post by MikeB »

lkaufman wrote: ...

I suppose this is all single core and all using default settings?

Is SF xy modified by you or by someone else? Is it public? Is it substantially different from normal SF or just one or two mods?
Single Core , ponder off , using 11 of the 12 cores on a Mac Pro ( keeping once core reserved for OS and for me), 256M hash. Otherwise default settings.

xy was modified and complied by me - not released, three modifications :

1: piece values
2. LMR formula
3, An additional extension that only extends under certain conditions

A slightly different version of the LMR formula failed in fishtest - but I believe the formula . as it is now, still has some merit at longer tc ( longer than 10 sec + 0.1 sec , the normal fishtest tc). So only three mods , but when touching lmr and piece values, ELO can move quite bit - either way - not always favorable :shock:
lkaufman
Posts: 6297
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA
Full name: Larry Kaufman

Re: 48,000 game RR with 16 engines 2m + 2s

Post by lkaufman »

MikeB wrote:
lkaufman wrote: ...

I suppose this is all single core and all using default settings?

Is SF xy modified by you or by someone else? Is it public? Is it substantially different from normal SF or just one or two mods?
Single Core , ponder off , using 11 of the 12 cores on a Mac Pro ( keeping once core reserved for OS and for me), 256M hash. Otherwise default settings.

xy was modified and complied by me - not released, three modifications :

1: piece values
2. LMR formula
3, An additional extension that only extends under certain conditions

A slightly different version of the LMR formula failed in fishtest - but I believe the formula . as it is now, still has some merit at longer tc ( longer than 10 sec + 0.1 sec , the normal fishtest tc). So only three mods , but when touching lmr and piece values, ELO can move quite bit - either way - not always favorable :shock:
Are the piece values you are trying the result of an automated tuning run or someone's subjective opinions?
Komodo rules!
User avatar
MikeB
Posts: 4889
Joined: Thu Mar 09, 2006 6:34 am
Location: Pen Argyl, Pennsylvania

Re: 48,000 game RR with 16 engines 2m + 2s

Post by MikeB »

lkaufman wrote:
MikeB wrote:
lkaufman wrote: ...

I suppose this is all single core and all using default settings?

Is SF xy modified by you or by someone else? Is it public? Is it substantially different from normal SF or just one or two mods?
Single Core , ponder off , using 11 of the 12 cores on a Mac Pro ( keeping once core reserved for OS and for me), 256M hash. Otherwise default settings.

xy was modified and complied by me - not released, three modifications :

1: piece values
2. LMR formula
3, An additional extension that only extends under certain conditions

A slightly different version of the LMR formula failed in fishtest - but I believe the formula . as it is now, still has some merit at longer tc ( longer than 10 sec + 0.1 sec , the normal fishtest tc). So only three mods , but when touching lmr and piece values, ELO can move quite bit - either way - not always favorable :shock:
Are the piece values you are trying the result of an automated tuning run or someone's subjective opinions?
well after 36 years involved with chess computers and programs - I have an opinion on what is best - I do agree with the current theory that as game progresses, piece values change - my basic assumption is that at the opening and middle game - a rook and a pawn should be less than b&n and q should be more than two rooks...and more than 3 minors. SF is a little different since it uses what I would call non standard piece values - but those rules above should still apply . I saw no reason to even try playing around with endgame piece values - to me they look spot on.
lkaufman
Posts: 6297
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA
Full name: Larry Kaufman

Re: 48,000 game RR with 16 engines 2m + 2s

Post by lkaufman »

MikeB wrote:
lkaufman wrote:
MikeB wrote:
lkaufman wrote: ...

I suppose this is all single core and all using default settings?

Is SF xy modified by you or by someone else? Is it public? Is it substantially different from normal SF or just one or two mods?
Single Core , ponder off , using 11 of the 12 cores on a Mac Pro ( keeping once core reserved for OS and for me), 256M hash. Otherwise default settings.

xy was modified and complied by me - not released, three modifications :

1: piece values
2. LMR formula
3, An additional extension that only extends under certain conditions

A slightly different version of the LMR formula failed in fishtest - but I believe the formula . as it is now, still has some merit at longer tc ( longer than 10 sec + 0.1 sec , the normal fishtest tc). So only three mods , but when touching lmr and piece values, ELO can move quite bit - either way - not always favorable :shock:
Are the piece values you are trying the result of an automated tuning run or someone's subjective opinions?
well after 36 years involved with chess computers and programs - I have an opinion on what is best - I do agree with the current theory that as game progresses, piece values change - my basic assumption is that at the opening and middle game - a rook and a pawn should be less than b&n and q should be more than two rooks...and more than 3 minors. SF is a little different since it uses what I would call non standard piece values - but those rules above should still apply . I saw no reason to even try playing around with endgame piece values - to me they look spot on.
I suppose you meant to say q should be less than 3 minors. But with Stockfish the cross/quadratic terms are supposed to make these things right; I guess it's your opinion that they fail to do so. But just getting the relative piece values right won't be any good without retuning the cross terms, but I'm sure you addressed this issue somehow.
Komodo rules!