Test programs

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Rebel, chrisw

Krzysztof Grzelak
Posts: 1554
Joined: Tue Jul 15, 2014 12:47 pm

Re: Test programs

Post by Krzysztof Grzelak »

Commercial chess engines tournament

The commercial chess engines tournament was played on the GUI Shredder Classic 5.

Shredder Classic 5 GUI setting during tournament:

- All One Book = Perfect 2021
- All One Hash Size = 1024 MB
- Tournament type = Round Robin
- Cycles = 5
- Threads = 8
- Tablebases = Nalimov
- Time = 5 Minutes/Game + 3 Seconds/Move

Engine setup during the tournament:


Deep Hiarcs 14 WCSC x64

Threads = 8
CPU Usage = 100
Busy Threads = On
Swindle Opponent = On
Retain Hash = On
Position Learning = On
Optimistic Search = On
Combinations = On
Hiarcs Draw Value = 0
Search Selectivity = 7
Playing Style = Active
Hyper-Modern Play = On


Deep Onno 1.2.70 x64

Threads = 8
Verbose Level = 3
Random Intensity = 9
Random Seed = 5489
TB Min Depth = 128
TB Draw = null scores


Deep Rybka 4.1 x64

Max CPUs = 2048
Random = 0
CPU Usage = 8
NalimovProbeFrequency = 128
Main Process Prority = Normal
Child Process Prority = Normal
MultiPV_cp = 20000
TC Buffer = 3
TC Normal Move Time = 80
TC Max Move Time = 100
Rook Endgame Scaling = 100


Deep Sjeng 3.0 x64

CPUs = 8
Drawscore (centipawns) = 0


Zappa Mexico II x64

Threads = auto
Enable Null Move = On
Enable History Pruning = On
Enable Singular Extensions = On
Enable Mate Extensions = On
Enable Futility Pruning = On
Aggressive Futility Pruning = On
TB Min Depth = 4
Null Move R4 Min Depth = 100
Singular Margin = 75
Singular Margin PV = 50
Eval Passed Pawn Scoring = 100
Eval Pawn Scoring = 100
Eval Minor Scoring = 100
Eval Major Scoring = 100
Eval Kingsafety Scoring = 100
Internet Lag = 0
Contempt = 0


The tournament ended with the Deep Rybka program winning. Second place went to Zappa Mexico program and third place to Deep Hiarcs program.

Tournament table


Image

Tournament games https://files.fm/u/bxp9b5qxs
Krzysztof Grzelak
Posts: 1554
Joined: Tue Jul 15, 2014 12:47 pm

Re: Test programs

Post by Krzysztof Grzelak »

Match Ethereal 14.00 x64 - KomodoDragon 3.2 x64

Match result: 0-6-94 (100 games)

The match was played on the following equipment:

processor - AMD Ryzen Threadripper 2990WX 3.0GHz
hard drive - Samsung 970 EVO Plus NVMe M.2 SSD 2 TB
memory ram - Kingston HyperX Fury 128 GB DDR4 2400MHz CL15
graphics card - ASUS Tuf GeForce RTX 3080Ti Gaming LHR 12GB
monitor - Nec MultiSync EA245WMi
keyboard - A4Tech KV-300H Slim
mouse - Razer DeathAdder Essential
operating system - Microsoft Windows 11 Professional 64 bit

During the match I was using the Top15-2022 debut book, time - 5 minutes + 3 seconds. I have used 6-men Syzygy table bases. The match was played on the GUI Cutochess 1.2 by Guenther Simon.

Setting GUI Cutochess 1.2 by Guenther Simon

Tournament type:
Round Robin= On

Rounds:
Rounds= 100
Save unfinished games= On
Swap sides= On

Games:
Time Control:
Time= 5,00
Increment= 3,00sec

Suite:
PGN/EPD file= D:/Book/Top15-2022
Depth= 16 plies
Opening book= Random

Draw adjudication:
Move number= 35
Move count= 5
Score= 15 cp

Adjudication:
Move count= 5
Score= 1000 cp
Two-Sided= On

Game Length:
Limit= Off

Tablebase adjudication= On

Setting the engines during the game.

Ethereal 14.00 x64

Protocol=uci
Hash=1024
Threads=32
MultiPV=1
MoveOverhead=300
SyzygyPath=D:\Syzygy
SyzygyProbeDepth=2
EvalFile=e1400.standard.nnue

KomodoDragon 3.2 x64

Protocol=uci
Hash=1024
Threads = 32
Table Memory = 128
Use LMR = On
Null Move Pruning = On
Overhead ms = 50
Time Usage = 0
Use Syzygy = On
SyzygyPath = D:\Syzygy
SyzygyProbleDepth = 2
SyzygyProbleLimit = 6
Syzygy50MoveRule = On
Contempt = 16
Personality = Default
Skill Time = 1


Parties of the match Ethereal 14.00 - KomodoDragon 3.2 https://files.fm/u/vkxv6t73k


Image
Jouni
Posts: 3324
Joined: Wed Mar 08, 2006 8:15 pm

Re: Test programs

Post by Jouni »

Really Hash=1024 Threads = 32 for Time= 5,00 Increment= 3,00sec !?
Jouni
Krzysztof Grzelak
Posts: 1554
Joined: Tue Jul 15, 2014 12:47 pm

Re: Test programs

Post by Krzysztof Grzelak »

Jouni wrote: Mon Feb 13, 2023 4:18 pm Really Hash=1024 Threads = 32 for Time= 5,00 Increment= 3,00sec !?
Yes, enough.
Krzysztof Grzelak
Posts: 1554
Joined: Tue Jul 15, 2014 12:47 pm

Re: Test programs

Post by Krzysztof Grzelak »

Match RubiChess 20230410 x64 - Clover 4.0 x64

Match result: 14-0-86 (100 games)

The match was played on the following equipment:

processor - AMD Ryzen Threadripper 2990WX 3.0 GHz
hard drive - Samsung 970 EVO Plus NVMe M.2 SSD 2 TB
memory ram - Kingston HyperX Fury 128 GB DDR4 2400MHz CL15
graphics card - ASUS Tuf GeForce RTX 3080Ti Gaming LHR 12GB
monitor - Nec MultiSync EA245WMi
keyboard - A4Tech KV-300H Slim
mouse - Razer DeathAdder Essential
operating system - Microsoft Windows 11 Professional 64 bit

During the match I was using the Perfect 2023 debut book, time - 5 minutes + 3 seconds. I have used 6-men Syzygy table bases. The match was played on the GUI Cutochess 1.2 by Guenther Simon.

Setting GUI Cutochess 1.2 by Guenther Simon

Tournament type:
Round Robin= On

Rounds:
Rounds= 100
Save unfinished games= On
Swap sides= On

Games:
Time Control:
Time= 5,00
Increment= 3,00sec

Opening book:
Polyglot file= D:/Book/Perfect2023.bin
Deep= 8 full moves
Access mode= Disk

Draw adjudication:
Move number= 35
Move count= 5
Score= 15 cp

Adjudication:
Move count= 5
Score= 1000 cp
Two-Sided= On

Game Length:
Limit= Off

Tablebase adjudication= On

Setting the engines during the game.

RubiChess 20230410 x64

Protocol= uci
Hash= 16384
Threads= 32
Allow Large Pages= On
Contempt= 0
ContemptRatio= 4
Move_Overhead= 100
MultiPV= 1
NNUENetpath= nn-fdccaaabd3-20230314.nnue
Syzygy50MoveRule = On
SyzygyPath= D:\Syzygy
SyzygyProbleLimit= 6
Use_NNUE= On

Clover 4.0 x64

Protocol= uci
Hash= 16384
Threads = 32
SyzygyPath= D:\Syzygy
Other options unchanged

Parties of the match RubiChess 20230410 - Clover 4.0 https://files.fm/u/cb988dw84
Krzysztof Grzelak
Posts: 1554
Joined: Tue Jul 15, 2014 12:47 pm

Re: Test programs

Post by Krzysztof Grzelak »

Match Lco 0.30-dag-a9b25c2b-BT2-3650000 x64 - Lco 0.30-dag-a9b25c2b-BT2-4510000 x64

Match result: 0-0-100 (100 games)

The match was played on the following equipment:

processor - AMD Ryzen Threadripper 2990WX 3.0 GHz
hard drive - Samsung 970 EVO Plus NVMe M.2 SSD 2 TB
memory ram - Kingston HyperX Fury 128 GB DDR4 2400MHz CL15
graphics card - ASUS Tuf GeForce RTX 3080Ti Gaming LHR 12GB
monitor - Nec MultiSync EA245WMi
keyboard - A4Tech KV-300H Slim
mouse - Razer DeathAdder Essential
operating system - Microsoft Windows 11 Professional 64 bit

During the match I was using the Perfect 2023 debut book, time - 5 minutes + 3 seconds. I have used 6-men Syzygy table bases. The match was played on the GUI Cutochess 1.2 by Guenther Simon.

Setting GUI Cutochess 1.2 by Guenther Simon

Tournament type:
Round Robin= On

Rounds:
Rounds= 100
Save unfinished games= On
Swap sides= On

Games:
Time Control:
Time= 5,00
Increment= 3,00sec

Opening book:
Polyglot file= D:/Book/Perfect2023.bin
Deep= 8 full moves
Access mode= Disk

Draw adjudication:
Move number= 35
Move count= 5
Score= 15 cp

Adjudication:
Move count= 5
Score= 1000 cp
Two-Sided= On

Game Length:
Limit= Off

Tablebase adjudication= On

Setting the engines during the game.

Lco 0.30-dag-a9b25c2b-BT2-3650000 x64

Protocol= uci
Network IDs= BT2-768x15smolgen-12h-do-01-3650000.pb.gz
Backend= cuda-fp16
BackendOptions= backendcuda-fp16,multi_streamtrue,threads2
Threads= 2
MiniBatchSize= 216
NNCacheSize= 2000
FpuValue= 0.5830500680121913
CPuct= 2.241594716715595
CPuctBase= 45669
CPuctFactor= 3.973
MaxPrefetch= 37
Contempt= 0
ScoreType= WDL_mu
PolicyTemperature= 1.4061131420002002
MovesLeftSlope= 0.0027
MovesLeftThreshold= 0.8
SmartPruningFactor=2
SmartPruningMinimumBatches=300
StrictTiming= On
VerboseMoveStats= On
Syzygypath= D:\Syzygy
WDLContemptAttenuation= 0.6
WDLEvalObjectivity= 0
MoveOverheadMs= 1000
TimeManager= legacy

Lco 0.30-dag-a9b25c2b-BT2-4510000 x64

Protocol= uci
Network IDs= BT2-768x15smolgen-12h-do-01-swa-4510000.pb.gz
Backend= cuda-fp16
BackendOptions= backendcuda-fp16,multi_streamtrue,threads2
Threads= 2
MiniBatchSize= 216
NNCacheSize= 2000
FpuValue= 0.5830500680121913
CPuct= 2.241594716715595
CPuctBase= 45669
CPuctFactor= 3.973
MaxPrefetch= 37
Contempt= 0
ScoreType= WDL_mu
PolicyTemperature= 1.4061131420002002
MovesLeftSlope= 0.0027
MovesLeftThreshold= 0.8
SmartPruningFactor= 2
SmartPruningMinimumBatches= 300
StrictTiming= On
VerboseMoveStats= On
Syzygypath= D:\Syzygy
WDLContemptAttenuation= 0.6
WDLEvalObjectivity= 0
MoveOverheadMs= 1000
TimeManager= legacy

Parties of the match Lco 0.30-dag-a9b25c2b-BT2-3650000 - Lco 0.30-dag-a9b25c2b-BT2-4510000 https://files.fm/u/2xrqgasc8
Krzysztof Grzelak
Posts: 1554
Joined: Tue Jul 15, 2014 12:47 pm

Re: Test programs

Post by Krzysztof Grzelak »

Match Chess System Tal 2.00 v21-E1019 x64 - Viridithas 10.0.0 x64

Match result: 10-0-90 (100 games)

The match was played on the following equipment:

processor - AMD Ryzen Threadripper 2990WX 3.0 GHz
hard drive - Samsung 970 EVO Plus NVMe M.2 SSD 2 TB
memory ram - Kingston HyperX Fury 128 GB DDR4 2400MHz CL15
graphics card - ASUS Tuf GeForce RTX 3080Ti Gaming LHR 12GB
monitor - Nec MultiSync EA245WMi
keyboard - A4Tech KV-300H Slim
mouse - Razer DeathAdder Essential
operating system - Microsoft Windows 11 Professional 64 bit

During the match I was using the Balsa 270423 debut book, time - 5 minutes + 3 seconds. I have used 6-men Syzygy table bases. The match was played on the GUI Cutochess 1.2 by Guenther Simon.

Setting GUI Cutochess 1.2 by Guenther Simon

Tournament type:
Round Robin= On

Rounds:
Rounds= 100
Save unfinished games= On
Swap sides= On

Games:
Time Control:
Time= 5,00
Increment= 3,00sec

Opening suite:
PGN/EPD file= D:/Book/Balsa_270423.pgn
Depth= 16 plies
Opening order= Random

Draw adjudication:
Move number= 35
Move count= 5
Score= 15 cp

Adjudication:
Move count= 5
Score= 1000 cp
Two-Sided= On

Game Length:
Limit= Off

Tablebase adjudication= On

Setting the engines during the game.

Chess System Tal 2.00 v21-E1019 x64

Protocol= uci
Hash= 8192
Threads= 32
Evalcorrect= 202
Rep2_Draw_Trolling= On
TC_Move_Overhead= 7
TC_Aggression= 100

Viridithas 10.0.0 x64

Protocol= uci
Hash= 8192
Threads= 32
UseNNUE= On
SyzygyPath= D:\Syzygy
SyzygyProbleLimit= 6
SyzygyProbleDepth= 2
Contempt= 0

Parties of the match Chess System Tal 2.00 v21-E1019 x64 - Viridithas 10.0.0 x64 https://files.fm/u/x765zeaqw
Krzysztof Grzelak
Posts: 1554
Joined: Tue Jul 15, 2014 12:47 pm

Re: Test programs

Post by Krzysztof Grzelak »

Tournament Engines.

The following chess engines took part in the tournament:

Berserk 12 x64
Booot 7.2 x64
Clover 6.0 x64
Igel 3.5.0 x64
Minic 3.39 x64
rofChade 3.1 x64
RubiChess 20230918 x64
Seer 2.7 x64
Uralochka 3.40a x64
Viridithas 11.0.0 x64


The engine tournament was played on the following equipment:

processor - AMD Ryzen Threadripper 2990WX 3.0 GHz
hard drive - Samsung 970 EVO Plus NVMe M.2 SSD 2 TB
memory ram - Kingston HyperX Fury 128 GB DDR4 2400MHz CL15
graphics card - Two graphics cards - ASUS Tuf GeForce RTX 3080Ti Gaming LHR 12GB
monitor - Nec MultiSync EA245WMi
keyboard - A4Tech KV-300H Slim
mouse - DeathAdder Essential
operating system - Microsoft Windows 10 Professional 64 bit

During the tournament I was using the Top15-2022 debut book, time - 3 minutes + 2 seconds. I have used 6-men Syzygy table bases. The match was played on the GUI Cutochess 1.2 by Guenther Simon.

Setting GUI Cutochess 1.2 by Guenther Simon

Tournament type:
Round Robin= On

Rounds:
Rounds= 4
Save unfinished games= On
Swap sides= On

Games:
Time Control:
Time= 3,00
Increment= 2,00sec

Opening suite:
PGN/EPD file= D:/Book/Top15-2022.pgn
Depth= 8 plies
Opening order= Random

Draw adjudication:
Move number= 35
Move count= 5
Score= 15 cp

Adjudication:
Move count= 5
Score= 1000 cp
Two-Sided= On

Game Length:
Limit= Off

Tablebase adjudication= On

Setting engines during the tournament.

All engines were set as follows:

Threads= 32
Hash= 4096
Syzygy= 6 men
Pawn Hash= 1024


The remaining options in the programs remain unchanged.

Tournament Engines ended with victory of the program Berserk 12 by Jay Honnold, second and third place was programs
RubiChess 20230918 by Andreas Matthies and Seer 2.7 by Connor McMonigle.Thanks to the other tournament participants.

Tournament table and other information from the tournament.

Image

Player: Berserk 12 x64
"Draw by 3-fold repetition": 1
"Draw by adjudication": 18
"Draw by adjudication: SyzygyTB": 4
"Loss: White wins by adjudication": 1
"Win: Black wins by adjudication": 4
"Win: White wins by adjudication": 7
"Win: White wins by adjudication: SyzygyTB": 1

Player: RubiChess 20230918 x64
"Draw by 3-fold repetition": 2
"Draw by adjudication": 21
"Draw by adjudication: SyzygyTB": 9
"Win: Black wins by adjudication": 2
"Win: White wins by adjudication": 2

Player: Seer 2.7 x64
"Draw by 3-fold repetition": 7
"Draw by adjudication": 19
"Draw by adjudication: SyzygyTB": 3
"Loss: Black wins by adjudication": 1
"Loss: White wins by adjudication": 2
"Win: White wins by adjudication": 4

Player: Viridithas 11.0.0 x64
"Draw by 3-fold repetition": 9
"Draw by adjudication": 22
"Draw by adjudication: SyzygyTB": 3
"Loss: White wins by adjudication": 1
"Win: Black wins by adjudication: SyzygyTB": 1

Player: Clover 6.0 x64
"Draw by 3-fold repetition": 6
"Draw by adjudication": 26
"Loss: Black wins by adjudication": 1
"Loss: White wins by adjudication": 1
"Win: White wins by adjudication": 2

Player: rofChade 3.1 x64
"Draw by 3-fold repetition": 3
"Draw by adjudication": 23
"Draw by adjudication: SyzygyTB": 5
"Loss: White wins by adjudication": 3
"Win: White wins by adjudication": 2

Player: Minic 3.39 x64
"Draw by 3-fold repetition": 8
"Draw by adjudication": 18
"Draw by adjudication: SyzygyTB": 4
"Loss: Black wins by adjudication": 1
"Loss: Black wins by adjudication: SyzygyTB": 1
"Loss: White wins by adjudication": 2
"Win: White wins by adjudication": 2

Player: Uralochka 3.40a x64
"Draw by 3-fold repetition": 9
"Draw by adjudication": 20
"Draw by adjudication: SyzygyTB": 4
"Loss: White wins by adjudication": 2
"Loss: White wins by adjudication: SyzygyTB": 1

Player: Igel 3.5.0 x64
"Draw by 3-fold repetition": 10
"Draw by adjudication": 20
"Draw by adjudication: SyzygyTB": 3
"Loss: Black wins by adjudication": 1
"Loss: White wins by adjudication": 2

Player: Booot 7.2 x64
"Draw by 3-fold repetition": 7
"Draw by adjudication": 19
"Draw by adjudication: SyzygyTB": 3
"Loss: Black wins by adjudication": 2
"Loss: White wins by adjudication": 5


Tournament games " Tournament Engines " - https://files.fm/u/7m3hpqc7a2
Krzysztof Grzelak
Posts: 1554
Joined: Tue Jul 15, 2014 12:47 pm

Re: Test programs

Post by Krzysztof Grzelak »

Match LCZero 0.31-dag-e4726ea7-BT3-2860000 x64 - KomodoDragon 3.3 x64

Match result 5 - 0 - 95 (100 games)


The match was played on the following equipment:

processor - AMD Ryzen Threadripper 2990WX 3.0 GHz
hard drive - Samsung 970 EVO Plus NVMe M.2 SSD 2 TB
memory ram - Kingston HyperX Fury 128 GB DDR4 2400MHz CL15
graphics card - Two graphics cards - ASUS Tuf GeForce RTX 3080Ti Gaming LHR 12GB
monitor - Nec MultiSync EA245WMi
keyboard - A4Tech KV-300H Slim
mouse - DeathAdder Essential
operating system - Microsoft Windows 10 Professional 64 bit


During the match I was using the Balsa_270423 book - 5 minutes + 3 seconds. I have used 6-men Syzygy table bases. The match was played on the GUI Cutochess 1.2 by Guenther Simon.

Setting GUI Cutochess 1.2 by Guenther Simon

Tournament type:
Round Robin=On

Rounds:
Rounds=100
Save unfinished games=On
Swap sides=On

Games:
Time Control:
Time=5,00
Increment=3,00sec

Opening suite:
PGN/EPD file=D:/Book/Balsa_270423.pgn
Depth=16 plies
Opening book=Random

Draw adjudication:
Move number=35
Move count=5
Score=15 cp

Resign adjudication:
Move count=5
Score=1000 cp
Two-Sided=On

Game Length:
Limit=Off

Tablebase adjudication=On

Setting the engines during the game.

LCZero 0.31-dag-e4726ea7-BT3-2860000 x64

Protocol=uci
Network IDs=BT3-2860000
Backend=demux
BackendOptions=backend=cuda-fp16,(gpu=0,policy_head=vanilla,value_head=winner),(gpu=1,policy_head=vanilla,value_head=winner),
multi_stream=true,threads=2
Threads=3
MinibatchSize=216
VerboseMoveStats=true
FpuValue=0.98416
CPuct=2.897
CPuctBase=45669
CPuctFactor=3.973
PolicyTemperature=1.4
MoveRuleBucketing=true
MovesLeftSlope=0.0027
MovesLeftThreshold=0.8
Contempt=Stockfish=0
ContemptMaxValue=600
Ponder=false
RamLimitMb=16384
ScoreType=WDL_mu
SmartPruningFactor=2
SmartPruningMinimumBatches=300
TaskWorkers=3
SyzygyPath=D:\Syzygy
MoveOverheadMs=1000
StrictTiming=true
TimeManager=legacy
WDLCalibrationElo=3600
WDLContemptAttenuation=1
WDLDrawRateReference=0.61
WDLEvalObjectivity=0

KomodoDragon 3.3 x64

Protocol=uci
Hash=16384
Threads=32
Contempt=0
Overheadms=50
SyzygyPath=D:/Syzygy
TimeUsage=0
TableMemory=256
SyzygyProbeLimit=6
SyzygyProbeDepth=2
SmartSyzygy=false
OwnBook=false
Ponder=false


Parties of the match LCZero 0.31-dag-e4726ea7-BT3-2860000 - KomodoDragon 3.3 https://files.fm/u/mmevk3yhst
Krzysztof Grzelak
Posts: 1554
Joined: Tue Jul 15, 2014 12:47 pm

Re: Test programs

Post by Krzysztof Grzelak »

Match Ethereal 14.25 x64 - Obsidian 11.0 x64

Match result 1 - 1 - 98 (100 games)

The match was played on the following equipment:

processor - AMD Ryzen Threadripper 2990WX 3.0 GHz
hard drive - Samsung 970 EVO Plus NVMe M.2 SSD 2 TB
memory ram - Kingston HyperX Fury 128 GB DDR4 2400MHz CL15
graphics card - Two graphics cards - ASUS Tuf GeForce RTX 3080Ti Gaming LHR 12GB
monitor - Nec MultiSync EA245WMi
keyboard - A4Tech KV-300H Slim
mouse - DeathAdder Essential
operating system - Microsoft Windows 10 Professional 64 bit


During the match I was using the 2600+8moves book - 5 minutes + 3 seconds. I have used 6-men Syzygy table bases. The match was played on the GUI Cutochess 1.2 by Guenther Simon.


Setting GUI Cutochess 1.2 by Guenther Simon

Tournament type:
Round Robin - On

Rounds:
Rounds - 100
Save unfinished games - On
Swap sides - On

Games:
Time Control:
Time - 5,00
Increment - 3,00sec

Opening suite:
PGN/EPD file - D:/Book/2600+8moves.pgn
Depth - 16 plies
Opening book - Random

Draw adjudication:
Move number - 60
Move count - 5
Score - 15 cp

Resign adjudication:
Move count - 5
Score - 1000 cp
Two-Sided - On

Game Length:
Limit - Off

Tablebase adjudication: - On

Setting the engines during the game.


Ethereal 14.25 x64

Protocol - uci
Hash - 4096
Threads - 16
MoveOverhead - 300
SyzygyPath - D:\Syzygy
SyzygyProbeDepth - 2
EvalFile - ethy.v1425.std.nn
Normalize - On


Obsidian 11.0 x64

Protocol - uci
Hash - 4096
Threads - 16
Move Overhead - 10
SyzygyPath - D:\Syzygy


Parties of the match Ethereal 14.25 - Obsidian 11.0 https://files.fm/u/222cekdwv9