Help and Tips for Engine matches or Tournaments?

[leavenfish]

I am TOTALLY NEW to this so any help would be appreciated!

I am interested in running some matches or tournaments. I am having a hard time finding some good guidelines.

I am using Fritz interface and would like to use that, not any other...but could perhaps be persuaded.

Currently I am 'trying' to test Komodo 5.1r2 vs The latest Stockfish developmental version. Since SF is so much faster I was only using a set ply (19) to see how each engine performed using the old Silver Test Suite.

I can't get the interface to ONLY do ply (wants to default to blitz 4'min/2sec) so I set it to 120min/40 but with 19ply. This DOES seem to limit each engine to 19 play...but so many of the games get to about 60 moves deep and then the game stops and goes to the next starting position. I have move limit set to 150. There is plenty of time left.

So, I am trying to see how each engine 'performs' over a game while only looking at a set 'ply'. Anyone with any idea of what I might be doing wrong or any tips to try and do what I am aiming at?

6 games in - 1 win each, 3 draws...but 1 game that stopped about move 60.

I also want to run a tournament where each engine only looks a certain ply and decides on a move....but I can't find a way to do that in Fritz 14.

Any tips - or links you might have to help; specific to what I am hoping to do or for engine matches/tournaments in general?

THANKS!

[leavenfish]

Essentially what I am trying to do is this:

http://www.fastgm.de/fixed-depth.html

But for different engines...and doing so in a tournament format would be nice as well.

But again, any general or specific recommendations appreciated.

Why? Just wondering about the relationship between 'knowledge' an engine has at specific ply...and how that correlates to progress in engines over their development. Some would say Komodo was better tuned...made better evaluations in earlier version, but might not 'play' (or be as fast) as more recent versions. Me, I don't know...

[leavenfish]

I just purchased Komodo and the one year subscription, I have two others versions; and would like to do some testing. I may have been too specific in this post so, let me be more general...

1. Could anyone point me to a website (or even a good thread or two here you might have bookmarked) for someone new to engine testing? Be it engine vs engine or simply comparing choices at different points in my opening repertoire.

2. What (non-expensive) GUI's are among the best to test on? I have Chessbase and Fritz (Komodo 8) GUI...just downloaded Lucas Chess...I had an old version of Arena and must have deleted it by accident and an old version of Aquarium which...is just too complicated for my liking. I really like Chess Position Trainer...but you can't really do any 'testing' in it.

3. Any tips that might not be so commonly known as to testing?

At the moment I am stepping thru an old repertoire book: A Strategic Opening Repertoire by Donaldson and Hansen in a Chessbase file and have already found some interesting difference in Stockfish 8 and (any) of the Komodo engines: Horvath - Csiszar 15th Zalakaros Open 1996 where on one sideline however long I let Komodo analyse, it does not consider a Bg5 Stockfish sets its sights on early and will not deviate from...doesn't even consider it among the top 3 choices. I kind of like the Komodo choice...

ANY response will be appreciated.

[Graham Banks]

I use ChessGUI for testing, as it can accommodate winboard, uci and java engines.

Opening books are easy to create within the GUI from pgn files.
You can set up to play so that each side plays an opening line from both sides.

You have the option of tablebases adjudication.
It will allow the use of eggb, Nalimov, Gaviota and Syzygy tablebases.

You can set your own GUI adjudication settings.

You can run round robin, swiss, knockout and team tournaments/matches.

I run engine v engine tournaments, gauntlets and matches as you're probably aware.
If you're interested in being part of a testing group, pm me.

Graham.

[Adam Hair]

The primary GUIs for engine vs engine matches (in alphabetical order):

Aquarium
Arena
ChessBase
ChessGUI
Cutechess GUI
Hiarcs Chess Explorer
Scid vs PC
Shredder GUI
Winboard

While I think it is possible to run fixed ply matches with any of these GUIs using a program called Inbetween, the three I recommend using are Arena, Cutechess GUI, and Winboard. Cutechess and Winboard are more versatile; Arena is easier for a beginner to use.

For analysis, I imagine that ChessBase is as good as any GUI, but Arena is also useful.

I created a short guide for creating tournaments under Winboard and Arena a few years ago that may been helpful to you.

[leavenfish]

Thanks Graham!

I did download ChessGUI last night and set up a match on my i7-6700 16GB machine between

Komodo 10.3 and asmFish (2016-12-16)
Game/5 with a 10 Sec Gain
Silver Suite
Ponder OFF
NO Tablebase
Stock settings for each I presume...

After 18 games:
Komodo 10.3 = 11.5
asmFish (2016-12-16) = 6.5


I may have it totally set up wrong and should probably abort this as ALL SIX of the Komodo wins were on time - in positions that were at most equal.

[leavenfish]

Thanks Adam - the Guide is exactly what I need!

I will probably download Arena again (and 1or 2 others) ....had it a few years ago and never tried to learn it and must have accidentally deleted it.

[Modern Times]

One problem with ChessGUI is that it is no longer supported or developed. If it is working fine for you, then that obviously isn't an issue, but it might be. Having said that, Arena for example is not bug-free by any means.