First just want to say what a fantastic job you have done with Winboard and thank you.
Now let me explain what I am doing. I am running UCI engines under Winboard with the Polyglot adapter. Say I am testing Stockfish 1.8A and Stockfish 1.8B and I want to play them against a variety of engines to measure which one is stronger. Is there a way I can set up a batch file with something like this:
LucenaTheLucid wrote:First just want to say what a fantastic job you have done with Winboard and thank you.
Now let me explain what I am doing. I am running UCI engines under Winboard with the Polyglot adapter. Say I am testing Stockfish 1.8A and Stockfish 1.8B and I want to play them against a variety of engines to measure which one is stronger. Is there a way I can set up a batch file with something like this:
Would games played at a time control of 10 seconds be meaningful if played on the ordinary average joe's computer to measure strength differences between certain versions?
Also would x/second per-move also tell a good story of which engine version is stronger?
LucenaTheLucid wrote:First just want to say what a fantastic job you have done with Winboard and thank you.
Now let me explain what I am doing. I am running UCI engines under Winboard with the Polyglot adapter. Say I am testing Stockfish 1.8A and Stockfish 1.8B and I want to play them against a variety of engines to measure which one is stronger. Is there a way I can set up a batch file with something like this:
But now instead of setting /tc1 have it play games with seconds per move? For instance a gauntlet where games are .5 or .1 seconds per move?
Hello Luis Smith,
It may be lèse majesté to speak when you specifically asked for H.G. but if you are testing UCI engines only you may want to have a look at this software by Nathan Thom:
When you are using those ultra-fast time-controls with winboard there might be an issue of engines hanging, the adapter tidying up communications among itself and the gui and the engine, the gui waiting for termination and start etc. Those issues will wreck a test run in my opinion.
Winboard does not seem to accept fractional settings directly but there are ways around it....
Wow Roger, I was not away some such tool existed! I wonder how accurate are the results in a game played with only 1 second as compared to a game which is played in 10 minutes? Or even 30?
LucenaTheLucid wrote:Dr. Hyatt a couple of side question?
Would games played at a time control of 10 seconds be meaningful if played on the ordinary average joe's computer to measure strength differences between certain versions?
Also would x/second per-move also tell a good story of which engine version is stronger?
A couple of issues.
(1) in general, I've found that fast games are perfectly acceptable, so long as you are careful to not go so fast that a program starts to lose on time. After makihng several changes, you can always do an occasional verification using longer time controls, just to be safe...
(2) a fixed time per move is OK, although you cut out a part of the engine's character, since you do not allow it to use more or less time for some moves, which most programs will do.
(3) I do not like just playing games between A and A'. My testing has shown that this often gives results that are either inflated, or even wrong. It's better to play against several programs, preferably that are about the same strength as the engine you are testing, or a bit stronger. Don't test against a bunch of patsies, nor should you play against opponents you can only win one out of every hundred games.
A couple of issues.
(1) in general, I've found that fast games are perfectly acceptable, so long as you are careful to not go so fast that a program starts to lose on time. After makihng several changes, you can always do an occasional verification using longer time controls, just to be safe...
Yes this was the issue I was running into while trying to find some good Stockfish settings. I was using 10 seconds per game with .1 increment. Which is exactly why I want to use fixed time per move.
(2) a fixed time per move is OK, although you cut out a part of the engine's character, since you do not allow it to use more or less time for some moves, which most programs will do.
(3) I do not like just playing games between A and A'. My testing has shown that this often gives results that are either inflated, or even wrong. It's better to play against several programs, preferably that are about the same strength as the engine you are testing, or a bit stronger. Don't test against a bunch of patsies, nor should you play against opponents you can only win one out of every hundred games.
Agreed. I was planning on running the 2 individual Stockfish settings against these engines:
I was rechecking the games I ran with /tc 1 /inc 1 -firstTimeOdds 6 -secondTimeOdds 6 which puts the game at 10sec/0.1sec inc and there were NO lost games on time.
I don't know how fast his computer is but if he is goning to run very fast time controlls it might be advisab le to make sure that pondering is turned off as this might severely limit the time factor in a negative way.
Just a thought
Bill