tuning info
Moderators: hgm, Rebel, chrisw
-
- Posts: 855
- Joined: Sun May 23, 2010 1:32 pm
tuning info
Hi everyone,
I just finished a good session of tuning of my engine using a technique very similar to texel method, minimising the error of evaluation against game results.
I'd like now to try to tune some other parameters using an spsa like approach.
I have studied the perl spsa tuner and looked at fishtest code.
The spsa Perl tuner doesn't work very well with vajolet and I don't like to debug the uci interaction between them. We already have a very good tournament software, cutechess and I was asking myself if already exist some tool or fork of cutechess that allow to tune an engine.
Are you aware of some project or tuning script already using cutechess? Otherwise my idea is to add a tuner to cutechess.
Thank you all
I just finished a good session of tuning of my engine using a technique very similar to texel method, minimising the error of evaluation against game results.
I'd like now to try to tune some other parameters using an spsa like approach.
I have studied the perl spsa tuner and looked at fishtest code.
The spsa Perl tuner doesn't work very well with vajolet and I don't like to debug the uci interaction between them. We already have a very good tournament software, cutechess and I was asking myself if already exist some tool or fork of cutechess that allow to tune an engine.
Are you aware of some project or tuning script already using cutechess? Otherwise my idea is to add a tuner to cutechess.
Thank you all
-
- Posts: 859
- Joined: Mon Aug 10, 2009 10:05 pm
- Location: Italy
- Full name: Stefano Gemma
Re: tuning info
Does Vajolet have a parameter on the commadn line to load parameters from a file? I could test genetic optimization using Sabrina tournaments mode. I don't know SPSA but it could be added as a learning algorithm.elcabesa wrote:Hi everyone,
I just finished a good session of tuning of my engine using a technique very similar to texel method, minimising the error of evaluation against game results.
I'd like now to try to tune some other parameters using an spsa like approach.
I have studied the perl spsa tuner and looked at fishtest code.
The spsa Perl tuner doesn't work very well with vajolet and I don't like to debug the uci interaction between them. We already have a very good tournament software, cutechess and I was asking myself if already exist some tool or fork of cutechess that allow to tune an engine.
Are you aware of some project or tuning script already using cutechess? Otherwise my idea is to add a tuner to cutechess.
Thank you all
Author of Drago, Raffaela, Freccia, Satana, Sabrina.
http://www.linformatica.com
http://www.linformatica.com
-
- Posts: 4185
- Joined: Tue Mar 14, 2006 11:34 am
- Location: Ethiopia
Re: tuning info
Clop + cutechess-cli is good for tuning search parameters.
It could be slow if you have too many parameters but often times there are just a handful of search parameters.
For example, I only have one futility_pruning_margin to tune in search.
Daniel
It could be slow if you have too many parameters but often times there are just a handful of search parameters.
For example, I only have one futility_pruning_margin to tune in search.
Daniel
-
- Posts: 855
- Joined: Sun May 23, 2010 1:32 pm
Re: tuning info
I don't like clop, but probably I can his approach.
write a python script for spsa that use cutechess to play game.. I'll give it a try
write a python script for spsa that use cutechess to play game.. I'll give it a try
-
- Posts: 4366
- Joined: Fri Mar 10, 2006 5:23 am
- Location: http://www.arasanchess.org
Re: tuning info
There is a script that comes with CLOP that you can probably modify.
You might also take a look also at:
https://github.com/jdart1/arasan-chess/ ... sitions.py
Which shows how to invoke cutechess-cli and parse the results.
--Jon
You might also take a look also at:
https://github.com/jdart1/arasan-chess/ ... sitions.py
Which shows how to invoke cutechess-cli and parse the results.
--Jon
-
- Posts: 4185
- Joined: Tue Mar 14, 2006 11:34 am
- Location: Ethiopia
Re: tuning info
I think clop may support different optimization algorithms.
I see an C2SPSA.cpp in there though i am not sure what it is used for.
P.S: Does anybody know why latest cutechess-cli does not display results on stdout ?
It plays the game and saves it to a pgn but there is no result output on the screen.
I tried cutechess-cli 1.0 and 0.85, on ubuntu 17.10 -- Qt version 5.9.1
Daniel
I see an C2SPSA.cpp in there though i am not sure what it is used for.
P.S: Does anybody know why latest cutechess-cli does not display results on stdout ?
It plays the game and saves it to a pgn but there is no result output on the screen.
I tried cutechess-cli 1.0 and 0.85, on ubuntu 17.10 -- Qt version 5.9.1
Daniel
-
- Posts: 687
- Joined: Mon Apr 19, 2010 7:07 pm
- Location: Sweden
- Full name: Peter Osterlund
Re: tuning info
I had this problem in fedora 24. I solved it by adding the following to ~/.config/QtProject/qtlogging.ini:Daniel Shawul wrote:P.S: Does anybody know why latest cutechess-cli does not display results on stdout ?
It plays the game and saves it to a pgn but there is no result output on the screen.
I tried cutechess-cli 1.0 and 0.85, on ubuntu 17.10 -- Qt version 5.9.1
Code: Select all
[Rules]
*.debug=true
-
- Posts: 4185
- Joined: Tue Mar 14, 2006 11:34 am
- Location: Ethiopia
Re: tuning info
It works like a charm!
Thanks a lot Peter!
Daniel
Thanks a lot Peter!
Daniel
-
- Posts: 4833
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: tuning info
Couple of notes when using spsa fromelcabesa wrote:The spsa Perl tuner doesn't work very well with vajolet and I don't like to debug the uci interaction between them.
https://github.com/zamar/spsa/blob/master/spsa.pl
1. Be aware of the game adjudication on stalemate.
When your engine received a stalemate position what bestmove it prints?
Stockfish would print
Code: Select all
bestmove (none)
then the code in line 446,
Code: Select all
$flag_stalemate = 1 if ($array[1] eq '(none)');
Code: Select all
$flag_stalemate = 1 if ($array[1] eq '0000');
Code: Select all
# Check for mate in one
if ($#array >= 9 && $array[0] eq 'info' && $array[1] eq 'depth' &&
$array[7] eq 'score' && $array[8] eq 'mate' && $array[9] eq '1')
{
$flag_mate = 1;
$winner = $engine_to_move;
}
Code: Select all
info depth 1 score mate 1 nodes 20 nps 1818 tbhits 0 time 11 pv e2e4
Code: Select all
if ($#array >= 5 && $array[0] eq 'info' && $array[1] eq 'depth' &&
$array[3] eq 'score' && $array[4] eq 'mate' && $array[5] eq '1')
{
...
}
Code: Select all
# Record score
if ($#array >= 7 && $array[0] eq 'info' && $array[1] eq 'depth' &&
$array[7] eq 'score')
{
$score = $array[9] if ($array[8] eq 'cp');
$score = +100000 if ($array[8] eq 'mate' && $array[9] > 0);
$score = -100000 if ($array[8] eq 'mate' && $array[9] < 0);
}
Code: Select all
$#array >= 7
Code: Select all
$#array >= 9
-
- Posts: 855
- Joined: Sun May 23, 2010 1:32 pm
Re: tuning info
I don't like reinventing the well, we already have the beatiful cutechess-cli to play games.
I think I'll probably port spsa script to python and use cutechess to play games
I think I'll probably port spsa script to python and use cutechess to play games