Yet another parameter tuner using optuna framework

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Ferdy
Posts: 4285
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Yet another parameter tuner using optuna framework

Post by Ferdy » Mon Sep 14, 2020 3:34 am

Created a parameter tuner which might be useful to your chess engine.

https://github.com/fsmosca/Optuna-Game-Parameter-Tuner

I got a promising optimization result on my engine Deuterium at https://github.com/fsmosca/Optuna-Game- ... mization-3
I thought with only couple of piece values it can at least equalize my default but it actually defeated my default.

The parameter to be optimized is hard-coded, you have to modify the tuner.py. The tournament manager is cutechess-cli which can be found in tourney_manager/cutechess folder. See the Setup section, you need to install those dependencies.

It is based on optuna framework at https://github.com/optuna/optuna

Joerg Oster
Posts: 783
Joined: Fri Mar 10, 2006 3:29 pm
Location: Germany

Re: Yet another parameter tuner using optuna framework

Post by Joerg Oster » Wed Sep 16, 2020 10:34 am

Thank you!
Looks very interesting.
Jörg Oster

chrisw
Posts: 3641
Joined: Tue Apr 03, 2012 2:28 pm

Re: Yet another parameter tuner using optuna framework

Post by chrisw » Wed Sep 16, 2020 11:53 am

Ferdy wrote:
Mon Sep 14, 2020 3:34 am
Created a parameter tuner which might be useful to your chess engine.

https://github.com/fsmosca/Optuna-Game-Parameter-Tuner

I got a promising optimization result on my engine Deuterium at https://github.com/fsmosca/Optuna-Game- ... mization-3
I thought with only couple of piece values it can at least equalize my default but it actually defeated my default.

The parameter to be optimized is hard-coded, you have to modify the tuner.py. The tournament manager is cutechess-cli which can be found in tourney_manager/cutechess folder. See the Setup section, you need to install those dependencies.

It is based on optuna framework at https://github.com/optuna/optuna

This looks extremely brilliant. I took a look at the Optuna site docs also. Trying to work out 'how it works', tell me if I understand it right, probably I don't of course.

1. You have a base engine, e1 and same engine, e2, except you can send parameter change instructions to e2

2. Uses cutechess tournament mode, with output the result of a quick match (25 rounds?)

3. I'm not sure if parameter changes to a 'quick match' change all five parameters at a time, or just one?

4. Optuna tries parameter changes (within your supplied range?), and using smart algorithm, decides where to concentrate it's fire, so to speak.

5. After N iterations, it gives you new values for the selected parameters.

One thing I don't understand are the supplied output graphs, and what they are showing (apart from Optuna deciding that PAWNVAlUE is the 'most important' tunable item.

One other thing. The Optuna presentation says it can handle the functions (cutechess output) in parallel, are you using that feature, or is cutechess script already running the tournament in parallel?

Ferdy
Posts: 4285
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Yet another parameter tuner using optuna framework

Post by Ferdy » Wed Sep 16, 2020 1:17 pm

chrisw wrote:
Wed Sep 16, 2020 11:53 am

This looks extremely brilliant. I took a look at the Optuna site docs also. Trying to work out 'how it works', tell me if I understand it right, probably I don't of course.

1. You have a base engine, e1 and same engine, e2, except you can send parameter change instructions to e2
Yes they are the same engine, e1 which I call test engine and e2 as base engine. The e1 will always take the optimizer suggested param values. The e2 will always take the best param values. After a match if e1 wins that is more than 50% score, the param values used by e1 will become the best param values. This best param values then will be used by e2 in the next match. The e1 will take the new suggested values from the optimizer. The score of the match must be reported to the optimizer before taking its suggestions.
2. Uses cutechess tournament mode, with output the result of a quick match (25 rounds?)
Yes, number of game is settable, the more games the better.
3. I'm not sure if parameter changes to a 'quick match' change all five parameters at a time, or just one?
Sorry I don't understand the question.
4. Optuna tries parameter changes (within your supplied range?), and using smart algorithm, decides where to concentrate it's fire, so to speak.
Yes.
5. After N iterations, it gives you new values for the selected parameters.
Yes. They call it trials. A trial can be a 24-game match. After a trial we send the score of the match to the optimizer. After that it is safe to ask for the new suggested param values. Around here https://github.com/fsmosca/Optuna-Game- ... ner.py#L65
One thing I don't understand are the supplied output graphs, and what they are showing (apart from Optuna deciding that PAWNVAlUE is the 'most important' tunable item.
Slice plot: The y-axis is the score of the engine match, see the objective value label. The x-axis is the value of the parameter that has been tried. The color of the dot is the number of trials, trial number is high if it is dark, see the vertical legend at right. In here we can see if the tuner is making more suggestions that improves the objective value per parameter plot. We can also see here at what range of values has more dots or samples suggested. We can always see the y-axis if the dot performed well or not.
Image

The contour: There is a vertical bar legend at right. The lighter it is, the better. It is the match result of the game match or objective value. Let's take a look at the plot in the intersection of y-axis QueenValueOp and the x-axis RookValueEn. See the light area around south-west, that is the area where QueenValueOp and RookValueEn has good score in the match. The dark area in the plot means that the result of the match on those parameter values are not good.
Image
One other thing. The Optuna presentation says it can handle the functions (cutechess output) in parallel, are you using that feature, or is cutechess script already running the tournament in parallel?
I am using cutechess concurrency to run more than 1 game per trial or match.

I just successfully optimized the king safety weight, mobility weight and passer weight. I have tried improving it before without success but with optuna, I get promising result. See the method I use here. https://github.com/fsmosca/Optuna-Game- ... timization.

Joerg Oster
Posts: 783
Joined: Fri Mar 10, 2006 3:29 pm
Location: Germany

Re: Yet another parameter tuner using optuna framework

Post by Joerg Oster » Wed Sep 16, 2020 2:30 pm

Ferdy wrote:
Wed Sep 16, 2020 1:17 pm
3. I'm not sure if parameter changes to a 'quick match' change all five parameters at a time, or just one?
Sorry I don't understand the question.
I guess he wants to know if the tuner changes all parameters at once or one by one for a new trial.
In the document they refer to this as relational sampling and independent sampling.
Jörg Oster

Joerg Oster
Posts: 783
Joined: Fri Mar 10, 2006 3:29 pm
Location: Germany

Re: Yet another parameter tuner using optuna framework

Post by Joerg Oster » Wed Sep 16, 2020 2:53 pm

Ferdy wrote:
Wed Sep 16, 2020 1:17 pm
2. Uses cutechess tournament mode, with output the result of a quick match (25 rounds?)
Yes, number of game is settable, the more games the better.
The number of games per trial is probably dependent on the sensitivity of the parameters, no?
24 games seems very small in any case.

Btw, do you make use of the Pruning Algorithm by sending intermediate results of a trial?
Jörg Oster

chrisw
Posts: 3641
Joined: Tue Apr 03, 2012 2:28 pm

Re: Yet another parameter tuner using optuna framework

Post by chrisw » Wed Sep 16, 2020 3:30 pm

Joerg Oster wrote:
Wed Sep 16, 2020 2:30 pm
Ferdy wrote:
Wed Sep 16, 2020 1:17 pm
3. I'm not sure if parameter changes to a 'quick match' change all five parameters at a time, or just one?
Sorry I don't understand the question.
I guess he wants to know if the tuner changes all parameters at once or one by one for a new trial.
In the document they refer to this as relational sampling and independent sampling.
Yup, pretty much what I meant

chrisw
Posts: 3641
Joined: Tue Apr 03, 2012 2:28 pm

Re: Yet another parameter tuner using optuna framework

Post by chrisw » Wed Sep 16, 2020 3:41 pm

Ferdy wrote:
Wed Sep 16, 2020 1:17 pm
chrisw wrote:
Wed Sep 16, 2020 11:53 am

This looks extremely brilliant. I took a look at the Optuna site docs also. Trying to work out 'how it works', tell me if I understand it right, probably I don't of course.

1. You have a base engine, e1 and same engine, e2, except you can send parameter change instructions to e2
Yes they are the same engine, e1 which I call test engine and e2 as base engine. The e1 will always take the optimizer suggested param values. The e2 will always take the best param values. After a match if e1 wins that is more than 50% score, the param values used by e1 will become the best param values. This best param values then will be used by e2 in the next match. The e1 will take the new suggested values from the optimizer. The score of the match must be reported to the optimizer before taking its suggestions.
2. Uses cutechess tournament mode, with output the result of a quick match (25 rounds?)
Yes, number of game is settable, the more games the better.
3. I'm not sure if parameter changes to a 'quick match' change all five parameters at a time, or just one?
Sorry I don't understand the question.
As Georg Oster reply post.
4. Optuna tries parameter changes (within your supplied range?), and using smart algorithm, decides where to concentrate it's fire, so to speak.
Yes.
5. After N iterations, it gives you new values for the selected parameters.
Yes. They call it trials. A trial can be a 24-game match. After a trial we send the score of the match to the optimizer. After that it is safe to ask for the new suggested param values. Around here https://github.com/fsmosca/Optuna-Game- ... ner.py#L65
One thing I don't understand are the supplied output graphs, and what they are showing (apart from Optuna deciding that PAWNVAlUE is the 'most important' tunable item.
Slice plot: The y-axis is the score of the engine match, see the objective value label. The x-axis is the value of the parameter that has been tried. The color of the dot is the number of trials, trial number is high if it is dark, see the vertical legend at right. In here we can see if the tuner is making more suggestions that improves the objective value per parameter plot. We can also see here at what range of values has more dots or samples suggested. We can always see the y-axis if the dot performed well or not.
Image

The contour: There is a vertical bar legend at right. The lighter it is, the better. It is the match result of the game match or objective value. Let's take a look at the plot in the intersection of y-axis QueenValueOp and the x-axis RookValueEn. See the light area around south-west, that is the area where QueenValueOp and RookValueEn has good score in the match. The dark area in the plot means that the result of the match on those parameter values are not good.
Image
One other thing. The Optuna presentation says it can handle the functions (cutechess output) in parallel, are you using that feature, or is cutechess script already running the tournament in parallel?
I am using cutechess concurrency to run more than 1 game per trial or match.

I just successfully optimized the king safety weight, mobility weight and passer weight. I have tried improving it before without success but with optuna, I get promising result. See the method I use here. https://github.com/fsmosca/Optuna-Game- ... timization.
Looks very good. I still need to wrap my head around the graphs, will try later.

One thing, from your Github (and I don't want to flood you with suggested mods, I know how irritating that can be):
Second in order for the parameter values to be considered the best and replace the old best, it has to defeat the old best by more than 0.55 or 55% score. Normally this is only 0.5 or 50%.
I tried something like this in the past (except with random kicks to the parameters, not smart ones as you are doing with Optima), where the parameters, P1, give a better result than P0 - was to move P fractionally towards P1, rather than the full thing.
This could work in the case of the close 50-55% range. Just an idea, but I expect you are full of ideas already!

Joerg Oster
Posts: 783
Joined: Fri Mar 10, 2006 3:29 pm
Location: Germany

Re: Yet another parameter tuner using optuna framework

Post by Joerg Oster » Wed Sep 16, 2020 6:33 pm

chrisw wrote:
Wed Sep 16, 2020 3:30 pm
Joerg Oster wrote:
Wed Sep 16, 2020 2:30 pm
Ferdy wrote:
Wed Sep 16, 2020 1:17 pm
3. I'm not sure if parameter changes to a 'quick match' change all five parameters at a time, or just one?
Sorry I don't understand the question.
I guess he wants to know if the tuner changes all parameters at once or one by one for a new trial.
In the document they refer to this as relational sampling and independent sampling.
Yup, pretty much what I meant
If I understand it correctly, Optuna will eventually do both and also a mixture of both, to find out about the correlation of the parameters.
Jörg Oster

No4b
Posts: 23
Joined: Thu Jun 18, 2020 1:21 pm
Location: Moscow
Full name: Alexander Litov

Re: Yet another parameter tuner using optuna framework

Post by No4b » Wed Sep 16, 2020 6:58 pm

Very interesting tool!
I will definitely try to work with it.

Am i understanding correctly that the engine itself obtain parameters via command line fe as "QueenValueOp=975"?
And I need only one copy of the engine in the folder, tuner just will execute two copies of it with a different parameters set?

Post Reply