tuning for the uninformed

Discussion of chess software programming and technical issues.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Henk
Posts: 5103
Joined: Mon May 27, 2013 8:31 am

Re: tuning for the uninformed

Post by Henk » Thu Nov 23, 2017 7:21 pm

I don't know how to get an estimate of the value of a position without using other engines.

Maybe use your own engine and re-search it on shallow depth. But who will say that that gives a right estimate if your current evaluation is bad.

You can not evaluate every test position manually for that is too much work.

sandermvdb
Posts: 124
Joined: Sat Jan 28, 2017 12:29 pm
Location: The Netherlands

Re: tuning for the uninformed

Post by sandermvdb » Thu Nov 23, 2017 7:26 pm

Henk wrote:I don't know how to get an estimate of the value of a position without using other engines.

Maybe use your own engine and re-search it on shallow depth. But who will say that that gives a right estimate if your current evaluation is bad.

You can not evaluate every test position manually for that is too much work.
quiet-labeled.epd contains the outcome of every position :)

Henk
Posts: 5103
Joined: Mon May 27, 2013 8:31 am

Re: tuning for the uninformed

Post by Henk » Thu Nov 23, 2017 7:36 pm

sandermvdb wrote:
Henk wrote:I don't know how to get an estimate of the value of a position without using other engines.

Maybe use your own engine and re-search it on shallow depth. But who will say that that gives a right estimate if your current evaluation is bad.

You can not evaluate every test position manually for that is too much work.
quiet-labeled.epd contains the outcome of every position :)
Don't understand. What is quiet-labeled.epd?

sandermvdb
Posts: 124
Joined: Sat Jan 28, 2017 12:29 pm
Location: The Netherlands

Re: tuning for the uninformed

Post by sandermvdb » Thu Nov 23, 2017 7:43 pm

Henk wrote:
sandermvdb wrote:
Henk wrote: Maybe use your own engine and re-search it on shallow depth. But who will say that that gives a right estimate if your current evaluation is bad.

You can not evaluate every test position manually for that is too much work.
quiet-labeled.epd contains the outcome of every position :)
Don't understand. What is quiet-labeled.epd?
Sorry, that is one of the testsets by Alexandru Mosoi, the author of Zurichess. It conains quiet positions including the outcome of the game.

flok

Re: tuning for the uninformed

Post by flok » Fri Nov 24, 2017 9:15 am

sandermvdb wrote:The basic idea is pretty simple: calculate the error of the evaluation when it is compared to the actual outcome of the positions.
What do you mean by that?
Do you mean the following:
- fen as input
- calc move with an eval val
- calc eval of the move that should've been moved
- compare these two (how? percentual difference? or what?)

sandermvdb
Posts: 124
Joined: Sat Jan 28, 2017 12:29 pm
Location: The Netherlands

Re: tuning for the uninformed

Post by sandermvdb » Fri Nov 24, 2017 9:30 am

flok wrote:
sandermvdb wrote:The basic idea is pretty simple: calculate the error of the evaluation when it is compared to the actual outcome of the positions.
What do you mean by that?
Do you mean the following:
- fen as input
- calc move with an eval val
- calc eval of the move that should've been moved
- compare these two (how? percentual difference? or what?)
No. I mean:
- fen as input
- calculate evaluation
- calculate error: compare evaluation score with the actual outcome. If the evaluation calculates that white has a big advantage but black wins -> big error. The exact formula is described on the cpw. This is my (pseudo) implementation, where K=1.3:

Code: Select all

public double calculateTotalError() {
	double totalError = 0;
	for &#40;Entry<String, Double> entry &#58; fens.entrySet&#40;)) &#123; // fens contains all positions, including the outcome
		ChessBoard cb = new ChessBoard&#40;entry.getKey&#40;));
		totalError += Math.pow&#40;entry.getValue&#40;) - calculateSigmoid&#40;Eval.calculateScore&#40;cb&#41;), 2&#41;;
	&#125;
	totalError /= fens.size&#40;);
	return totalError;
&#125;

public double calculateSigmoid&#40;int score&#41; &#123;
	return 1 / &#40;1 + Math.pow&#40;10, -1.3 * score / 400&#41;);
&#125;

CheckersGuy
Posts: 272
Joined: Wed Aug 24, 2016 7:49 pm

Re: tuning for the uninformed

Post by CheckersGuy » Fri Nov 24, 2017 11:04 pm

sandermvdb wrote:
flok wrote:If anyone is willing to explain the Texel tuning method tht would be great!

Sofar I understand I have to let it play (well, run QS + eval on FENs) millions of games and then do something with the evaluation-value. But what? I don't understand the wiki explanation.
The basic idea is pretty simple: calculate the error of the evaluation when it is compared to the actual outcome of the positions. Lower a particular evaluation parameter and check if the error has improved, if not, higher the parameter, if again not improved, keep the original value. Do this for all parameters until you have reached the lowest error.
This is the local search algorithm but I would assume that it is better to run some gradient based algorithm first (Maybe gradient descent or gauss-newton). Then if the error doesn't change by much anymore I would switch to local search.

I am going to implement texel tuning this week and see what I get :P

Henk
Posts: 5103
Joined: Mon May 27, 2013 8:31 am

Re: tuning for the uninformed

Post by Henk » Sat Nov 25, 2017 11:30 am

All does not work if search space has a great many local optima and only very few global optima that you are interested in. But simulated annealing taking too long.

CheckersGuy
Posts: 272
Joined: Wed Aug 24, 2016 7:49 pm

Re: tuning for the uninformed

Post by CheckersGuy » Sat Nov 25, 2017 1:58 pm

Henk wrote:All does not work if search space has a great many local optima and only very few global optima that you are interested in. But simulated annealing taking too long.
Local search and any other practical algorithm to minimize the error will end up in a local optimum.

Henk
Posts: 5103
Joined: Mon May 27, 2013 8:31 am

Re: tuning for the uninformed

Post by Henk » Sat Nov 25, 2017 2:48 pm

CheckersGuy wrote:
Henk wrote:All does not work if search space has a great many local optima and only very few global optima that you are interested in. But simulated annealing taking too long.
Local search and any other practical algorithm to minimize the error will end up in a local optimum.
Wasn't it that if it optimizes enough parameters you won't get trapped in a local optimum. I can't remember.

Post Reply