Engine testing: search vs eval

Discussion of chess software programming and technical issues.

Moderators: hgm, Dann Corbit, Harvey Williamson

Rémi Coulom
Posts: 438
Joined: Mon Apr 24, 2006 8:06 pm

Re: Engine testing: search vs eval

Post by Rémi Coulom »

lucasart wrote:
ZirconiumX wrote:Richard,

The work has been done for you.

http://remi.coulom.free.fr/CLOP/

It can tune 12 values at the same time.

Matthew:out
Have you ever tried it ?

I spent quite a lot of time trying to get CLOP to converge, but it never did, and the QLR max never stayed inside the window. All I wanted to do is optimize 5 parameters (5 piece values). I tried all sort of combinations to reduce the dimension (like force knight=bishop or even force everything byt N=B=x and optimize x), but I never got any sign of convergence, even after 10,000 games. And I was using pretty large windows too.

I wonder if anyone managed to get it working at least for piece value tuning. (5 dimension).
Are you sure you have no bug in your script? Tuning 5 piece values should be no problem at all for CLOP. One the web page of CLOP, you'll find example data for piece values.

I can take a look at your data if you send it to me.

Rémi
User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 4:27 pm

Re: Engine testing: search vs eval

Post by Don »

Rémi Coulom wrote:
lucasart wrote:
ZirconiumX wrote:Richard,

The work has been done for you.

http://remi.coulom.free.fr/CLOP/

It can tune 12 values at the same time.

Matthew:out
Have you ever tried it ?

I spent quite a lot of time trying to get CLOP to converge, but it never did, and the QLR max never stayed inside the window. All I wanted to do is optimize 5 parameters (5 piece values). I tried all sort of combinations to reduce the dimension (like force knight=bishop or even force everything byt N=B=x and optimize x), but I never got any sign of convergence, even after 10,000 games. And I was using pretty large windows too.

I wonder if anyone managed to get it working at least for piece value tuning. (5 dimension).
Are you sure you have no bug in your script? Tuning 5 piece values should be no problem at all for CLOP. One the web page of CLOP, you'll find example data for piece values.

I can take a look at your data if you send it to me.

Rémi
Larry and I have used Clop and have had some success with it. It clearly does what it is supposed to do. It is not perfect and did not always improve what we already had, but it was especially useful in an advisory role - it sometime would clue us in to changes in weights that we would then explore.
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
Richard Allbert
Posts: 792
Joined: Wed Jul 19, 2006 9:58 am

Re: Engine testing: search vs eval

Post by Richard Allbert »

Hi Don,

I've reduced all weights to very small values.

I've run this base version at 20s+0.2s over 8400 games vs 7 oppenents, using the crafty openings epd. I'm now running the same at 4s+0.1s to see if the results are similar, the faster the TC control the better.

On another computer, I'm running the STS suite.

I've built into the engine a function which runs through the 1300 STS positions tuning specified weights, e.g if I specify Rook on Seventh between 2 and 22 increment of 4, it will run these variations through the STS suite, showing me the score in csv format.

Maybe this will give me a good starting point for the values!

I'll post the results, although this will take some time

Regards

Richard