Crafty eval tweak

[jarkkop]

Best result so far. Give these parameters a try.

-----------------Crafty-233-32-jaOrig-----------------
Crafty-233-32-jaOrig - Rybka v1.1.w32 : 34.0/100 20-52-28 (001==10=0=01101==0000=0==100=00=100001=01001==0==101010=00=1=001==0101100000000==1=0=0000000=000==01) 34% -115
-----------------Rybka v1.1.w32-----------------
Rybka v1.1.w32 - Crafty-233-32-jaOrig : 66.0/100 52-20-28 (110==01=1=10010==1111=1==011=11=011110=10110==1==010101=11=0=110==1010011111111==0=1=1111111=111==10) 66% +115

personality 11 -94 94
personality 12 -333 333
personality 13 -323 323
personality 14 -492 492
personality 15 -1046 1046


old results

-----------------Crafty-233-32-ja-----------------
Crafty-233-32-ja - Rybka v1.1.w32 : 29.5/100 18-59-23 (==000==0=00001010000==1000=100000=00==0=01=10=0010001000000001=010=1101=0010=010=10000===0100=100000) 30% -147
-----------------Rybka v1.1.w32-----------------
Rybka v1.1.w32 - Crafty-233-32-ja : 70.5/100 59-18-23 (==111==1=11110101111==0111=011111=11==1=10=01=1101110111111110=101=0010=1101=101=01111===1011=011111) 71% +156


-----------------Crafty-233-32-ja-----------------
Crafty-233-32-ja - Rybka v1.1.w32 : 27.0/100 14-60-26 (0=10====100100000010=0000==1=0=00100000=0010=00010=0000100=000001=0=10000=0=0010===10=00=0==0=000010) 27% -173
-----------------Rybka v1.1.w32-----------------
Rybka v1.1.w32 - Crafty-233-32-ja : 73.0/100 60-14-26 (1=01====011011111101=1111==0=1=11011111=1101=11101=1111011=111110=1=01111=1=1101===01=11=1==1=111101) 73% +173

[jarkkop]

Could you check also other parameters like the bad_trade? Not all of them seem to passed thru personality command. Atleast passed pawns seem to be hardcoded in evaluate.c in 23.3 and thus not changabeble thru this interface.

[bob]

Could you check also other parameters like the bad_trade? Not all of them seem to passed thru personality command. Atleast passed pawns seem to be hardcoded in evaluate.c in 23.3 and thus not changabeble thru this interface.

This is on my to-do list. As always, development precedes clean-up. And, in fact, there are some "magic numberts" in the eval that need to become variables at some point so that they can be changed. It is a planned topic to address...

[jarkkop]

Original crafty23.4 is with default parameters is 17 ELO weaker than crafty with below parameters. The modified crafty had better results against Rybka 1.1 also. Can you verify this with your cluster Bob?

personality 11 -99 99
personality 12 -353 353
personality 13 -352 352
personality 14 -521 521
personality 15 -1031 1031

1000 game 1sec+10ms game ended victory for mod.
Score of crafty vs craftyOrig: 384 - 334 - 282 (525-475)


These games are silver suite 1min+1sec

Crafty-233-32-ja - Rybka v1.1.w32 : 27.0/100 14-60-26 (0=10====100100000010=0000==1=0=00100000=0010=00010=0000100=000001=0=10000=0=0010===10=00=0==0=000010) 27% -173

Crafty-234-32-ja - Rybka v1.1.w32 : 38.0/100 23-47-30 (==1==101=000==000000011000100=10=0000010==0==1=001=00==1=0=11000=01=10=00000001=1=1100====011=01010=) 38% -85

Crafty-234-32-ja mod - Rybka v1.1.w32 : 40.5/100 22-41-37 (1001=00===00=01=====001011001100=0=001==011=0101=0=0000001==00====1=0=1=1=00==11=11==1==0=00000==000) 41% -63

[bob]

Original crafty23.4 is with default parameters is 17 ELO weaker than crafty with below parameters. The modified crafty had better results against Rybka 1.1 also. Can you verify this with your cluster Bob?

personality 11 -99 99
personality 12 -353 353
personality 13 -352 352
personality 14 -521 521
personality 15 -1031 1031

1000 game 1sec+10ms game ended victory for mod.
Score of crafty vs craftyOrig: 384 - 334 - 282 (525-475)


These games are silver suite 1min+1sec

Crafty-233-32-ja - Rybka v1.1.w32 : 27.0/100 14-60-26 (0=10====100100000010=0000==1=0=00100000=0010=00010=0000100=000001=0=10000=0=0010===10=00=0==0=000010) 27% -173

Crafty-234-32-ja - Rybka v1.1.w32 : 38.0/100 23-47-30 (==1==101=000==000000011000100=10=0000010==0==1=001=00==1=0=11000=01=10=00000001=1=1100====011=01010=) 38% -85

Crafty-234-32-ja mod - Rybka v1.1.w32 : 40.5/100 22-41-37 (1001=00===00=01=====001011001100=0=001==011=0101=0=0000001==00====1=0=1=1=00==11=11==1==0=00000==000) 41% -63

Early results are not particularly good:

Code: Select all

    Crafty-23.5-2        2666    4    4 30000   62%  2570   21% 
    Crafty-23.5-1        2665    4    4 30000   62%  2570   21% 
    Crafty-23.4-1        2664    4    4 30000   62%  2570   21% 
    Crafty-23.4-2        2663    4    4 30000   62%  2570   20% 
    Crafty-23.5R01       2658    7    7  8144   61%  2570   21% 
    Crafty-23.3-1        2650    4    4 30000   60%  2570   21% 
    Crafty-23.3-2        2649    4    4 30000   60%  2570   21% 
    Crafty-23.2-1        2600    4    4 30000   54%  2570   21% 
    Crafty-23.2-2        2596    4    4 30000   54%  2570   21% 

Has a while to go...

I am using something like 28 nodes as the cluster is busy, so 56 games at a time.

23.5R01 is 23.5 + the personality changes (piece values) you suggested...

[jarkkop]

Thanks for trying thou. It must have been luck then. Everything you say is within the error bars of my game count.

[bob]

Thanks for trying thou. It must have been luck then. Everything you say is within the error bars of my game count.

That's the problem when things are pretty "close". Takes a _ton_ of games to accurately measure the effect...

[jarkkop]

Will you post final result of test batch you started when it is finished?

Next I have to be more ambitious. I will try tune the material values. The Crafty model at the moment is very simple and there has to be more optimal values. I'll let you know if I found anything useful.

[bob]

Will you post final result of test batch you started when it is finished?

Next I have to be more ambitious. I will try tune the material values. The Crafty model at the moment is very simple and there has to be more optimal values. I'll let you know if I found anything useful.

Finished just as I got home:

Code: Select all

    Crafty-23.5-2        2663    4    4 30000   62%  2566   21%
    Crafty-23.5-1        2661    4    4 30000   62%  2566   21%
    Crafty-23.4-1        2660    4    4 30000   62%  2566   21%
    Crafty-23.4-2        2660    4    4 30000   62%  2566   20%
    Crafty-23.5R01-2     2659    4    4 30000   62%  2566   21%
    Crafty-23.5R01-1     2657    4    4 30000   62%  2566   21%
    Crafty-23.3-1        2647    4    4 30000   60%  2566   21%
    Crafty-23.3-2        2646    4    4 30000   60%  2566   21%

We tuned each piece value individually to get to the point we are at...

[jarkkop]

After so many games everything is is still within error margin. As you said when engines are close then the games needed is enormous to put them in order.