Piece weights with regression analysis (in Russian)

[PK]

Can You give similar output for Stockfish? It's interesting to see if it will differ from human values.

[Michel]

The logistic regression methods for tuning are really nice and I enjoyed reading Buro's paper. Obviously from a practical viewpoint such tuning methods work quite well.

Question: is there any research into why such statistically correct evaluation functions are expected to work?

The problem I have is that "statistical correctness" does not propagate through the game tree. In other words: if the leaf evaluation is statistically correct then there is a priori no reason why the result of a 1-ply search would be statistically correct.

The propagation through the game tree is via min/max functions and these have bad statistical properties...

[Xann]

Hi Michel,

The logistic regression methods for tuning are really nice and I enjoyed reading Buro's paper. Obviously from a practical viewpoint such tuning methods work quite well.

Question: is there any research into why such statistically correct evaluation functions are expected to work?

The problem I have is that "statistical correctness" does not propagate through the game tree. In other words: if the leaf evaluation is statistically correct then there is a priori no reason why the result of a 1-ply search would be statistically correct.

The propagation through the game tree is via min/max functions and these have bad statistical properties...

I fully agree with you, and propagating a random eval will quickly convince the sceptic. Machine learning in general expects you to use its results directly. Sometimes a bias such as a cost matrix (http://cse-wiki.unl.edu/wiki/index.php/ ... e_Learning) is introduced.

I haven't seen search/eval interaction modelled anywhere, but don't follow papers closely anymore. It seems very difficult, but could lead to biased error functions that give better results when combined with deep searches.

One element as to why it works at all for me is that regression can compensate its drawbacks by handling a large number of weights, even in the millions.

Fabien.

[brtzsnr]

My article on a polular Russian IT portal:
http://habrahabr.ru/post/254753/

It is in Russian, but I believe the text is quite understandable even without translation.

I must be doing something wrong. For 7442 games at 40/2+0.05 (two seconds for 40 moves plus 0.05 increment) between zurichess and itself I get very strange values. Using Texel's Tuning method I get good values.

Code: Select all

./pgnlearn.exe ../../pgn/games.pgn 
Reading file: ../../pgn/games.pgn
Games: 7442
Created file: .mat
Loading dataset...
[ 12807 x 5 ]
Solving (gradient method)...
Iter 0: [ 0 0 0 0 0 ] -> 0.693147
Iter 1000: [ 0.173482 0.210955 0.288834 0.391243 0.17849 ] -> 0.690838
Iter 2000: [ 0.174092 0.212807 0.2914 0.472017 0.187411 ] -> 0.69083
Iter 3000: [ 0.174207 0.213172 0.29191 0.487982 0.188865 ] -> 0.69083
Iter 4000: [ 0.17423 0.213245 0.292011 0.49115 0.18915 ] -> 0.69083
Iter 5000: [ 0.174234 0.213259 0.292031 0.491779 0.189206 ] -> 0.69083
Iter 6000: [ 0.174235 0.213262 0.292035 0.491904 0.189218 ] -> 0.69083
Iter 7000: [ 0.174236 0.213263 0.292036 0.491928 0.18922 ] -> 0.69083
Iter 8000: [ 0.174236 0.213263 0.292036 0.491933 0.18922 ] -> 0.69083
Iter 9000: [ 0.174236 0.213263 0.292036 0.491934 0.18922 ] -> 0.69083
Iter 10000: [ 0.174236 0.213263 0.292036 0.491934 0.18922 ] -> 0.69083

PIECE VALUES:

Pawn:   100
Knight: 122.399
Bishop: 167.61
Rook:   282.339
Queen:  108.6
Press ENTER to finish

[WinPooh]

My article on a polular Russian IT portal:
http://habrahabr.ru/post/254753/

It is in Russian, but I believe the text is quite understandable even without translation.

I must be doing something wrong. For 7442 games at 40/2+0.05 (two seconds for 40 moves plus 0.05 increment) between zurichess and itself I get very strange values. Using Texel's Tuning method I get good values.

Code: Select all

./pgnlearn.exe ../../pgn/games.pgn 
Reading file: ../../pgn/games.pgn
Games: 7442
Created file: .mat
Loading dataset...
[ 12807 x 5 ]
Solving (gradient method)...
Iter 0: [ 0 0 0 0 0 ] -> 0.693147
Iter 1000: [ 0.173482 0.210955 0.288834 0.391243 0.17849 ] -> 0.690838
Iter 2000: [ 0.174092 0.212807 0.2914 0.472017 0.187411 ] -> 0.69083
Iter 3000: [ 0.174207 0.213172 0.29191 0.487982 0.188865 ] -> 0.69083
Iter 4000: [ 0.17423 0.213245 0.292011 0.49115 0.18915 ] -> 0.69083
Iter 5000: [ 0.174234 0.213259 0.292031 0.491779 0.189206 ] -> 0.69083
Iter 6000: [ 0.174235 0.213262 0.292035 0.491904 0.189218 ] -> 0.69083
Iter 7000: [ 0.174236 0.213263 0.292036 0.491928 0.18922 ] -> 0.69083
Iter 8000: [ 0.174236 0.213263 0.292036 0.491933 0.18922 ] -> 0.69083
Iter 9000: [ 0.174236 0.213263 0.292036 0.491934 0.18922 ] -> 0.69083
Iter 10000: [ 0.174236 0.213263 0.292036 0.491934 0.18922 ] -> 0.69083

PIECE VALUES:

Pawn:   100
Knight: 122.399
Bishop: 167.61
Rook:   282.339
Queen:  108.6
Press ENTER to finish

Probably it is due to my gradient solver - it is very simple and straightforward, looks like some local minimum was caught. Another reason can be PGN file - the parser can't handle comments or folded lines of moves.