Leela Nets for Human Training and "Style"

[Ferdy]

3- player comparison on mobility.










Result table, topped by Capablanca.

Code: Select all

Best on Mobility. Given mobility=100cp
name         mobcp  scorecp_pred
Capablanca   100    21.59   
Tal          100    20.99   
Fischer      100    16.20  

[Ferdy]

3-player comparison on king safety.

Code: Select all

Best on KingSafetyCP. Given KingSafetyCP=100
name         mobcp  scorecp_pred
Fischer      100    67.96   
Capablanca   100    44.28   
Tal          100    18.15   

[Ferdy]

3-player comparison on Passed pawn.

Code: Select all

Best on PasserCP. Given PasserCP=100
name         mobcp  scorecp_pred
Capablanca   100    22.45   
Fischer      100    17.27   
Tal          100    -8.43   

[chrisw]

3-player comparison on king safety.

Code: Select all

Best on KingSafetyCP. Given KingSafetyCP=100
name         mobcp  scorecp_pred
Fischer      100    67.96   
Capablanca   100    44.28   
Tal          100    18.15   

Nice.
Okay, so as I understand this ...
Player is white, positions are white to move, game result is a white win, evals are normalised 0-1?
Clarifying questions ...
where do you get the feature evaluation from? the root or the final move in the PV?
I guess in an ideal world, we should have a straight line with positive slope? As a control, presumably SF should show this behaviour?
What happens to negative values in the normalisation, they are in the range 0.0 to O.5?
(I’m guessing draws, losses and black are going to be processed later, somehow?)

Results are “curious”, how to interpret, or are we just at early stages? Are you motivated to continue from what you found so far?

[Ferdy]

Nice.
Okay, so as I understand this ...
Player is white, positions are white to move, game result is a white win, evals are normalised 0-1?

Yes

Clarifying questions ...
where do you get the feature evaluation from? the root or the final move in the PV?

Root, as I am using eval command of Stockfish.
Example.


I am only taking the king safety value (2.29, or cp=2.29*100) of Mg under total column for this particular epd. I don't know yet how Stockfish would interploate the Mg and Eg values.

I guess in an ideal world, we should have a straight line with positive slope? As a control, presumably SF should show this behaviour?

Ideally yes.

What happens to negative values in the normalisation, they are in the range 0.0 to O.5?

Yes, I am using,
norm_x = (x - min(data)) / (max(data) - min(data))

data = [-100, 10, 45, 150, -36, 75, -20, 64]
x = -100
norm_x = (-100 - (-100)) / (150 - (-100)) = 0
x = -36
norm_x = (-36 - (-100)) / (150 - (-100)) = 0.256

(I’m guessing draws, losses and black are going to be processed later, somehow?)

Results are “curious”, how to interpret, or are we just at early stages? Are you motivated to continue from what you found so far?

I have an issue on Tal getting negative on PassedPawn.

Code: Select all

Best on PasserCP. Given PasserCP=100
name         mobcp  scorecp_pred
Capablanca   100    22.45   
Fischer      100    17.27   
Tal          100    -8.43   

The plot has points with low value of score and high value of passer, this could be the reason.



However there is another method of getting regression results without normalizing the values, after all our values are already on same units all in cp.

I am using sklearn's linear_model.LinearRegression().

Code: Select all

from sklearn import linear_model
regr = linear_model.LinearRegression()
regr.fit(X, Y)
intercept = regr.intercept_
coef = regr.coef_

X is the feature in list.
Y is the pos score in list.

If I will not apply a normalization, I get this.

Code: Select all

Best on PasserCP. Given PasserCP=100
name         mobcp  scorecp_pred
Fischer      100    200.44  
Capablanca   100    199.27  
Tal          100    93.30 

Completely different result but Tal's value is now positive.

If I have to continue this, I need to analyze every position at more than 1s of Stockfish analysis to get the score as dependent for regression. Currently, I am only using 1s. Also need to decide of whether to normalize or not. Also need to interpolate the Mg and Eg values of feature under Total column in Stockfish eval command results.

I tend to prefer without applying normalization to [0, 1] because of that negative value.
The function linear_model.LinearRegression()
has normalize parameter defaulted to False.

Code: Select all

regr = linear_model.LinearRegression(normalize=False)

If I will not apply normalization to [0, 1] to the data and use,
a. regr = linear_model.LinearRegression(normalize=False)
b. regr = linear_model.LinearRegression(normalize=True)

The (a) and (b) results are the same. The normalize parameter value has no effect.

[chrisw]

Nice.
Okay, so as I understand this ...
Player is white, positions are white to move, game result is a white win, evals are normalised 0-1?

Yes

Clarifying questions ...
where do you get the feature evaluation from? the root or the final move in the PV?

Root, as I am using eval command of Stockfish.
Example.


I am only taking the king safety value (2.29, or cp=2.29*100) of Mg under total column for this particular epd. I don't know yet how Stockfish would interploate the Mg and Eg values.

I guess in an ideal world, we should have a straight line with positive slope? As a control, presumably SF should show this behaviour?

Ideally yes.

What happens to negative values in the normalisation, they are in the range 0.0 to O.5?

Yes, I am using,
norm_x = (x - min(data)) / (max(data) - min(data))

data = [-100, 10, 45, 150, -36, 75, -20, 64]
x = -100
norm_x = (-100 - (-100)) / (150 - (-100)) = 0
x = -36
norm_x = (-36 - (-100)) / (150 - (-100)) = 0.256

(I’m guessing draws, losses and black are going to be processed later, somehow?)

Results are “curious”, how to interpret, or are we just at early stages? Are you motivated to continue from what you found so far?

I have an issue on Tal getting negative on PassedPawn.

Code: Select all

Best on PasserCP. Given PasserCP=100
name         mobcp  scorecp_pred
Capablanca   100    22.45   
Fischer      100    17.27   
Tal          100    -8.43   

The plot has points with low value of score and high value of passer, this could be the reason.



However there is another method of getting regression results without normalizing the values, after all our values are already on same units all in cp.

I am using sklearn's linear_model.LinearRegression().

Code: Select all

from sklearn import linear_model
regr = linear_model.LinearRegression()
regr.fit(X, Y)
intercept = regr.intercept_
coef = regr.coef_

X is the feature in list.
Y is the pos score in list.

yes, I do pretty much the same (in a different context). Standard code.

Code: Select all

# linear regression
    from sklearn.linear_model import LinearRegression

    x, y = np.array(x), np.array(y)
    model = LinearRegression().fit(x, y)
    r_sq = model.score(x, y)
    intercept, coefficients = model.intercept_, model.coef_
    print('coefficient of determination:', r_sq)
    print('intercept:', intercept)
    print('coefficients:', coefficients, sep='\n')

If I will not apply a normalization, I get this.

Code: Select all

Best on PasserCP. Given PasserCP=100
name         mobcp  scorecp_pred
Fischer      100    200.44  
Capablanca   100    199.27  
Tal          100    93.30 

Completely different result but Tal's value is now positive.

there is some dependency on the intercept value, probably easier to see when more variables done at once.
And yes, why normalise at all? The Regression model should be able to cope.

If I have to continue this, I need to analyze every position at more than 1s of Stockfish analysis to get the score as dependent for regression. Currently, I am only using 1s. Also need to decide of whether to normalize or not. Also need to interpolate the Mg and Eg values of feature under Total column in Stockfish eval command results.

can you send Mg, Eg for both sides to the model? eg four variables. Could be that a GM is asymmetric

I tend to prefer without applying normalization to [0, 1] because of that negative value.
The function linear_model.LinearRegression()
has normalize parameter defaulted to False.

Code: Select all

regr = linear_model.LinearRegression(normalize=False)

If I will not apply normalization to [0, 1] to the data and use,
a. regr = linear_model.LinearRegression(normalize=False)
b. regr = linear_model.LinearRegression(normalize=True)

The (a) and (b) results are the same. The normalize parameter value has no effect.

That figures.
If time is an issue, 100ms? SF is pretty good even at fast times.
I'ld be inclined to give the Regression function all variables and let it build a regression_weights_list[] for a bunch of GMs.

then deltas off the pool mean, get standard deviation each feature, then convert the deltas to sigmas. Then who, and with which feature has high sigmas .....? idea being that "style" is reflected in the outliers.

[Ferdy]

can you send Mg, Eg for both sides to the model? eg four variables. Could be that a GM is asymmetric

Some test result without norm to [0,1], also added the 4 variables on mobility, i.e WMobilityMgCP, WMobilityEgCP, BMobilityMgCP, BMobilityEgCP.

Regression summary results when each feature has a value of 100cp.

Example
Player: Tal
Feature: PasserCP
PasserCP: 100
ScoreCP prediction: 93 (expected position evaluation according to Stockfish at 1s of search)

Code: Select all

       Feature  Tal  Capablanca  Fischer
      PasserCP   93         199      200
      ThreatCP  125         167      153
  KingSafetyCP  122         171      167
    MobilityCP  212         240      217
 WMobilityMgCP  139         178      168
 WMobilityEgCP  108         147      137
 BMobilityMgCP   79         122      121
 BMobilityEgCP  132         163      156

Looks like overall Capablanca is dangerous, he tops on Threat, KingSafety and Mobility and very close in Passer.

[supersharp77]

can you send Mg, Eg for both sides to the model? eg four variables. Could be that a GM is asymmetric

Some test result without norm to [0,1], also added the 4 variables on mobility, i.e WMobilityMgCP, WMobilityEgCP, BMobilityMgCP, BMobilityEgCP.

Regression summary results when each feature has a value of 100cp.

Example
Player: Tal
Feature: PasserCP
PasserCP: 100
ScoreCP prediction: 93 (expected position evaluation according to Stockfish at 1s of search)

Code: Select all

       Feature  Tal  Capablanca  Fischer
      PasserCP   93         199      200
      ThreatCP  125         167      153
  KingSafetyCP  122         171      167
    MobilityCP  212         240      217
 WMobilityMgCP  139         178      168
 WMobilityEgCP  108         147      137
 BMobilityMgCP   79         122      121
 BMobilityEgCP  132         163      156

Looks like overall Capablanca is dangerous, he tops on Threat, KingSafety and Mobility and very close in Passer.

Tal would rather "Play a beautiful mating pattern rather than push the passer.hence the low pp score.Fischer really believed in the power of the passed pawn hence the 200 score..Tal played some closed types of positions (ex KingsIndianDefence) Tals king safety number is quite low (willing to take great risks) Capa great with 'positional threats' hence his high number Fischers Numbers also strong across the board..Both Fischer and Capa known as 'great technicians'
more "machine like" especially in the late middlegame and endgame...I would take a look at Ponomariov..Topalov and Grischuk (all fierce attackers)

[chrisw]

can you send Mg, Eg for both sides to the model? eg four variables. Could be that a GM is asymmetric

Some test result without norm to [0,1], also added the 4 variables on mobility, i.e WMobilityMgCP, WMobilityEgCP, BMobilityMgCP, BMobilityEgCP.

Regression summary results when each feature has a value of 100cp.

Example
Player: Tal
Feature: PasserCP
PasserCP: 100
ScoreCP prediction: 93 (expected position evaluation according to Stockfish at 1s of search)

Code: Select all

       Feature  Tal  Capablanca  Fischer
      PasserCP   93         199      200
      ThreatCP  125         167      153
  KingSafetyCP  122         171      167
    MobilityCP  212         240      217
 WMobilityMgCP  139         178      168
 WMobilityEgCP  108         147      137
 BMobilityMgCP   79         122      121
 BMobilityEgCP  132         163      156

Looks like overall Capablanca is dangerous, he tops on Threat, KingSafety and Mobility and very close in Passer.

I have to admit to struggling to wrap my head around interpreting this

[Ferdy]

can you send Mg, Eg for both sides to the model? eg four variables. Could be that a GM is asymmetric

Some test result without norm to [0,1], also added the 4 variables on mobility, i.e WMobilityMgCP, WMobilityEgCP, BMobilityMgCP, BMobilityEgCP.

Regression summary results when each feature has a value of 100cp.

Example
Player: Tal
Feature: PasserCP
PasserCP: 100
ScoreCP prediction: 93 (expected position evaluation according to Stockfish at 1s of search)

Code: Select all

       Feature  Tal  Capablanca  Fischer
      PasserCP   93         199      200
      ThreatCP  125         167      153
  KingSafetyCP  122         171      167
    MobilityCP  212         240      217
 WMobilityMgCP  139         178      168
 WMobilityEgCP  108         147      137
 BMobilityMgCP   79         122      121
 BMobilityEgCP  132         163      156

Looks like overall Capablanca is dangerous, he tops on Threat, KingSafety and Mobility and very close in Passer.

I have to admit to struggling to wrap my head around interpreting this

If you give an advantage of 100cp (static eval) on passed pawn to Tal, Tal's position score is only 93cp (according to Stockfish at 1s), but if you give that 100cp (static eval) to Capablanca, Capablanca's position score is 199cp. This makes Capablanca dangerous to play over Tal. The constant is 100cp passed pawn advantage.

For other feature like Mobility, you give Tal 100cp mobility advantage, Tal's position score is only 212cp. But if you give that 100cp to Capablanca, its position score is 240cp. Capablanca is more dangerous here too.