STS - List the Order of Importance

[noctiferus]

My statist software JMP licence has expired, and have to wait till monday to get the new.

From what I have (data analysis excel only -:( ) Bison seems to be an outlier: is it possible that CEGT elo rfers to a newer edition?
Moreover, the X6 estimate (the negative one), seems to be not relevant, may well be 0.

A doubt: are we sure that those indicated as best replies in X6 are realy always the best? it could explain his possible negative effect.

[noctiferus]

As you may have read in the forum, I'm blocked by the unaivalabilyty of JMP, so I cant'do some analyses. Just a few impressions, with my limited tools:

- looking to some of your simple regression graphs Y vs STS-k, I believe that some engines (low Elo) are very influential (did you compute VIF and studentized residuals?).
for example. in X6 one observation looks as heavily conditioning the slope. If it is an outlier, regression would drastically change.

I'd suggest Swami to include some few more CEGT weak engine, and if possible, also some very strong ones, in order to increase Elo range, that helps to reduce the width on the conf intervals, because of the inflactive factor at the denominator of the conf interv variance.

- another idea: I'm not sure we should use raw weights for inferring the importance of STSx wrt Elo. I'm thinking about it, but IMHO we should standardize engine performance STS-x ratings, to avoid biases due to different STS difficulties.

Sorry for the delay in my analyses.
e.
_

[swami]

I'd suggest Swami to include some few more CEGT weak engine, and if possible, also some very strong ones, in order to increase Elo range, that helps to reduce the width on the conf intervals, because of the inflactive factor at the denominator of the conf interv variance.

Sure I will run the test over the next few weeks and try to include atleast 20 more engines that are listed in the ratings site also.

I will update the excel file and let you know in this thread.

[noctiferus]

Good. Also some very low engines (2200 or so) would be useful,together with a few 2800+. In the meantime I' try to get the new licence: now I'm dwnloading JMP trial from home. let's see.

[noctiferus]

Up to now I found an almost acceptable predictive Cegt Elo model (Sts values non normalized), with a mean standard error around the estimates about 45 Elos (no diagnostic trouble with the possible exception of Typhoon). Some funny things appear, however... More to follow.
When finished I'd send you a summary-comment +detailed technical results, if you want (and can) upload them here.
Ciao.
E.

PS- I believe even more strongly that a (very few) very weak and very strong engines would give origin to better models.

[swami]

PS- I believe even more strongly that a (very few) very weak and very strong engines would give origin to better models.

Yes, that's what I had said earlier. Give more data and there will be more accuracy. That's the statistics.

I will be only downloading engines that already have established rating.

I'm also considering adding UEL ratings list in another column because most of the recently released UCI engines seems to have been tested.

[Jouni]

I hope You do calculations also with total used time. At least there is more accuracy - 4 digit with one second accuracy

Jouni

[noctiferus]

yes we need more, and on a bigger range: actually, the predictions are rather unstable, for what concers out-of-sample engines

[swami]

I hope You do calculations also with total used time. At least there is more accuracy - 4 digit with one second accuracy

Jouni

Yeah, I could consider that factor too. Thanks for letting me know!

There's this as well:

Total out of 1000 - partial credit scoring.

Arena unfortunately doesn't do this.

But I've gotten information from atleast 2 engine authors that they are willing to offer me the tool to do that!

They are designing the tool for calculating the partial credit scores: (see below for example)

r5k1/pp1n1rpp/2pb1q2/2N5/1P3B2/P2P2P1/3Q1PKP/1R2R3 b - - bm Bxc5; c0 "Bxc5=10, Bxf4=4, Nxc5=5, Qd4=7"; id "STS(v5.0) Bishop vs Knight.097";

@ Enrico, Would these scores be most useful?

[swami]

They are designing the tool for calculating the partial credit scores: (see below for example)

r5k1/pp1n1rpp/2pb1q2/2N5/1P3B2/P2P2P1/3Q1PKP/1R2R3 b - - bm Bxc5; c0 "Bxc5=10, Bxf4=4, Nxc5=5, Qd4=7"; id "STS(v5.0) Bishop vs Knight.097";

To give example sample run of Umko 0.1 @ ChessGUI...

Umko 0.1 @ ChessGUI

Re-Capturing

71 / 100
849/1000
fxe5
0.031
361.698
1000.000
0.62
rqr3k1/4bpp1/pBbp1n1p/Pp2p3/3PPP2/2N1QNP1/1P4P1/R4R1K w - - bm fxe5; c0 "fxe5=10, Rae1=2, d5=1, dxe5=4";
f4e5 d6e5 f3e5 c6e8 a1c1 e7d6 f1f6 g7f6 e5g4 d6e7 c1f1 e8d7 g4h6 g8f8