Texel tuning Zurichess quiet like generator

Discussion of chess software programming and technical issues.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
xr_a_y
Posts: 663
Joined: Sat Nov 25, 2017 1:28 pm
Location: France

Re: Texel tuning Zurichess quiet like generator

Post by xr_a_y » Fri Aug 09, 2019 6:39 am

jdart wrote:
Fri Aug 09, 2019 1:58 am
Keeping only the ones with two players > 2600 elo
I don't think there is advantage in excluding games from weaker players. You want a variety of positions. The objective is to have a decent sample of possible positions that an actual search may evaluate and these will be quite varied and many of them uneven (for example, positions the q-search will cut off due to high eval).

--Jon
Yes but I also need the outcome of the game to be "reliable" and I suspect top engines are more prone to blunder.

fabianVDW
Posts: 45
Joined: Fri Mar 15, 2019 7:46 pm
Location: Germany
Full name: Fabian von der Warth

Re: Texel tuning Zurichess quiet like generator

Post by fabianVDW » Fri Aug 09, 2019 4:25 pm

I didn't have success with the dataset, sadly. STC tests went -100 Elo. My tuned values are from ZuriChess dataset which is also definitly very high class (considering every position was played out with sf)
Author of FabChess: https://github.com/fabianvdW/FabChess
A UCI compliant chess engine written in Rust.
fabianvonderwarth@gmail.com

User avatar
xr_a_y
Posts: 663
Joined: Sat Nov 25, 2017 1:28 pm
Location: France

Re: Texel tuning Zurichess quiet like generator

Post by xr_a_y » Fri Aug 09, 2019 5:52 pm

fabianVDW wrote:
Fri Aug 09, 2019 4:25 pm
I didn't have success with the dataset, sadly. STC tests went -100 Elo. My tuned values are from ZuriChess dataset which is also definitly very high class (considering every position was played out with sf)
Yes what I want to do next is playing 3 fast games per position with sf to verify the outcome. Thanks for testing.

fabianVDW
Posts: 45
Joined: Fri Mar 15, 2019 7:46 pm
Location: Germany
Full name: Fabian von der Warth

Re: Texel tuning Zurichess quiet like generator

Post by fabianVDW » Fri Aug 09, 2019 7:11 pm

NP, and if your super dataset with 3sf games per position is finished I am also willing to test again :P

BTW, on your current dataset I have very high Error with FabChess in Tuning. Error is about 0.129 IIRC which would be a residual of 0.36. Very high... On zuri dataset I have Error of 0.058 or something
Author of FabChess: https://github.com/fabianvdW/FabChess
A UCI compliant chess engine written in Rust.
fabianvonderwarth@gmail.com

User avatar
xr_a_y
Posts: 663
Joined: Sat Nov 25, 2017 1:28 pm
Location: France

Re: Texel tuning Zurichess quiet like generator

Post by xr_a_y » Fri Aug 09, 2019 8:32 pm

Have you fit your sigmoid constant to the new data set?

fabianVDW
Posts: 45
Joined: Fri Mar 15, 2019 7:46 pm
Location: Germany
Full name: Fabian von der Warth

Re: Texel tuning Zurichess quiet like generator

Post by fabianVDW » Fri Aug 09, 2019 8:45 pm

Yeah, but k turned out to be around 0.23 which sounded way too low, that's why I set it back up to my standard (0.96). Anyway, I can also try with optimal k again, it was just so low it didn't sound right to me.
Author of FabChess: https://github.com/fabianvdW/FabChess
A UCI compliant chess engine written in Rust.
fabianvonderwarth@gmail.com

jdart
Posts: 3787
Joined: Fri Mar 10, 2006 4:23 am
Location: http://www.arasanchess.org

Re: Texel tuning Zurichess quiet like generator

Post by jdart » Fri Aug 09, 2019 10:58 pm

Don't know if it is useful but I uploaded a couple labelled data sets a few years back:

https://www.arasanchess.org/big3.epd.bz2 (about 5 million positions)

https://www.arasanchess.org/lichess-new-labeled.epd.bz2 (2.5 million positions, I think distinct from big3 - but not sure)

These are both quiet and non-quiet positions. They use the c2 EPD field to indicate the result (1.0 for White win, 0.5 for draw, 0 for Black win). There is also a c1 field which Arasan uses but you can ignore.

--Jon

fabianVDW
Posts: 45
Joined: Fri Mar 15, 2019 7:46 pm
Location: Germany
Full name: Fabian von der Warth

Re: Texel tuning Zurichess quiet like generator

Post by fabianVDW » Sat Aug 10, 2019 2:31 pm

Okay, I tried again with fitted K constant.

CCRL datastet ) k: 0.2375, Error: 0.12605 Residual: 0.355, STC Tests: Cancelled Early: +6 =3 -70, -391.7 Elo

Arasan Big 3) k: 0.5326, Tuned Tempo Bonus was at 300 CP, because Non-Quiet positions => Converted to quiet positions

This conversion was actually troublesome because there were about one thousand illegal positions (where the king could be captured) which had to be sorted out first. (How would you even label these??)

Arasan Big 3 Quiet) k: 0.8796, Error: 0.081 Residual: 0.2846, STC Tests: +301 =356 -341 -13.93 +/- 21.56 Elo

Lichess Quiet( no illegal positions there) ) k: 1.0887, Error: 0.06516 Residual: 0.25526, STC Tests: +244 =261 -203 +20.14 +/- 25.61 So that seems to be a success

I will start LTC regression tests for the tuned params of Lichess Quiet against Version 1.9 and Version 1.9.2 (private version)

Fabi
Author of FabChess: https://github.com/fabianvdW/FabChess
A UCI compliant chess engine written in Rust.
fabianvonderwarth@gmail.com

zenpawn
Posts: 284
Joined: Sat Aug 06, 2016 6:31 pm
Location: United States

Re: Texel tuning Zurichess quiet like generator

Post by zenpawn » Sat Aug 10, 2019 3:40 pm

fabianVDW wrote:
Sat Aug 10, 2019 2:31 pm

Lichess Quiet( no illegal positions there) ) k: 1.0887, Error: 0.06516 Residual: 0.25526, STC Tests: +244 =261 -203 +20.14 +/- 25.61 So that seems to be a success

I will start LTC regression tests for the tuned params of Lichess Quiet against Version 1.9 and Version 1.9.2 (private version)
Is "Lichess Quiet" the second link from Jon above (perhaps with non-quiet positions removed?) or something else?
Erin Dame
Author of RookieMonster

User avatar
xr_a_y
Posts: 663
Joined: Sat Nov 25, 2017 1:28 pm
Location: France

Re: Texel tuning Zurichess quiet like generator

Post by xr_a_y » Sat Aug 10, 2019 3:49 pm

Ok I found another bug ... sorry ...

Can I send you the new edp by mail ?

Post Reply