Added a last feature, add comments to the HTML. For instance these long names for NN engines, store your comments in legend.txt.
http://rebel13.nl/html/nn-500ms.html
Final release tomorrow.
Similarity tester - 2nd generation - BETA
Moderators: hgm, Rebel, chrisw
-
- Posts: 6995
- Joined: Thu Aug 18, 2011 12:04 pm
Re: Similarity tester - 2nd generation - BETA
90% of coding is debugging, the other 10% is writing bugs.
-
- Posts: 6995
- Joined: Thu Aug 18, 2011 12:04 pm
Re: Similarity tester - 2nd generation - BETA
90% of coding is debugging, the other 10% is writing bugs.
-
- Posts: 6995
- Joined: Thu Aug 18, 2011 12:04 pm
Re: Similarity tester - 2nd generation - BETA
Did another experiment, run engines at depth=1 only, canceling (the main) search, thus testing mainly the evaluation function for similarity. It isn't perfect since engines may vary how they handle checks in QS, I have also seen engines extend root moves. But in general it will just work.
Example:1 - http://rebel13.nl/html/depth1.html
And now a blast from the past - http://rebel13.nl/html/fruit.html
The original Rybka 1.0 looks okay, but it is known it obfuscates the depth with 2 plies so it actually has used depth=3. The patched version has this fixed and then the similarity with Fruit 2.1 is sky-high.
Example:1 - http://rebel13.nl/html/depth1.html
And now a blast from the past - http://rebel13.nl/html/fruit.html
The original Rybka 1.0 looks okay, but it is known it obfuscates the depth with 2 plies so it actually has used depth=3. The patched version has this fixed and then the similarity with Fruit 2.1 is sky-high.
90% of coding is debugging, the other 10% is writing bugs.
-
- Posts: 6995
- Joined: Thu Aug 18, 2011 12:04 pm
Re: Similarity tester - 2nd generation - BETA
Chris Whittington created SIMEX EPD sets of each 10,000 positions. Each set contains a specific piece distribution. In total 100 sets representing the most common board positions in use.
Distribution list - http://rebel13.nl/html/epd.txt
Example with SIMEX - http://rebel13.nl/html/cw2.html
Download of the 100 EPD sets - http://rebel13.nl/dl.html?file=dl/simex-epd.7z
Extract the download in the EPD folder.
Distribution list - http://rebel13.nl/html/epd.txt
Example with SIMEX - http://rebel13.nl/html/cw2.html
Download of the 100 EPD sets - http://rebel13.nl/dl.html?file=dl/simex-epd.7z
Extract the download in the EPD folder.
90% of coding is debugging, the other 10% is writing bugs.
-
- Posts: 4319
- Joined: Tue Apr 03, 2012 4:28 pm
Re: Similarity tester - 2nd generation - BETA
I might refine these if there’s any demand. At the moment, it selects all possible piece conformations by just the existence of KQRBNPkqrbnp, either pieces of that type for each side exist or they don’t.Rebel wrote: ↑Fri Aug 23, 2019 12:34 pm Chris Whittington created SIMEX EPD sets of each 10,000 positions. Each set contains a specific piece distribution. In total 100 sets representing the most common board positions in use.
Distribution list - http://rebel13.nl/html/epd.txt
Example with SIMEX - http://rebel13.nl/html/cw2.html
Download of the 100 EPD sets - http://rebel13.nl/dl.html?file=dl/simex-epd.7z
Extract the download in the EPD folder.
Might be better to refine also by pawn count so the files would be named on basis
KQRBNP(x)kqrbnp(y) where x and y represent, say, 0 pawns, or 1-3 pawns, or 4-6 pawns, or 7-8 pawns. Four pawn categories. Adds another factor of 16 to the possible combinations.
Then I rank the combinations by frequency and save out the 100 (can be more) combinations most often occurring in games.
-
- Posts: 1142
- Joined: Thu Dec 28, 2017 4:06 pm
- Location: Argentina
Re: Similarity tester - 2nd generation - BETA
Would be interesting to compare leela (with different networks) to all the other usual AB engines
Follow my tournament and some Leela gauntlets live at http://twitch.tv/ccls
-
- Posts: 89
- Joined: Sat Sep 13, 2014 4:12 pm
- Location: Zagreb, Croatia
- Full name: Branko Radovanović
Re: Similarity tester - 2nd generation - BETA
My impression - not sure if that's true or not, so I'd like to see it tested - is that SF10's play is more similar to LC0 than e.g. SF9 was. That would make sense because if LC0's style is the "chess truth", devoid of preconceptions, one would expect AB engines to gradually approach it. And therein lies the chance for top AB engines to compete against NNs: do (almost) the same, only more efficiently.CMCanavessi wrote: ↑Sun Aug 25, 2019 11:12 pm Would be interesting to compare leela (with different networks) to all the other usual AB engines
Also, for the same reason I'd expect AB engines to converge. I'm pretty sure e.g. Komodo and Stockfish are more similar now than they used to be 3 or 4 years ago. Unfortunately, when similarity tests are run, it's almost always one version per engine.
-
- Posts: 4319
- Joined: Tue Apr 03, 2012 4:28 pm
Re: Similarity tester - 2nd generation - BETA
not necessarily, if chess is a draw, or basically a stable game, as people claim. There’s many ways to get to a draw.Branko Radovanovic wrote: ↑Mon Aug 26, 2019 2:11 amMy impression - not sure if that's true or not, so I'd like to see it tested - is that SF10's play is more similar to LC0 than e.g. SF9 was. That would make sense because if LC0's style is the "chess truth", devoid of preconceptions, one would expect AB engines to gradually approach it.CMCanavessi wrote: ↑Sun Aug 25, 2019 11:12 pm Would be interesting to compare leela (with different networks) to all the other usual AB engines
And therein lies the chance for top AB engines to compete against NNs: do (almost) the same, only more efficiently.
Also, for the same reason I'd expect AB engines to converge. I'm pretty sure e.g. Komodo and Stockfish are more similar now than they used to be 3 or 4 years ago. Unfortunately, when similarity tests are run, it's almost always one version per engine.
-
- Posts: 6995
- Joined: Thu Aug 18, 2011 12:04 pm
Re: Similarity tester - 2nd generation - BETA
Did some Lc0 testing - http://rebel13.nl/html/kai.htmlBranko Radovanovic wrote: ↑Mon Aug 26, 2019 2:11 amMy impression - not sure if that's true or not, so I'd like to see it tested - is that SF10's play is more similar to LC0 than e.g. SF9 was. That would make sense because if LC0's style is the "chess truth", devoid of preconceptions, one would expect AB engines to gradually approach it. And therein lies the chance for top AB engines to compete against NNs: do (almost) the same, only more efficiently.CMCanavessi wrote: ↑Sun Aug 25, 2019 11:12 pm Would be interesting to compare leela (with different networks) to all the other usual AB engines
Also, for the same reason I'd expect AB engines to converge. I'm pretty sure e.g. Komodo and Stockfish are more similar now than they used to be 3 or 4 years ago. Unfortunately, when similarity tests are run, it's almost always one version per engine.
SF vs Lc0 very low similarity.
90% of coding is debugging, the other 10% is writing bugs.
-
- Posts: 89
- Joined: Sat Sep 13, 2014 4:12 pm
- Location: Zagreb, Croatia
- Full name: Branko Radovanović
Re: Similarity tester - 2nd generation - BETA
Indeed, and that's not surprising. No real difference between SF8 and SF10 vs NNs - that seems to prove my impression was not correct, though. And, finally, of all AB engines SF is the most similar to Lc0 (while remaining very far nevertheless) - that would again make sense to me.Rebel wrote: ↑Mon Aug 26, 2019 10:53 am Did some Lc0 testing - http://rebel13.nl/html/kai.html
SF vs Lc0 very low similarity.