Page 2 of 4
Re: Similarity tester - 2nd generation - BETA
Posted: Sun Aug 11, 2019 4:42 pm
by Rebel
I am moving into NN engines, Lco runs fine. I am now trying Allie, it runs ok from the command line, with simex I get:
C:\Users\Gebruiker\Desktop\simex-nn>mea.exe --engine lc0\allie_04.exe --name Allie_GTX_1060 --hash 64 --threads 1 --protocol uci --epd epd\simex.epd --movetime 100 --log
epd 1 / 8238
Traceback (most recent call last):
File "mea.py", line 980, in <module>
File "mea.py", line 936, in main
File "mea.py", line 169, in run
File "mea.py", line 384, in run_uci_engine
File "site-packages\chess\__init__.py", line 517, in from_uci
ValueError: 'mo' is not in list
[5032] Failed to execute script mea
Any idea?
Re: Similarity tester - 2nd generation - BETA
Posted: Sun Aug 11, 2019 6:09 pm
by Ferdy
Rebel wrote: ↑Sun Aug 11, 2019 4:36 pm
Ferdy wrote: ↑Sun Aug 11, 2019 1:56 pm
Downloaded that file an hour ago, no changes.
Check the file
dendogram.csv, should be present.
I missed it, got it now. If you do an update, could you name it dendrogram.csv?
Re: Similarity tester - 2nd generation - BETA
Posted: Sun Aug 11, 2019 6:27 pm
by Ferdy
Rebel wrote: ↑Sun Aug 11, 2019 4:42 pm
I am moving into NN engines, Lco runs fine. I am now trying Allie, it runs ok from the command line, with simex I get:
C:\Users\Gebruiker\Desktop\simex-nn>mea.exe --engine lc0\allie_04.exe --name Allie_GTX_1060 --hash 64 --threads 1 --protocol uci --epd epd\simex.epd --movetime 100 --log
epd 1 / 8238
Traceback (most recent call last):
File "mea.py", line 980, in <module>
File "mea.py", line 936, in main
File "mea.py", line 169, in run
File "mea.py", line 384, in run_uci_engine
File "site-packages\chess\__init__.py", line 517, in from_uci
ValueError: '
mo' is not in list
[5032] Failed to execute script mea
Any idea?
This is about converting the bestmove of engine from uci move to san move format. The engine has finished searching and it sent its bestmove. If you have the log, lets see that bestmove.
line 384 of the mea.py:
https://github.com/fsmosca/Multiple-mov ... ea.py#L384
Re: Similarity tester - 2nd generation - BETA
Posted: Sun Aug 11, 2019 7:59 pm
by Rebel
Ferdy wrote: ↑Sun Aug 11, 2019 6:27 pm
Rebel wrote: ↑Sun Aug 11, 2019 4:42 pm
I am moving into NN engines, Lco runs fine. I am now trying Allie, it runs ok from the command line, with simex I get:
C:\Users\Gebruiker\Desktop\simex-nn>mea.exe --engine lc0\allie_04.exe --name Allie_GTX_1060 --hash 64 --threads 1 --protocol uci --epd epd\simex.epd --movetime 100 --log
epd 1 / 8238
Traceback (most recent call last):
File "mea.py", line 980, in <module>
File "mea.py", line 936, in main
File "mea.py", line 169, in run
File "mea.py", line 384, in run_uci_engine
File "site-packages\chess\__init__.py", line 517, in from_uci
ValueError: '
mo' is not in list
[5032] Failed to execute script mea
Any idea?
This is about converting the bestmove of engine from uci move to san move format. The engine has finished searching and it sent its bestmove. If you have the log, lets see that bestmove.
line 384 of the mea.py:
https://github.com/fsmosca/Multiple-mov ... ea.py#L384
Code: Select all
[ 2019-08-11 16:29:10,695 - INFO ] Pos 1
[ 2019-08-11 16:29:10,695 - INFO ] EPD: 3q1rk1/3n1ppp/1p1p1b2/1R1P4/8/r1pNP1P1/P1Q2PBP/5RK1 w - - c0 "Rb3=1";
[ 2019-08-11 16:29:10,695 - INFO ] id None
[ 2019-08-11 16:29:10,695 - INFO ] FEN: 3q1rk1/3n1ppp/1p1p1b2/1R1P4/8/r1pNP1P1/P1Q2PBP/5RK1 w - - 0 1
[ 2019-08-11 16:29:10,695 - INFO ] Solutions: Rb3=1
[ 2019-08-11 16:29:10,695 - DEBUG ] >> ucinewgame
[ 2019-08-11 16:29:10,695 - DEBUG ] >> position fen 3q1rk1/3n1ppp/1p1p1b2/1R1P4/8/r1pNP1P1/P1Q2PBP/5RK1 w - - 0 1
[ 2019-08-11 16:29:10,695 - DEBUG ] >> go movetime 100
[ 2019-08-11 16:29:13,394 - DEBUG ] << No more time and no bestmove forced=f!
[ 2019-08-11 16:29:13,409 - INFO ] elapsed(ms) since go: 2724
Indeed, no bestmove.
Re: Similarity tester - 2nd generation - BETA
Posted: Mon Aug 12, 2019 4:41 pm
by Rebel
Ferdy wrote: ↑Sun Aug 11, 2019 6:09 pm
Rebel wrote: ↑Sun Aug 11, 2019 4:36 pm
Ferdy wrote: ↑Sun Aug 11, 2019 1:56 pm
Downloaded that file an hour ago, no changes.
Check the file
dendogram.csv, should be present.
I missed it, got it now. If you do an update, could you name it dendrogram.csv?
Yeah.
Re: Similarity tester - 2nd generation - BETA
Posted: Mon Aug 12, 2019 5:25 pm
by Ferdy
Can simex able to output like below?
Code: Select all
sim version 3
------ Stockfish_10 ------
52.02 Ethereal_11.25
51.81 Laser_1.7
51.52 RofChade_2.1
50.06 Gull_3
50.02 Defenchess_2.1
49.51 Andscacs_0.93
49.14 Wasp_3.60
48.93 Schooner_2.0.34
47.73 Booot_6.3.1
47.50 Texel_1.07
47.04 Senpai_2.0
46.98 Vajolet2_2.7
46.50 Rodent_0.277
46.30 RubiChess_1.5
46.25 Demolito_2019-07-15
45.87 Deuterium_v2019.1.36.50
45.64 Arasan_21.3
45.27 Fire_7.1
44.16 Godel_6.0
44.04 Hakkapeliitta_3.0
42.95 Pedone_1.9
42.57 Amoeba_3.0
42.43 Cheng_4.39
41.15 Winter_0.6
40.31 marvin_3.3.1
I am trying to observe figures and dendrogram as there are some methods to create it, at the moment this is ward. Dendrogram tends to get weird as data increases. But it is fun to see the clustering. One use for engine authors is to select sparring partners for their engine that is on a different clusters, to be able to adapt to different style. So I am planning of ploting the dendrogram only for similar strength engine, perhaps +/- 100 rating points.
Current plot in github at the moment, all are in.
Re: Similarity tester - 2nd generation - BETA
Posted: Mon Aug 12, 2019 6:56 pm
by Rebel
Tested Lc0 vs Allie, same network files, with good old SIM03
http://rebel13.nl/html/nn-500ms.html
http://rebel13.nl/html/nn-1000ms.html
I am not surprised.
Re: Similarity tester - 2nd generation - BETA
Posted: Mon Aug 12, 2019 8:27 pm
by Ferdy
Similarity Dendrogram v0.3 is released at
https://github.com/fsmosca/Similarity-D ... m/releases
Exe file is big around 50mb due to dependent modules such as scipy, numpy and matplotlib.
Command line:
Code: Select all
similaritydendrogram.exe --input dendrogram.csv --output sim.png
Help
Code: Select all
usage: similaritydendrogram.exe [-h] --input INPUT [--output OUTPUT]
[--method METHOD] [--figx FIGX] [--figy FIGY]
[--plotx PLOTX] [--dpi DPI] [--log]
Read engine similarity matrix and output dendrogram.
optional arguments:
-h, --help show this help message and exit
--input INPUT input similarity matrix file
--output OUTPUT output dendrogram image file,
default=similarity_dendrogram.png
--method METHOD input distance method [ward, complete, single, average,
weighted, centroid, median], default=ward
--figx FIGX input figure x value length, default=10
--figy FIGY input figure y value length, default=9
--plotx PLOTX input plot x value length, default=None
--dpi DPI input image dpi, default=300
--log Records program logging
Similarity Dendrogram v0.3
Re: Similarity tester - 2nd generation - BETA
Posted: Mon Aug 12, 2019 11:00 pm
by Rebel
Works great!
Can I include it in the SIMEX download?
Re: Similarity tester - 2nd generation - BETA
Posted: Mon Aug 12, 2019 11:13 pm
by Ferdy
Rebel wrote: ↑Mon Aug 12, 2019 11:00 pm
Works great!
Can I include it in the SIMEX download?
Yes of course.