Discussion of anything and everything relating to chess playing software and machines.
Moderators: hgm , Rebel , chrisw
elcabesa
Posts: 855Joined: Sun May 23, 2010 1:32 pm
Post by elcabesa Sat Jun 13, 2015 5:24 pm
hi, i tried to run it but i can't get yur beautiful output, where I'm doing mistake?
Code: Select all
C:\Users\elcab_000\Downloads\App_STS_Rating_v3>STS_Rating_v3 -f "STS1-STS15_LAN.
epd" -e "vajolet2 1.45x64 pop.exe" --proto uci -h 256 --getrating
STS Rating v3.0
Intel(R) Core(TM) i5-4210U CPU @ 1.70GHz
Physical cores : 2
System : Windows
Engine: vajolet2 1.45x64 pop.exe
Hash: 256, Threads: 1, MoveTime: 1.0s
Number of positions in STS1-STS15_LAN.epd: 1500
Your bench : 2.546908s
My bench : 2.553400s
Analysis Time to get CCRL 40/4 rating estimate : 199ms
Starting engine vajolet2 1.45x64 pop.exe ...
Vajolet2 1.45
Id: STS(v15.0), Position: 100
Press enter key to exit
C:\Users\elcab_000\Downloads\App_STS_Rating_v3>
elcabesa
Posts: 855Joined: Sun May 23, 2010 1:32 pm
Post by elcabesa Sat Jun 13, 2015 5:28 pm
I'm sorry , and I'm also stupid
I just noticed th STS_Rating.txt file just now........
Canoike
Posts: 125Joined: Tue Jan 17, 2012 8:08 pm
Post by Canoike Mon Sep 14, 2015 5:01 am
What are the specifications for running STS_EEAI_v1.exe or version 3 ? Which OS and what hardware ?
Ferdy
Posts: 4840Joined: Sun Aug 10, 2008 3:15 pmLocation: Philippines
Post by Ferdy Mon Sep 14, 2015 7:26 am
Canoike wrote: What are the specifications for running STS_EEAI_v1.exe or version 3 ? Which OS and what hardware ?
This is developed under windows os. I use wmi to detect couple of hardware specs. I can probably revise this to use python cpu_info module to get hardware info.
What os did you use when you get that error message? If windows what hardware?
Display of CPU info is not that important. I can just bypass this if this is the cause of the trouble.
Canoike
Posts: 125Joined: Tue Jan 17, 2012 8:08 pm
Post by Canoike Mon Sep 14, 2015 4:08 pm
I use Windows XP Professional 64 bits edition. The hardware is a PC with a Intel Core I5.
Ferdy
Posts: 4840Joined: Sun Aug 10, 2008 3:15 pmLocation: Philippines
Post by Ferdy Mon Sep 14, 2015 10:25 pm
Replaced wmi with psutil and cpuinfo to get cpu brand name and number of cores info. Done some error handling that even if reading of cpu info fails the program will continue to run.
This version also uses new sts epd now with san move for winboard engines that outputs move in san notation. Can handle winboard st command with decimal value like st 0.2.
Not done yet with refactoring and error handling improvement, but it is better if you try this one first.
Download:
http://www.mediafire.com/download/k4q6u ... ing_v9.rar
There is a file called RunSTSRating.bat. Inside there is a command line,
Code: Select all
STS_Rating_v9 -f "STS1-STS15_LAN_v2.epd" -e "CDrill 1599.exe" --proto uci -h 128 --getrating
Replace that CDrill 1599.exe with sf6.exe (stockfish 6) for example and make sure sf6.exe is in same directory with this sts_rating_v9.exe file.
Something like this.
Code: Select all
STS_Rating_v9 -f "STS1-STS15_LAN_v2.epd" -e "sf6.exe" --proto uci -h 128 --getrating
Canoike
Posts: 125Joined: Tue Jan 17, 2012 8:08 pm
Post by Canoike Tue Sep 15, 2015 1:46 am
Thank you.
It worked on my computer with this new version.
The CPU has 4 physical cores and not 4 logical cores.
Windows XP Professional 64 bits edition.
It would be nice if your utility works under Linux because the same engine runs faster under this OS. From the fastest to the slowest OS : 1) Linux 2) MacOs 3) guess...
Stockfish latest dev, my compil from Github with mingW :
Code: Select all
STS_Rating_v9 -f "STS1-STS15_LAN_v2.epd" -e "stockfish20150910.exe" --proto uci -h 128 --getrating
Intel(R) Core(TM) i5-3450S CPU @ 2.80GHz
Physical Cores: 1, Logical Cores: 4
Engine: Stockfish 120915 64 POPCNT
Hash: 128, Threads: 1, time/pos: 0.166s
Test duration: 00:04:41
Expected time to finish: 00:04:54
STS rating: 2978
STS ID STS1 STS2 STS3 STS4 STS5 STS6 STS7 STS8 STS9 STS10 STS11 STS12 STS13 STS14 STS15 ALL
NumPos 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 1500
BestCnt 87 64 72 74 71 75 53 68 66 78 65 52 67 62 42 996
Score 891 683 796 787 752 794 573 783 703 820 696 566 747 706 553 10850
Score(%) 89.1 68.3 79.6 78.7 75.2 79.4 57.3 78.3 70.3 82.0 69.6 56.6 74.7 70.6 55.3 72.3
Rating 3724 2798 3301 3261 3105 3292 2308 3243 2887 3408 2856 2277 3083 2900 2219 2978
Komodo 9.2 x64
Code: Select all
STS_Rating_v9 -f "STS1-STS15_LAN_v2.epd" -e "ko
modo-9.2-64bit.exe" --proto uci -h 128 --getrating
Intel(R) Core(TM) i5-3450S CPU @ 2.80GHz
Physical Cores: 1, Logical Cores: 4
Engine: Komodo 9.2 64-bit
Hash: 128, Threads: 1, time/pos: 0.165s
Test duration: 00:04:24
Expected time to finish: 00:04:52
STS rating: 2990
STS ID STS1 STS2 STS3 STS4 STS5 STS6 STS7 STS8 STS9 STS10 STS11 STS12 STS13 STS14 STS15 ALL
NumPos 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 1500
BestCnt 85 67 71 74 74 75 49 66 66 71 66 52 70 63 47 996
Score 874 741 789 770 768 800 561 736 706 778 725 573 764 703 602 10890
Score(%) 87.4 74.1 78.9 77.0 76.8 80.0 56.1 73.6 70.6 77.8 72.5 57.3 76.4 70.3 60.2 72.6
Rating 3648 3056 3270 3185 3177 3319 2255 3034 2900 3221 2985 2308 3159 2887 2437 2990
Ferdy
Posts: 4840Joined: Sun Aug 10, 2008 3:15 pmLocation: Philippines
Post by Ferdy Tue Sep 15, 2015 8:18 am
Canoike wrote:
Intel(R) Core(TM) i5-3450S CPU @ 2.80GHz
When you run the batch file, did you notice the following warning printed in console?
Code: Select all
Warning, psutil failed to determine the number of physical cores
Canoike
Posts: 125Joined: Tue Jan 17, 2012 8:08 pm
Post by Canoike Tue Sep 15, 2015 8:33 pm
When you run the batch file, did you notice the following warning printed in console?
Code: Select all
Warning, psutil failed to determine the number of physical cores[/quote]
No warning of psutil. Just this :
STS_Rating_v9 -f "STS1-STS15_LAN_v2.epd" -e "st
MikeB
Posts: 4889Joined: Thu Mar 09, 2006 6:34 amLocation: Pen Argyl, Pennsylvania
Post by MikeB Wed Sep 16, 2015 4:06 am
Ferdy wrote: Now v3 has basic support for winboard (WB) engines. It does not approximate STS rating for WB engines but only for UCI engines.
For WB engines analysis time can be sent by level command, say you want the engine to analyze at around 200ms/pos, you may use
--mps 40 --tc 0:8 command line options. Only those WB engines that output similar to "move e2e4" - LAN notation are considered, and only those engines that hve support for setboard and level commands. Also the tool has no control on number of threads and hash size of the engines. But for uci engines you can control the hash and the threads by --hash <value> and --threads <value> respectively.
Sample command lines.
Code: Select all
STS_Rating_v3 -f "STS1-STS15_LAN.epd" -e "LambChop_1099.exe -hash 128" --proto wb --mps 300 --tc 1 --log
or
Code: Select all
STS_Rating_v3 -f "STS1-STS15_LAN.epd" -e "C:\Chess\engines\nobook\RomiChess_P3L\RomiChessp3L64.exe" --proto wb --mps 40 --tc 0:8 --log
For uci engines.
Code: Select all
STS_Rating_v3 -f "STS1-STS15_LAN.epd" -e "Stockfish 6.exe" --proto uci -h 128 --getrating
In --getrating the tool will determine the time and the threads.
Usage:
Code: Select all
program -f <epdfile> -e <engname> -t <numthreads> --movetime <timeinms>
Options:
-f or --file <value>, for epd file input
-e or --engine <value>, name of engine
-h or --hash <value>, hash size in MB, default is 32 MB
-t or --threads <value>, for threads, Cores and Max CPUs setting
--movetime <value>, time in ms, default is 1000ms
--log, save engine log
--getrating, calculate CCRL 40/4 rating estimate for uci engines only
--mps <integer value>, moves per session for winboard engines
--tc <integer value in minutes or mm:ss>, timce control for winboard engines
--proto <uci or wb>, protocol the engine supports
Example:
Ex1. Analyze test.epd with 2 threads and 128 MB hash, at 3s/pos using sf 6.exe
STS_Rating -f test.epd -e "sf 6.exe" -h 128 -t 2 --movetime 3000
Tested WB engines.
Code: Select all
Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
Engine: LambChop_1099.exe -hash
Estimated time/pos: 0.200s
Test duration: 00:10:50
Expected time to finish: 00:05:45
Command: level 300 1 0
STS ID STS1 STS2 STS3 STS4 STS5 STS6 STS7 STS8 STS9 STS10 STS11 STS12 STS13 STS14 STS15 ALL
NumPos 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 1500
BestCnt 36 40 44 32 56 46 25 18 23 65 21 34 37 55 18 550
Score 439 478 560 419 625 704 398 344 405 717 351 429 524 659 376 7428
Score(%) 43.9 47.8 56.0 41.9 62.5 70.4 39.8 34.4 40.5 71.7 35.1 42.9 52.4 65.9 37.6 49.5
Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
Engine: Averno081
Estimated time/pos: 0.200s
Test duration: 00:10:28
Expected time to finish: 00:05:45
Command: level 300 1 0
STS ID STS1 STS2 STS3 STS4 STS5 STS6 STS7 STS8 STS9 STS10 STS11 STS12 STS13 STS14 STS15 ALL
NumPos 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 1500
BestCnt 60 41 50 54 67 47 43 30 38 60 38 48 47 50 39 712
Score 668 499 624 638 740 720 543 460 495 682 508 598 571 667 563 8976
Score(%) 66.8 49.9 62.4 63.8 74.0 72.0 54.3 46.0 49.5 68.2 50.8 59.8 57.1 66.7 56.3 59.8
Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
Engine: Myrddin_0.87-64
Estimated time/pos: 0.200s
Test duration: 00:05:00
Expected time to finish: 00:05:45
Command: level 40 0:8 0
STS ID STS1 STS2 STS3 STS4 STS5 STS6 STS7 STS8 STS9 STS10 STS11 STS12 STS13 STS14 STS15 ALL
NumPos 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 1500
BestCnt 52 47 43 44 65 54 40 22 37 58 34 46 64 53 30 689
Score 606 581 551 588 696 747 559 391 508 673 476 590 723 651 480 8820
Score(%) 60.6 58.1 55.1 58.8 69.6 74.7 55.9 39.1 50.8 67.3 47.6 59.0 72.3 65.1 48.0 58.8
Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
Engine: Natwarlal_v0.14
Estimated time/pos: 0.200s
Test duration: 00:10:18
Expected time to finish: 00:05:45
Command: level 300 1 0
STS ID STS1 STS2 STS3 STS4 STS5 STS6 STS7 STS8 STS9 STS10 STS11 STS12 STS13 STS14 STS15 ALL
NumPos 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 1500
BestCnt 54 52 41 44 64 51 42 33 29 54 39 59 66 64 28 720
Score 634 634 536 587 715 736 534 451 446 654 513 681 739 749 477 9086
Score(%) 63.4 63.4 53.6 58.7 71.5 73.6 53.4 45.1 44.6 65.4 51.3 68.1 73.9 74.9 47.7 60.6
Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
Engine: Gerbil_02_x64_ja
Estimated time/pos: 0.200s
Test duration: 00:06:14
Expected time to finish: 00:05:45
Command: level 40 0:8 0
STS ID STS1 STS2 STS3 STS4 STS5 STS6 STS7 STS8 STS9 STS10 STS11 STS12 STS13 STS14 STS15 ALL
NumPos 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 1500
BestCnt 37 30 31 35 58 53 30 26 24 55 31 37 45 49 29 570
Score 482 410 491 457 691 751 412 364 403 633 440 485 565 626 483 7693
Score(%) 48.2 41.0 49.1 45.7 69.1 75.1 41.2 36.4 40.3 63.3 44.0 48.5 56.5 62.6 48.3 51.3
Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
Engine: scorpio-276-64-ja
Estimated time/pos: 0.200s
Test duration: 00:07:54
Expected time to finish: 00:05:45
Command: level 40 0:8 0
STS ID STS1 STS2 STS3 STS4 STS5 STS6 STS7 STS8 STS9 STS10 STS11 STS12 STS13 STS14 STS15 ALL
NumPos 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 1500
BestCnt 73 58 61 67 69 68 54 32 46 67 45 67 59 65 35 866
Score 784 687 739 761 751 849 617 474 592 740 554 764 697 755 608 10372
Score(%) 78.4 68.7 73.9 76.1 75.1 84.9 61.7 47.4 59.2 74.0 55.4 76.4 69.7 75.5 60.8 69.1
Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
Engine: C:\Chess\engines\nobook\RomiChess_P3L\RomiChessp3L64
Estimated time/pos: 0.200s
Test duration: 00:05:47
Expected time to finish: 00:05:45
Command: level 40 0:8 0
STS ID STS1 STS2 STS3 STS4 STS5 STS6 STS7 STS8 STS9 STS10 STS11 STS12 STS13 STS14 STS15 ALL
NumPos 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 1500
BestCnt 51 49 51 45 63 56 46 30 42 67 41 46 51 63 22 723
Score 605 575 612 572 718 763 585 457 517 742 532 583 641 724 407 9033
Score(%) 60.5 57.5 61.2 57.2 71.8 76.3 58.5 45.7 51.7 74.2 53.2 58.3 64.1 72.4 40.7 60.2
Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
Engine: C:\Chess\engines\nobook\Thinker54D\ThinkerInert 5.4D x64 SP
Estimated time/pos: 0.200s
Test duration: 00:05:55
Expected time to finish: 00:05:45
Command: level 40 0:8 0
STS ID STS1 STS2 STS3 STS4 STS5 STS6 STS7 STS8 STS9 STS10 STS11 STS12 STS13 STS14 STS15 ALL
NumPos 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 1500
BestCnt 69 67 70 59 67 59 53 51 55 72 60 60 63 67 43 915
Score 762 751 775 692 763 808 681 662 667 799 705 687 725 766 621 10864
Score(%) 76.2 75.1 77.5 69.2 76.3 80.8 68.1 66.2 66.7 79.9 70.5 68.7 72.5 76.6 62.1 72.4
Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
Engine: C:\Chess\engines\nobook\Satana.2.0.7\Satana.2.0.7.w64bit
Estimated time/pos: 0.200s
Test duration: 00:06:51
Expected time to finish: 00:05:45
Command: level 40 0:8 0
STS ID STS1 STS2 STS3 STS4 STS5 STS6 STS7 STS8 STS9 STS10 STS11 STS12 STS13 STS14 STS15 ALL
NumPos 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 1500
BestCnt 18 28 22 25 34 34 27 18 18 61 14 15 19 34 5 372
Score 334 385 339 322 430 597 355 314 275 676 279 247 287 458 176 5474
Score(%) 33.4 38.5 33.9 32.2 43.0 59.7 35.5 31.4 27.5 67.6 27.9 24.7 28.7 45.8 17.6 36.5
Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
Engine: C:\Chess\engines\nobook\shallow-rev688-win-ja\shallow-rev688-win-ja\Windows\shallow-rev688-64-ja
Estimated time/pos: 0.200s
Test duration: 00:06:52
Expected time to finish: 00:05:45
Command: level 40 0:8 0
STS ID STS1 STS2 STS3 STS4 STS5 STS6 STS7 STS8 STS9 STS10 STS11 STS12 STS13 STS14 STS15 ALL
NumPos 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 1500
BestCnt 50 43 38 55 58 52 41 28 37 54 36 46 59 67 32 696
Score 607 535 525 639 663 723 542 447 504 642 520 574 707 747 517 8892
Score(%) 60.7 53.5 52.5 63.9 66.3 72.3 54.2 44.7 50.4 64.2 52.0 57.4 70.7 74.7 51.7 59.3
Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
Engine: C:\Chess\engines\nobook\Nemeton\Nemeton\Nemeton_1
Estimated time/pos: 0.200s
Test duration: 00:06:17
Expected time to finish: 00:05:45
Command: level 40 0:8 0
STS ID STS1 STS2 STS3 STS4 STS5 STS6 STS7 STS8 STS9 STS10 STS11 STS12 STS13 STS14 STS15 ALL
NumPos 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 1500
BestCnt 39 40 45 42 57 55 33 20 25 63 38 45 55 45 25 627
Score 528 507 576 544 635 763 480 395 380 704 503 551 672 612 458 8308
Score(%) 52.8 50.7 57.6 54.4 63.5 76.3 48.0 39.5 38.0 70.4 50.3 55.1 67.2 61.2 45.8 55.4
Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
Engine: C:\Chess\engines\nobook\Knightx1.92\Knightx192
Estimated time/pos: 0.400s
Test duration: 00:08:52
Expected time to finish: 00:10:45
Command: level 40 0:16 0
STS ID STS1 STS2 STS3 STS4 STS5 STS6 STS7 STS8 STS9 STS10 STS11 STS12 STS13 STS14 STS15 ALL
NumPos 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 1500
BestCnt 47 49 47 46 57 46 40 27 35 60 38 49 55 64 27 687
Score 567 555 600 589 642 709 518 397 475 689 506 580 654 756 452 8689
Score(%) 56.7 55.5 60.0 58.9 64.2 70.9 51.8 39.7 47.5 68.9 50.6 58.0 65.4 75.6 45.2 57.9
Download:
http://www.mediafire.com/download/3xyap ... ing_v3.rar
Very nice Ferdy!
Could you post/send the source so I can compile a mac version?
Best, Mike
