SIMEX 2.1

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
Rebel
Posts: 4788
Joined: Thu Aug 18, 2011 10:04 am

SIMEX 2.1

Post by Rebel » Sun Sep 01, 2019 3:44 pm

Made an update, changes:

1. New MEA that fixes a speed issue when running under Win10. MEA was extremely slow, is OK now and even runs faster on Win 7 systems.

2. Cosmetic changes to the HTML output that makes it better readable. It will no longer mark versions of the same engine as orange (similarity 60-64%) or red (65% or higher) by checking the first 2 characters of engines.

Example: http://rebel13.nl/html/ccrl.html

41 high rated CCRL engines and no single 60% found!

One sane list.

My congratulations to the current chess programmers of these engines, especially the new kids on the block while so many strong open source engines are available on Github.

Download SIMEX 2.1 - http://rebel13.nl/dl/simex2.7z

Documentation - http://rebel13.nl/misc/simex.html
90% of coding is debugging, the other 10% is writing bugs.

User avatar
Ovyron
Posts: 2823
Joined: Tue Jul 03, 2007 2:30 am

Re: SIMEX 2.1

Post by Ovyron » Mon Sep 02, 2019 8:16 am

Rebel wrote:
Sun Sep 01, 2019 3:44 pm
41 high rated CCRL engines and no single 60% found!
I see similarities that go beyond that on the Stockfish versions, yet they're not colored yellow as the bottom of the page seem to claim they should be?
Great spirits have always encountered violent opposition from mediocre minds.

User avatar
Guenther
Posts: 3109
Joined: Wed Oct 01, 2008 4:33 am
Location: Regensburg, Germany
Full name: Guenther Simon
Contact:

Re: SIMEX 2.1

Post by Guenther » Mon Sep 02, 2019 9:56 am

Rebel wrote:
Sun Sep 01, 2019 3:44 pm
Made an update, changes:

1. New MEA that fixes a speed issue when running under Win10. MEA was extremely slow, is OK now and even runs faster on Win 7 systems.

2. Cosmetic changes to the HTML output that makes it better readable. It will no longer mark versions of the same engine as orange (similarity 60-64%) or red (65% or higher) by checking the first 2 characters of engines.

Example: http://rebel13.nl/html/ccrl.html

...
Something seems wrong with the results of Pedone 1.8. I guess it did not obey the search time/depth?
Senpai 1 vs. 2 looks strange too and a few other results also seem systematically too low?
Current foe list count : [101]
http://rwbc-chess.de/chronology.htm

User avatar
Rebel
Posts: 4788
Joined: Thu Aug 18, 2011 10:04 am

Re: SIMEX 2.1

Post by Rebel » Mon Sep 02, 2019 2:58 pm

Guenther wrote:
Mon Sep 02, 2019 9:56 am
Rebel wrote:
Sun Sep 01, 2019 3:44 pm
Made an update, changes:

1. New MEA that fixes a speed issue when running under Win10. MEA was extremely slow, is OK now and even runs faster on Win 7 systems.

2. Cosmetic changes to the HTML output that makes it better readable. It will no longer mark versions of the same engine as orange (similarity 60-64%) or red (65% or higher) by checking the first 2 characters of engines.

Example: http://rebel13.nl/html/ccrl.html

...
Something seems wrong with the results of Pedone 1.8. I guess it did not obey the search time/depth?
Senpai 1 vs. 2 looks strange too
I double checked both Pedone and Senpai, the (odd looking) numbers are correct. There are 2 ways to check, the log-file of each or even more simple the created epd in the EPD_OUT folder. EPD's should have "acd 1". If not the engine in question ignores the depth=1 setting. For instance Laser and Booot, they shouldn't be in the overview, I noticed too late.
90% of coding is debugging, the other 10% is writing bugs.

User avatar
Rebel
Posts: 4788
Joined: Thu Aug 18, 2011 10:04 am

Re: SIMEX 2.1

Post by Rebel » Mon Sep 02, 2019 3:00 pm

Ovyron wrote:
Mon Sep 02, 2019 8:16 am
Rebel wrote:
Sun Sep 01, 2019 3:44 pm
41 high rated CCRL engines and no single 60% found!
I see similarities that go beyond that on the Stockfish versions, yet they're not colored yellow as the bottom of the page seem to claim they should be?
There are some irregularities, I will post them later.
90% of coding is debugging, the other 10% is writing bugs.

jorose
Posts: 269
Joined: Thu Jan 22, 2015 2:21 pm
Location: Zurich, Switzerland
Full name: Jonathan Rosenthal

Re: SIMEX 2.1

Post by jorose » Mon Sep 02, 2019 3:07 pm

Does Winter already support such a tool? What do I need to add to an engine in order to support a tool like SIMEX?

I'm kind of curious what engines Winter (especially Winter dev) is related to :D

EDIT: I'm trying things out in WINE now. I get the following error at the moment:

Can't recognize 'sort DATASIMEX.DATA /rec 65000 /o DATASIMEX.DATA' as an internal or external command, or batch script.
-Jonathan

chrisw
Posts: 2187
Joined: Tue Apr 03, 2012 2:28 pm

Re: SIMEX 2.1

Post by chrisw » Mon Sep 02, 2019 3:29 pm

Rebel wrote:
Sun Sep 01, 2019 3:44 pm
Made an update, changes:

1. New MEA that fixes a speed issue when running under Win10. MEA was extremely slow, is OK now and even runs faster on Win 7 systems.

2. Cosmetic changes to the HTML output that makes it better readable. It will no longer mark versions of the same engine as orange (similarity 60-64%) or red (65% or higher) by checking the first 2 characters of engines.

Example: http://rebel13.nl/html/ccrl.html

41 high rated CCRL engines and no single 60% found!

One sane list.

My congratulations to the current chess programmers of these engines, especially the new kids on the block while so many strong open source engines are available on Github.

Download SIMEX 2.1 - http://rebel13.nl/dl/simex2.7z

Documentation - http://rebel13.nl/misc/simex.html
what is the origin of the testing suite? who composed it, or did it just grow, and when was it composed? Do we know how it was composed?

User avatar
Rebel
Posts: 4788
Joined: Thu Aug 18, 2011 10:04 am

Re: SIMEX 2.1

Post by Rebel » Mon Sep 02, 2019 5:16 pm

jorose wrote:
Mon Sep 02, 2019 3:07 pm
Does Winter already support such a tool? What do I need to add to an engine in order to support a tool like SIMEX?

I'm kind of curious what engines Winter (especially Winter dev) is related to :D

EDIT: I'm trying things out in WINE now. I get the following error at the moment:

Can't recognize 'sort DATASIMEX.DATA /rec 65000 /o DATASIMEX.DATA' as an internal or external command, or batch script.
I tried Winter, did not work. I also suspect MicroSoft sort.exe won't work under WINE, but I am no expert.
90% of coding is debugging, the other 10% is writing bugs.

User avatar
Rebel
Posts: 4788
Joined: Thu Aug 18, 2011 10:04 am

Re: SIMEX 2.1

Post by Rebel » Mon Sep 02, 2019 5:41 pm

chrisw wrote:
Mon Sep 02, 2019 3:29 pm
what is the origin of the testing suite? who composed it, or did it just grow, and when was it composed? Do we know how it was composed?
Don isn't longer among us, but maybe Larry can tell.

My best guess is that the 8238 positions were derived from comp-comp games. If you scroll through the simex epd's in the epd_out folder scores reasonable well fit a -1.00/+1.00 window. Also I tried a dozen position searches with 2.9 million games, no matches except known openings.
90% of coding is debugging, the other 10% is writing bugs.

chrisw
Posts: 2187
Joined: Tue Apr 03, 2012 2:28 pm

Re: SIMEX 2.1

Post by chrisw » Mon Sep 02, 2019 8:49 pm

Rebel wrote:
Mon Sep 02, 2019 5:41 pm
chrisw wrote:
Mon Sep 02, 2019 3:29 pm
what is the origin of the testing suite? who composed it, or did it just grow, and when was it composed? Do we know how it was composed?
Don isn't longer among us, but maybe Larry can tell.

My best guess is that the 8238 positions were derived from comp-comp games. If you scroll through the simex epd's in the epd_out folder scores reasonable well fit a -1.00/+1.00 window. Also I tried a dozen position searches with 2.9 million games, no matches except known openings.
There's quite a skew in the test positions, many have all the pieces on the board. Take a look, histograms of piece count, and piece weight for the epd set.
simex-8352-histogram-piece-counts.png
simex-8352-histogram-piece-weights.png
was it Don who chose them? Why early game positions?

Edit: I should learn to label things correct! 8328 it should say.

Post Reply