SIMEX 2.1

[Rebel]

Made an update, changes:

1. New MEA that fixes a speed issue when running under Win10. MEA was extremely slow, is OK now and even runs faster on Win 7 systems.

2. Cosmetic changes to the HTML output that makes it better readable. It will no longer mark versions of the same engine as orange (similarity 60-64%) or red (65% or higher) by checking the first 2 characters of engines.

Example: http://rebel13.nl/html/ccrl.html

41 high rated CCRL engines and no single 60% found!

One sane list.

My congratulations to the current chess programmers of these engines, especially the new kids on the block while so many strong open source engines are available on Github.

Download SIMEX 2.1 - http://rebel13.nl/dl/simex2.7z

Documentation - http://rebel13.nl/misc/simex.html

[Ovyron]

41 high rated CCRL engines and no single 60% found!

I see similarities that go beyond that on the Stockfish versions, yet they're not colored yellow as the bottom of the page seem to claim they should be?

[Guenther]

Made an update, changes:

1. New MEA that fixes a speed issue when running under Win10. MEA was extremely slow, is OK now and even runs faster on Win 7 systems.

2. Cosmetic changes to the HTML output that makes it better readable. It will no longer mark versions of the same engine as orange (similarity 60-64%) or red (65% or higher) by checking the first 2 characters of engines.

Example: http://rebel13.nl/html/ccrl.html

...

Something seems wrong with the results of Pedone 1.8. I guess it did not obey the search time/depth?
Senpai 1 vs. 2 looks strange too and a few other results also seem systematically too low?

[Rebel]

Made an update, changes:

1. New MEA that fixes a speed issue when running under Win10. MEA was extremely slow, is OK now and even runs faster on Win 7 systems.

2. Cosmetic changes to the HTML output that makes it better readable. It will no longer mark versions of the same engine as orange (similarity 60-64%) or red (65% or higher) by checking the first 2 characters of engines.

Example: http://rebel13.nl/html/ccrl.html

...

Something seems wrong with the results of Pedone 1.8. I guess it did not obey the search time/depth?
Senpai 1 vs. 2 looks strange too

I double checked both Pedone and Senpai, the (odd looking) numbers are correct. There are 2 ways to check, the log-file of each or even more simple the created epd in the EPD_OUT folder. EPD's should have "acd 1". If not the engine in question ignores the depth=1 setting. For instance Laser and Booot, they shouldn't be in the overview, I noticed too late.

[Rebel]

41 high rated CCRL engines and no single 60% found!

I see similarities that go beyond that on the Stockfish versions, yet they're not colored yellow as the bottom of the page seem to claim they should be?

There are some irregularities, I will post them later.

[jorose]

Does Winter already support such a tool? What do I need to add to an engine in order to support a tool like SIMEX?

I'm kind of curious what engines Winter (especially Winter dev) is related to

EDIT: I'm trying things out in WINE now. I get the following error at the moment:

Can't recognize 'sort DATASIMEX.DATA /rec 65000 /o DATASIMEX.DATA' as an internal or external command, or batch script.

[chrisw]

Made an update, changes:

1. New MEA that fixes a speed issue when running under Win10. MEA was extremely slow, is OK now and even runs faster on Win 7 systems.

2. Cosmetic changes to the HTML output that makes it better readable. It will no longer mark versions of the same engine as orange (similarity 60-64%) or red (65% or higher) by checking the first 2 characters of engines.

Example: http://rebel13.nl/html/ccrl.html

41 high rated CCRL engines and no single 60% found!

One sane list.

My congratulations to the current chess programmers of these engines, especially the new kids on the block while so many strong open source engines are available on Github.

Download SIMEX 2.1 - http://rebel13.nl/dl/simex2.7z

Documentation - http://rebel13.nl/misc/simex.html

what is the origin of the testing suite? who composed it, or did it just grow, and when was it composed? Do we know how it was composed?

[Rebel]

Does Winter already support such a tool? What do I need to add to an engine in order to support a tool like SIMEX?

I'm kind of curious what engines Winter (especially Winter dev) is related to

EDIT: I'm trying things out in WINE now. I get the following error at the moment:

Can't recognize 'sort DATASIMEX.DATA /rec 65000 /o DATASIMEX.DATA' as an internal or external command, or batch script.

I tried Winter, did not work. I also suspect MicroSoft sort.exe won't work under WINE, but I am no expert.

[Rebel]

what is the origin of the testing suite? who composed it, or did it just grow, and when was it composed? Do we know how it was composed?

Don isn't longer among us, but maybe Larry can tell.

My best guess is that the 8238 positions were derived from comp-comp games. If you scroll through the simex epd's in the epd_out folder scores reasonable well fit a -1.00/+1.00 window. Also I tried a dozen position searches with 2.9 million games, no matches except known openings.

[chrisw]

what is the origin of the testing suite? who composed it, or did it just grow, and when was it composed? Do we know how it was composed?

Don isn't longer among us, but maybe Larry can tell.

My best guess is that the 8238 positions were derived from comp-comp games. If you scroll through the simex epd's in the epd_out folder scores reasonable well fit a -1.00/+1.00 window. Also I tried a dozen position searches with 2.9 million games, no matches except known openings.

There's quite a skew in the test positions, many have all the pieces on the board. Take a look, histograms of piece count, and piece weight for the epd set.

simex-8352-histogram-piece-counts.png

simex-8352-histogram-piece-weights.png

was it Don who chose them? Why early game positions?

Edit: I should learn to label things correct! 8328 it should say.