Is EMAN the future number one?

[MikeB]

I just kicked off a 1000 game match ,using 500 hand selected positions, selected for sharpness as well as some selected for overall chess opening theory popularity. In the one corner , Eman with all learning and experience files turned off for this round ( don't worry , at some point we'll have another match with learning fully kicked in and enabled after a bit of training. The training will actually take more time than the playing to give it full justice.) In the other corner , the soon to be released Honey-XI-r2 with Profound Analysis, testing at 15 Elo higher than Honey-XI-r1 in self play ( meaning rating difference is exaggerated - to what degree ? who knows, it is impossible for me to say) .

First results will be reported after the first hour which would be in about 50 minutes ( around 1:20 AM my time) from this post and then every 1/2 hour after that. I expect this match to be relatively close and after some days of rigorous training , Eman , in all probability and if it works as designed and if I trained it properly ( perhaps a big 'if' there) , will dominate.

results :
https://www.dropbox.com/s/po89jovub1qcf ... a.txt?dl=0

pgn:
https://www.dropbox.com/s/m3eqawvy5a9m1 ... 0.pgn?dl=0

Note - both files will be blank initially., you will some games being posted to the pgn file before the first results are reported - the reports are all script based, two scripts run off the Mac and I have one script running off Windows - they meet in a dropbox folder. ( three scripts, maybe 50-60 lines of code in total), The games are being run on Windows. Only 50 threads are used for the games. This keeps 14 threads in reserve for me to do other things and it also boosts the nps a little bit. If anyone is curious about the scripts , let me know. Cutechess is managing the match.

[Glarean]

I just kicked off a 1000 game match,

Interesting experiment that I will follow closely. Especially because of the many threads used and the rather long TC these games will even be above average in chess terms.
Yes, the experience file at Eman during the matches can be a success factor.
But if the default opening choice is very wide, the influence shouldn't be too big, I guess. In my experience, "Experience" is extremely helpful, especially in connection with analyses (Forward-Backward-Analysis)

I personally delete this "experience" after a few games as a precaution, so that it remains fair, because most eman-match opponents have not implied this learning.
Which version of Eman did you use? Unfortunately this is not in your PGN file...

using 500 hand selected positions, selected for sharpness as well as some selected for overall chess opening theory popularity.

I've examined your 176 games more closely. Here are some of the anomalies:
- There is an above-average number of White victories.
Normal ratio in computer games: 2:1 in favor of White
In your match: 3:1
- In the black victories, strangely enough, there are twice as many Honey victories as Eman victories.
A balanced use of the opening handicap is extremely important in such experiments...
Ok, there were "only" 176 games, of course, statistical swings are possible.

In the one corner , Eman with all learning and experience files turned off for this round ( don't worry , at some point we'll have another match with learning fully kicked in

I am curious to see whether the "Experience" will really have a demonstrable impact.

and enabled after a bit of training.

Training?

In the other corner , the soon to be released Honey-XI-r2 with Profound Analysis, testing at 15 Elo higher than Honey-XI-r1

We are waiting with interest for your second version. 15 Elo ist not so little.
Anyway, I have here a number of test positions that will show me the difference between r1 and r2 quite clearly...

PS: The author of Eman will certainly be happy that there are now already two people in this forum who are dealing with his engine...

[AndrewGrant]

PS: The author of Eman will certainly be happy that there are now already two people in this forum who are dealing with his engine...

He won't be happy when one of them rightfully requests the source code.

[corres]

If I say "I used Joe's lawnmower" Joe does not have to verify that I used it. Maybe he did not see me do it, but I admitted to it.
The Eman site says it's a clone. Therefore, the SF group does not have to say "It's a clone."
We already know it because Khalid Omar said so.
Furthermore, every time he does a patch (for example : https://cumnor-chess-server.com/phpbb/v ... .php?t=217) he explains that he has back-patched with the current Stockfish code. That is only possible if it is a clone.
He says its a clone. It is a clone. No puzzling or wondering or throwing up our hands and furrowing our worried brow.
It walks like a duck, it quacks like a duck, it has a label on it that says, "I am a duck."
If you would like to imagine that it's not a duck, nobody will stop you.
But it's still a duck.

I think Eman is not a clone of Strockfish, but a derivative. If Eman is a derivative of Stockfish, the plus knowledge of Eman would be separated in the source of Eman. Without this Khalid can not make patches to Eman.
In general I think it is too great the fuss around Eman. The difference in Elo for Eman is rather minimal. For position analysis the reparation of time management has no effect. Saving of the value of positions is an old trick and it make slower Eman, the bigger the learning file of Eman the smaller the speed of Eman. For an average opening repertoire Eman need such a big learning file what is unusable, etc.

Note
There are some Stockfish derivative what are hyped by the author with big words without real contents.
Stockfish is the true ruler! Some Elo here or there is unimportant!

[Glarean]

the bigger the learning file of Eman the smaller the speed of Eman. For an average opening repertoire Eman need such a big learning file what is unusable

That's also my experience. The learn file and the automatic generation of an Eman book is especially useful when analyzing games. There it can be extremely helpful when examining and playing out positions by saving time.
At the moment I know of no better tool for analyzing than Eman. However, Shredder is also interesting in this aspect, there you can specifically "weight" analyzed positions.

[corres]

...
results :
https://www.dropbox.com/s/po89jovub1qcf ... a.txt?dl=0
pgn:
https://www.dropbox.com/s/m3eqawvy5a9m1 ... 0.pgn?dl=0

You can not know better place than dropbox?
Dropbox is a dirty place.
I do not like if strangers rummage in my HDD.

[corres]

the bigger the learning file of Eman the smaller the speed of Eman. For an average opening repertoire Eman need such a big learning file what is unusable

That's also my experience. The learn file and the automatic generation of an Eman book is especially useful when analyzing games. There it can be extremely helpful when examining and playing out positions by saving time.
At the moment I know of no better tool for analyzing than Eman. However, Shredder is also interesting in this aspect, there you can specifically "weight" analyzed positions.

If you play during your life - let us say 10000 games, how often you meet the same position?
I think it is more better to use a good book or Brainfish and analyze the lacking positions with a good engine, like Stockfish/Brainfish.
A good engine can weight the positions in the book. This behavior is in Chessbase Fritz engines from the 1990' years!

[MikeB]

I just kicked off a 1000 game match ,using 500 hand selected positions, selected for sharpness as well as some selected for overall chess opening theory popularity. In the one corner , Eman with all learning and experience files turned off for this round ( don't worry , at some point we'll have another match with learning fully kicked in and enabled after a bit of training. The training will actually take more time than the playing to give it full justice.) In the other corner , the soon to be released Honey-XI-r2 with Profound Analysis, testing at 15 Elo higher than Honey-XI-r1 in self play ( meaning rating difference is exaggerated - to what degree ? who knows, it is impossible for me to say) .

First results will be reported after the first hour which would be in about 50 minutes ( around 1:20 AM my time) from this post and then every 1/2 hour after that. I expect this match to be relatively close and after some days of rigorous training , Eman , in all probability and if it works as designed and if I trained it properly ( perhaps a big 'if' there) , will dominate.

results :
https://www.dropbox.com/s/po89jovub1qcf ... a.txt?dl=0

pgn:
https://www.dropbox.com/s/m3eqawvy5a9m1 ... 0.pgn?dl=0

Note - both files will be blank initially., you will some games being posted to the pgn file before the first results are reported - the reports are all script based, two scripts run off the Mac and I have one script running off Windows - they meet in a dropbox folder. ( three scripts, maybe 50-60 lines of code in total), The games are being run on Windows. Only 50 threads are used for the games. This keeps 14 threads in reserve for me to do other things and it also boosts the nps a little bit. If anyone is curious about the scripts , let me know. Cutechess is managing the match.

Final results:

Code: Select all

Current date : time (EDST)
Date: 06/10/20 : 07:56:21
Rank Name         Rating   Δ     +    -     #     Σ    Σ%     W    L    D   W%    =%   OppR 
---------------------------------------------------------------------------------------------------------
   1 Honey-XI-r2   3556   0.0   10   10  1000  517.5  51.7  129   94  777  12.9  77.7  3544 
   2 Eman          3544  12.0   10   10  1000  482.5  48.2   94  129  777   9.4  77.7  3556 
---------------------------------------------------------------------------------------------------------

This is a wrap for the release of Honey XI-r2, look for an announcement here or you may check Github later today , afterwards, in a few days I will rerun this 1000 game match with Eman using its experience.bin file. I should have all the binaries created by end of today, Github will be updated with release source a few hours earlier.

[corres]

...
Only 50 threads are used for the games. This keeps 14 threads in reserve for me to do other things and it also boosts the nps a little bit. If anyone is curious about the scripts , let me know. Cutechess is managing the match.

Have you any experience about Elo difference between Threadripper 16 physical cores and Threadripper 32 physical cores if clock speed is about the same? Maybe from a table of tests result.

[Glarean]

I think it is more better to use a good book or Brainfish and analyze the lacking positions with a good engine, like Stockfish/Brainfish.
A good engine can weight the positions in the book. This behavior is in Chessbase Fritz engines from the 1990' years!

CB books are not so helpful when it comes to deep analyses and position evaluations. For interactive analysis, statistics are less important. I prefer Shredder or Eman. In the past I often worked with IDeA (aquarium).