stockfish 10 vs. Mephisto III S Glasgow

[Raphexon]

I don't think you understand the central facts in the theory of chess programming:

If you measure the strength of Elos starting from 1000 nodes /move and then
always doubles the nodes/move, then the playing strength of Stockfish after 20 doubles increases to about 3500 ELO
while the playing strength of Mephisto 3 converges against a value somewhere between 1700-1900 ELO

Mephisto 3 is a trivial program that today can be programmed by the most talented programmers on a weekend.

1600 vs 800 nodes per move, SF gets around 165 ELO.
3200 vs 1600 nodes per move nets around 185 ELO.
6400 vs 3200 nodes per move nets around 215 ELO.
12800 vs 6400 nodes per move nets around 260 ELO.
25600 vs 12800 nodes per move nets around 220 ELO.

The strength at which SF gets for every doubling at extremely low node counts is staggering.
I have the PGN files.

http://www.mediafire.com/file/qy73sq9zz ... 0.pgn/file
http://www.mediafire.com/file/i3aagu8ok ... 0.pgn/file
http://www.mediafire.com/file/2ns0uowjg ... 0.pgn/file
http://www.mediafire.com/file/43mb5bj0y ... 0.pgn/file

Also OP is concluding things with a single sample, it'd be funny if 1000 node SF wins the rest of the games.

[mclane]

I don't think you understand the central facts in the theory of chess programming:

If you measure the strength of Elos starting from 1000 nodes /move and then
always doubles the nodes/move, then the playing strength of Stockfish after 20 doubles increases to about 3500 ELO
while the playing strength of Mephisto 3 converges against a value somewhere between 1700-1900 ELO

Mephisto 3 is a trivial program that today can be programmed by the most talented programmers on a weekend.

Why do you want to increase the strength of Stockfish ?
When i have a car that uses 60 PS and it drives me convenient from a to b, why do I need a car engine that is doing 200 PS ?

My point is that using millions and millions of NPS and huge search trees makes no sense for chess.
It is not needed.

Why do your “most talented programmers” who can “program Mephisto III on a Weekend” to create chess engines that waste energy and resources to produce chess engines that have no understanding of chess and instead brutally use hardware power and resources to outcompute the situation without understanding it ?

Wouldn’t it be more convenient instead of programming Stockfish 11 or 12 to buy a new pc that is twice or thrice the speed of the old pc ?
That’s also increasing the elo.
And it is as senseless as continuing from stockfish 10 to a 11.

Instead of telling us that this is intelligent just buy a faster computer. That’s in the same way intelligent .

[mclane]

It does not ?
It beated Stockfish.

With a software from 1984,
That’s 35 years old.

[mclane]

It does not ?
It beated Stockfish.

With a software from 1984,
That’s 35 years old.

If Mephisto III has no chess understanding when using 1-3 NPS on 8 bit cpu or 3-5 on 16 bit cpu,
How did it get the championship title against stupid opponents that did 1500 or more NPS ?
Factor 500 or 300 (depending which hardware) more NPS .
Where the search trees of the brute force opponents so inefficient?
Why did the other programs needed 300-500 times more nps to reach similar playing strength?
Obviously all these calculated positions did not help for the situation and were waste of time.

[Ferdy]

Scroll down on the link below to see the binaries available for download - all flavors available - windows, Linux and macOS. A huge callout to Dann Corbit, Lucas Monge and John Stanback for providing the binaries.

https://github.com/MichaelB7/Stockfish/releases/tag/X2

Tried to setup McCain X2 in Cutechess GUI to setup novice and play some games with other engines at elo 1400 together with CDrill 1400, but UCI_Elo and Levels are missing.



I checked the console uci reply and got this.

Code: Select all

option name Levels type combo default var World_Champion var Super_GM var GM  var Deep_Thought var SIM var Cray_Blitz var IM var Master var Expert var Class_A var Class_B var Class_C var Class_D var Boris var Novice var None
option name UCI_ELO type spin default 1300 min 1300 max 2850

The first line is wrong, correct is:

Code: Select all

option name Levels type combo default World_Champion var World_Champion var Super_GM ...

Remove the var between default and World_Champion.

The second line is a bit wrong perhaps. Correct is:

Code: Select all

option name UCI_Elo type spin default 1300 min 1300 max 2850

Use lowercase l and o in Elo.

[koedem]

If we're doing strange fixed nodes testing, then why not put Leela vs Mephisto at 1000 nodes. Surely that will be close. (I predict Leela will score 100-0 in a hundred game match) Obviously if your node contains much more calculations your engine will be stronger at fixed node count. Instead it should be tested at fixed (be it slow) hardware.

Not sure why my link to the experiment got ignored, but on same (old) hardware, Fritz Bahrain vs Houdini 6 (i.e. advantage Fritz because it is used to the hardware whereas Houdini is not), Fritz got smashed 0.5-99.5. Some pretty darn impressive improvement in the last 15 years only.

[MikeB]

Scroll down on the link below to see the binaries available for download - all flavors available - windows, Linux and macOS. A huge callout to Dann Corbit, Lucas Monge and John Stanback for providing the binaries.

https://github.com/MichaelB7/Stockfish/releases/tag/X2

Tried to setup McCain X2 in Cutechess GUI to setup novice and play some games with other engines at elo 1400 together with CDrill 1400, but UCI_Elo and Levels are missing.



I checked the console uci reply and got this.

Code: Select all

option name Levels type combo default var World_Champion var Super_GM var GM  var Deep_Thought var SIM var Cray_Blitz var IM var Master var Expert var Class_A var Class_B var Class_C var Class_D var Boris var Novice var None
option name UCI_ELO type spin default 1300 min 1300 max 2850

The first line is wrong, correct is:

Code: Select all

option name Levels type combo default World_Champion var World_Champion var Super_GM ...

Remove the var between default and World_Champion.

The second line is a bit wrong perhaps. Correct is:

Code: Select all

option name UCI_Elo type spin default 1300 min 1300 max 2850

Use lowercase l and o in Elo.

Nice catches Ferdy - thank you. I only use xBoard on the Mac and there it worked fine - can you try it with Winboard? I know it works fine with with Arena and Lucas Chess as well, or so I was told anyway.

Sources were updated with you suggestions:
https://github.com/MichaelB7/Stockfish/tree/McCain

[Ferdy]

Scroll down on the link below to see the binaries available for download - all flavors available - windows, Linux and macOS. A huge callout to Dann Corbit, Lucas Monge and John Stanback for providing the binaries.

https://github.com/MichaelB7/Stockfish/releases/tag/X2

Tried to setup McCain X2 in Cutechess GUI to setup novice and play some games with other engines at elo 1400 together with CDrill 1400, but UCI_Elo and Levels are missing.



I checked the console uci reply and got this.

Code: Select all

option name Levels type combo default var World_Champion var Super_GM var GM  var Deep_Thought var SIM var Cray_Blitz var IM var Master var Expert var Class_A var Class_B var Class_C var Class_D var Boris var Novice var None
option name UCI_ELO type spin default 1300 min 1300 max 2850

The first line is wrong, correct is:

Code: Select all

option name Levels type combo default World_Champion var World_Champion var Super_GM ...

Remove the var between default and World_Champion.

The second line is a bit wrong perhaps. Correct is:

Code: Select all

option name UCI_Elo type spin default 1300 min 1300 max 2850

Use lowercase l and o in Elo.

Nice catches Ferdy - thank you. I only use xBoard on the Mac and there it worked fine - can you try it with Winboard? I know it works fine with with Arena and Lucas Chess as well, or so I was told anyway.

Sources were updated with you suggestions:
https://github.com/MichaelB7/Stockfish/tree/McCain

UCI_Elo is now read by cutechess, but Levels is still wrong.

Code: Select all

option name Levels type combo default World_Champion var Super_GM var GM  var Deep_Thought var SIM var Cray_Blitz var IM var Master var Expert var Class_A var Class_B var Class_C var Class_D var Boris var Novice var None

Correct is:

Code: Select all

option name Levels type combo default World_Champion var World_Champion var Super_GM var GM  var Deep_Thought var SIM var Cray_Blitz var IM var Master var Expert var Class_A var Class_B var Class_C var Class_D var Boris var Novice var None

There should be 2 World_Champion
1. default World_Champion
2. var World_Champion

[MikeB]

There should be 2 World_Champion
1. default World_Champion
2. var World_Champion

Got it - thanks for pointing this out.

Source updated:

https://github.com/MichaelB7/Stockfish/tree/McCain

[Raphexon]

If we're doing strange fixed nodes testing, then why not put Leela vs Mephisto at 1000 nodes. Surely that will be close. (I predict Leela will score 100-0 in a hundred game match) Obviously if your node contains much more calculations your engine will be stronger at fixed node count. Instead it should be tested at fixed (be it slow) hardware.

Not sure why my link to the experiment got ignored, but on same (old) hardware, Fritz Bahrain vs Houdini 6 (i.e. advantage Fritz because it is used to the hardware whereas Houdini is not), Fritz got smashed 0.5-99.5. Some pretty darn impressive improvement in the last 15 years only.

Leela would probably smash Mephisto on a single node search.

Either way, I tried calculating how many NPS could get on a CPU from 1984.
According to wikipedia an i5-2500k gets 83000 MIPS (million instructions per second) at 3.3 ghz, so 20750 on one core.
Stockfish should manage 1.25 MNPS on a single core of an i5-2500K
https://en.wikipedia.org/wiki/Instructions_per_second


That gives us about 60 nps per MIP, top CPUs of 1984 could do 5 MIPS. So about 300 nps.
300 nps is more than enough for Stockfish to decimte Mephisto.
I'd wager 20-30 NPS would be more than enough already.
Not that Stockfish is compatible with hardware that old...

But there have been people who managed to get SF to work on very old hardware including an intel 80386DX 33Mhz (4.3 MIPS according to wikilink) and it did about 500-1000 NPS, so my previous estimation was very conservative.
http://www.talkchess.com/forum3/viewtopic.php?t=63857

And we all know SF would smash Mephisto at 500 nps.

Mephisto 3 Glasgow used a Motorola 68000 at 12 mhz which should be 2 MIPS.
So let Stockfish run on the equivalent of 200 nodes per second and you'd have a fair matchup adjusted for hardware. (Conservative nps estimate)

200 nps should be (more than) enough to beat Mephisto.