stockfish 10 vs. Mephisto III S Glasgow

[jp]

If we're doing strange fixed nodes testing, then why not put Leela vs Mephisto at 1000 nodes. Surely that will be close. (I predict Leela will score 100-0 in a hundred game match) Obviously if your node contains much more calculations your engine will be stronger at fixed node count. Instead it should be tested at fixed (be it slow) hardware.

Leela would probably smash Mephisto on a single node search.

Either way, I tried calculating how many NPS could get on a CPU from 1984.

How many nodes could Leela get on that hardware? It looks like you'd need to give it a long time per move even to get a 0 node "search".

[Ferdy]

There should be 2 World_Champion
1. default World_Champion
2. var World_Champion

Got it - thanks for pointing this out.

Source updated:

https://github.com/MichaelB7/Stockfish/tree/McCain

Thanks for the update, now it works with cutechess.



I name this settings "McCain X2a Novice"


And another one.



I name this "McCain X2a 1400"

So Levels has no effect if the UCI_LimitStrength is enabled. Am I right?

BTW McCain X2 UCI_ELO and Levels options are visible in Winboard, but have not tried playing games there.

[MikeB]

There should be 2 World_Champion
1. default World_Champion
2. var World_Champion

Got it - thanks for pointing this out.

Source updated:

https://github.com/MichaelB7/Stockfish/tree/McCain

Thanks for the update, now it works with cutechess.



I name this settings "McCain X2a Novice"


And another one.



I name this "McCain X2a 1400"

So Levels has no effect if the UCI_LimitStrength is enabled. Am I right?

BTW McCain X2 UCI_ELO and Levels options are visible in Winboard, but have not tried playing games there.

1. Correct UCI_LimitStrength must be checked to enable weaker settings ( either by level or Elo)
2. To play by Elo rating, level must be set to"None" with UCI_LimitStrength checked, the level settings have predefined Elo settings associated with each level that are not user adjustable. There is roughly a 100 Elo point spreads between each level. More than what I would have wanted, but there appears to be limitation on the number of dropdown items that are available in xBoard, that's probably also true in other GUIs. I used the maximum number of dropdowns that were visible in xBoard.

[Ferdy]

There should be 2 World_Champion
1. default World_Champion
2. var World_Champion

Got it - thanks for pointing this out.

Source updated:

https://github.com/MichaelB7/Stockfish/tree/McCain

Thanks for the update, now it works with cutechess.



I name this settings "McCain X2a Novice"


And another one.



I name this "McCain X2a 1400"

So Levels has no effect if the UCI_LimitStrength is enabled. Am I right?

BTW McCain X2 UCI_ELO and Levels options are visible in Winboard, but have not tried playing games there.

1. Correct UCI_LimitStrength must be checked to enable weaker settings ( either by level or Elo)

This seemed complicated at first. So If I use Levels Novice, I still have to check UCI_LimitStrength? If so since UCI_LimitStrength is checked does the UCI_Elo matters?
In the image that follows which one will be followed by engine, 2800 or Novice? I know the answer after reading number 2 below.

2. To play by Elo rating, level must be set to"None" with UCI_LimitStrength checked, the level settings have predefined Elo settings associated with each level that are not user adjustable.

All right this one is clear.

I would suggest though to separate functionality of UCI_LimitStrength / UCI_Elo pair and Levels option. The pair UCI_LimitStrength / UCI_Elo is defined in uci protocol. It would be convenient to the user to select Levels alone say Novice etc, without tinkering the UCI_LimitStrength. However if the user wants UCI_Elo, don't let Levels affect the game play, just make the Levels to None automatically without the need for the user to set it to None.

There is roughly a 100 Elo point spreads between each level. More than what I would have wanted, but there appears to be limitation on the number of dropdown items that are available in xBoard, that's probably also true in other GUIs. I used the maximum number of dropdowns that were visible in xBoard.

I have tried dropdown in Winboard up to 45 items, don't know if that is the max.

[Ferdy]

There should be 2 World_Champion
1. default World_Champion
2. var World_Champion

Got it - thanks for pointing this out.

Source updated:

https://github.com/MichaelB7/Stockfish/tree/McCain

Sorry Mike but there is another problem but related to combo type for other options. Perhaps you have to review all your combos.
This time it is related to Analysis Contempt and Book File 1, perhaps there are others.

Code: Select all

Warning: Invalid UCI option from McCain X2a 1400: option name Analysis Contempt type combo default var Off var White var Black var Both
Warning: Invalid UCI option from McCain X2a 1400: option name Book File 1 type combo default var None var Cerebellum var Champions var Alekhine var Anand var Botvinnik var Capablanca var Carlsen var Fischer var Karpov var Kasparov var Kortschnoi var Kramink var Lasker var Petrosian var Tal

[Uri Blass]

35 years of software development


...







Or is it 35 years of hardware development ?

It is 35 years of both.
The new software play chess better with the same hardware and the new hardware play chess better with the same software.
Maybe stockfish is not the best with the oldest hardware that it can use(do not know) but it is not optimized for hardware that nobody use today.

[Raphexon]

If we're doing strange fixed nodes testing, then why not put Leela vs Mephisto at 1000 nodes. Surely that will be close. (I predict Leela will score 100-0 in a hundred game match) Obviously if your node contains much more calculations your engine will be stronger at fixed node count. Instead it should be tested at fixed (be it slow) hardware.

Leela would probably smash Mephisto on a single node search.

Either way, I tried calculating how many NPS could get on a CPU from 1984.

How many nodes could Leela get on that hardware? It looks like you'd need to give it a long time per move even to get a 0 node "search".

With Leela's network being 52.5MB and her client being 8 MB, it'd be already a challenge to load her up.
I think you'd be talking about maybe at most a node per day. But realistically it wouldn't work at all.

[koedem]

If we're doing strange fixed nodes testing, then why not put Leela vs Mephisto at 1000 nodes. Surely that will be close. (I predict Leela will score 100-0 in a hundred game match) Obviously if your node contains much more calculations your engine will be stronger at fixed node count. Instead it should be tested at fixed (be it slow) hardware.

Leela would probably smash Mephisto on a single node search.

Either way, I tried calculating how many NPS could get on a CPU from 1984.

How many nodes could Leela get on that hardware? It looks like you'd need to give it a long time per move even to get a 0 node "search".

With Leela's network being 52.5MB and her client being 8 MB, it'd be already a challenge to load her up.
I think you'd be talking about maybe at most a node per day. But realistically it wouldn't work at all.

That was the point. That nodes are not a good comparison. And that instead one should test on (possibly simulated) equal hardware.

[Raphexon]

It does not ?
It beated Stockfish.

With a software from 1984,
That’s 35 years old.

If Mephisto III has no chess understanding when using 1-3 NPS on 8 bit cpu or 3-5 on 16 bit cpu,
How did it get the championship title against stupid opponents that did 1500 or more NPS ?
Factor 500 or 300 (depending which hardware) more NPS .
Where the search trees of the brute force opponents so inefficient?
Why did the other programs needed 300-500 times more nps to reach similar playing strength?
Obviously all these calculated positions did not help for the situation and were waste of time.

SF beats Mephisto on similar hardware.
Stockfish 6 gets 500-100 nps on an intel 80486 with 4.3 MIPS.
Mephisto uses a 2.1 MIP CPU, so you could have just given Stockfish half time odds.
With 40/120 time control for Mephisto 22500 nodes per move would have been fair for Stockfish.

And what about more nps makes a program intrinsically less efficient?
Mephisto uses a computationally more expensive function, therefore it has lower nps.
Other programs use a function that computationally less expensive, therefore they have higher nps.

If Mephisto is so good, then how come it only gets so few nps compared to other programs on the same hardware?
Must be insanely inefficient code.

[Raphexon]

If we're doing strange fixed nodes testing, then why not put Leela vs Mephisto at 1000 nodes. Surely that will be close. (I predict Leela will score 100-0 in a hundred game match) Obviously if your node contains much more calculations your engine will be stronger at fixed node count. Instead it should be tested at fixed (be it slow) hardware.

Leela would probably smash Mephisto on a single node search.

Either way, I tried calculating how many NPS could get on a CPU from 1984.

How many nodes could Leela get on that hardware? It looks like you'd need to give it a long time per move even to get a 0 node "search".

With Leela's network being 52.5MB and her client being 8 MB, it'd be already a challenge to load her up.
I think you'd be talking about maybe at most a node per day. But realistically it wouldn't work at all.

That was the point. That nodes are not a good comparison. And that instead one should test on (possibly simulated) equal hardware.

It's possible with Stockfish 6, as I've already showed.
SF6 on 80486 gets 500-1000 NPS, if you give it half-time odds vs Mephisto it's a fair matchup.
And SF at 250 nps would easily dominate Mephisto.