How many nodes could Leela get on that hardware? It looks like you'd need to give it a long time per move even to get a 0 node "search".Raphexon wrote: ↑Sat Apr 06, 2019 10:40 pmLeela would probably smash Mephisto on a single node search.koedem wrote: ↑Sat Apr 06, 2019 5:49 pm If we're doing strange fixed nodes testing, then why not put Leela vs Mephisto at 1000 nodes. Surely that will be close. (I predict Leela will score 100-0 in a hundred game match) Obviously if your node contains much more calculations your engine will be stronger at fixed node count. Instead it should be tested at fixed (be it slow) hardware.
Either way, I tried calculating how many NPS could get on a CPU from 1984.
stockfish 10 vs. Mephisto III S Glasgow
Moderators: hgm, Rebel, chrisw
-
- Posts: 1470
- Joined: Mon Apr 23, 2018 7:54 am
Re: stockfish 10 vs. Mephisto III S Glasgow
-
- Posts: 4833
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: stockfish 10 vs. Mephisto III S Glasgow
Thanks for the update, now it works with cutechess.MikeB wrote: ↑Sat Apr 06, 2019 7:45 pmGot it - thanks for pointing this out.
Source updated:
https://github.com/MichaelB7/Stockfish/tree/McCain
I name this settings "McCain X2a Novice"
And another one.
I name this "McCain X2a 1400"
So Levels has no effect if the UCI_LimitStrength is enabled. Am I right?
BTW McCain X2 UCI_ELO and Levels options are visible in Winboard, but have not tried playing games there.
-
- Posts: 4889
- Joined: Thu Mar 09, 2006 6:34 am
- Location: Pen Argyl, Pennsylvania
Re: stockfish 10 vs. Mephisto III S Glasgow
1. Correct UCI_LimitStrength must be checked to enable weaker settings ( either by level or Elo)Ferdy wrote: ↑Sun Apr 07, 2019 3:07 amThanks for the update, now it works with cutechess.MikeB wrote: ↑Sat Apr 06, 2019 7:45 pmGot it - thanks for pointing this out.
Source updated:
https://github.com/MichaelB7/Stockfish/tree/McCain
I name this settings "McCain X2a Novice"
And another one.
I name this "McCain X2a 1400"
So Levels has no effect if the UCI_LimitStrength is enabled. Am I right?
BTW McCain X2 UCI_ELO and Levels options are visible in Winboard, but have not tried playing games there.
2. To play by Elo rating, level must be set to"None" with UCI_LimitStrength checked, the level settings have predefined Elo settings associated with each level that are not user adjustable. There is roughly a 100 Elo point spreads between each level. More than what I would have wanted, but there appears to be limitation on the number of dropdown items that are available in xBoard, that's probably also true in other GUIs. I used the maximum number of dropdowns that were visible in xBoard.
-
- Posts: 4833
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: stockfish 10 vs. Mephisto III S Glasgow
This seemed complicated at first. So If I use Levels Novice, I still have to check UCI_LimitStrength? If so since UCI_LimitStrength is checked does the UCI_Elo matters?MikeB wrote: ↑Sun Apr 07, 2019 6:34 am1. Correct UCI_LimitStrength must be checked to enable weaker settings ( either by level or Elo)Ferdy wrote: ↑Sun Apr 07, 2019 3:07 amThanks for the update, now it works with cutechess.MikeB wrote: ↑Sat Apr 06, 2019 7:45 pmGot it - thanks for pointing this out.
Source updated:
https://github.com/MichaelB7/Stockfish/tree/McCain
I name this settings "McCain X2a Novice"
And another one.
I name this "McCain X2a 1400"
So Levels has no effect if the UCI_LimitStrength is enabled. Am I right?
BTW McCain X2 UCI_ELO and Levels options are visible in Winboard, but have not tried playing games there.
In the image that follows which one will be followed by engine, 2800 or Novice? I know the answer after reading number 2 below.
All right this one is clear.2. To play by Elo rating, level must be set to"None" with UCI_LimitStrength checked, the level settings have predefined Elo settings associated with each level that are not user adjustable.
I would suggest though to separate functionality of UCI_LimitStrength / UCI_Elo pair and Levels option. The pair UCI_LimitStrength / UCI_Elo is defined in uci protocol. It would be convenient to the user to select Levels alone say Novice etc, without tinkering the UCI_LimitStrength. However if the user wants UCI_Elo, don't let Levels affect the game play, just make the Levels to None automatically without the need for the user to set it to None.
I have tried dropdown in Winboard up to 45 items, don't know if that is the max.There is roughly a 100 Elo point spreads between each level. More than what I would have wanted, but there appears to be limitation on the number of dropdown items that are available in xBoard, that's probably also true in other GUIs. I used the maximum number of dropdowns that were visible in xBoard.
-
- Posts: 4833
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: stockfish 10 vs. Mephisto III S Glasgow
Sorry Mike but there is another problem but related to combo type for other options. Perhaps you have to review all your combos.MikeB wrote: ↑Sat Apr 06, 2019 7:45 pmGot it - thanks for pointing this out.
Source updated:
https://github.com/MichaelB7/Stockfish/tree/McCain
This time it is related to Analysis Contempt and Book File 1, perhaps there are others.
Code: Select all
Warning: Invalid UCI option from McCain X2a 1400: option name Analysis Contempt type combo default var Off var White var Black var Both
Warning: Invalid UCI option from McCain X2a 1400: option name Book File 1 type combo default var None var Cerebellum var Champions var Alekhine var Anand var Botvinnik var Capablanca var Carlsen var Fischer var Karpov var Kasparov var Kortschnoi var Kramink var Lasker var Petrosian var Tal
-
- Posts: 10282
- Joined: Thu Mar 09, 2006 12:37 am
- Location: Tel-Aviv Israel
Re: stockfish 10 vs. Mephisto III S Glasgow
It is 35 years of both.
The new software play chess better with the same hardware and the new hardware play chess better with the same software.
Maybe stockfish is not the best with the oldest hardware that it can use(do not know) but it is not optimized for hardware that nobody use today.
-
- Posts: 476
- Joined: Sun Mar 17, 2019 12:00 pm
- Full name: Henk Drost
Re: stockfish 10 vs. Mephisto III S Glasgow
With Leela's network being 52.5MB and her client being 8 MB, it'd be already a challenge to load her up.jp wrote: ↑Sun Apr 07, 2019 1:24 amHow many nodes could Leela get on that hardware? It looks like you'd need to give it a long time per move even to get a 0 node "search".Raphexon wrote: ↑Sat Apr 06, 2019 10:40 pmLeela would probably smash Mephisto on a single node search.koedem wrote: ↑Sat Apr 06, 2019 5:49 pm If we're doing strange fixed nodes testing, then why not put Leela vs Mephisto at 1000 nodes. Surely that will be close. (I predict Leela will score 100-0 in a hundred game match) Obviously if your node contains much more calculations your engine will be stronger at fixed node count. Instead it should be tested at fixed (be it slow) hardware.
Either way, I tried calculating how many NPS could get on a CPU from 1984.
I think you'd be talking about maybe at most a node per day. But realistically it wouldn't work at all.
-
- Posts: 105
- Joined: Fri Mar 18, 2016 10:45 pm
Re: stockfish 10 vs. Mephisto III S Glasgow
That was the point. That nodes are not a good comparison. And that instead one should test on (possibly simulated) equal hardware.Raphexon wrote: ↑Sun Apr 07, 2019 10:19 amWith Leela's network being 52.5MB and her client being 8 MB, it'd be already a challenge to load her up.jp wrote: ↑Sun Apr 07, 2019 1:24 amHow many nodes could Leela get on that hardware? It looks like you'd need to give it a long time per move even to get a 0 node "search".Raphexon wrote: ↑Sat Apr 06, 2019 10:40 pmLeela would probably smash Mephisto on a single node search.koedem wrote: ↑Sat Apr 06, 2019 5:49 pm If we're doing strange fixed nodes testing, then why not put Leela vs Mephisto at 1000 nodes. Surely that will be close. (I predict Leela will score 100-0 in a hundred game match) Obviously if your node contains much more calculations your engine will be stronger at fixed node count. Instead it should be tested at fixed (be it slow) hardware.
Either way, I tried calculating how many NPS could get on a CPU from 1984.
I think you'd be talking about maybe at most a node per day. But realistically it wouldn't work at all.
-
- Posts: 476
- Joined: Sun Mar 17, 2019 12:00 pm
- Full name: Henk Drost
Re: stockfish 10 vs. Mephisto III S Glasgow
SF beats Mephisto on similar hardware.mclane wrote: ↑Sat Apr 06, 2019 2:58 pm It does not ?
It beated Stockfish.
With a software from 1984,
That’s 35 years old.
If Mephisto III has no chess understanding when using 1-3 NPS on 8 bit cpu or 3-5 on 16 bit cpu,
How did it get the championship title against stupid opponents that did 1500 or more NPS ?
Factor 500 or 300 (depending which hardware) more NPS .
Where the search trees of the brute force opponents so inefficient?
Why did the other programs needed 300-500 times more nps to reach similar playing strength?
Obviously all these calculated positions did not help for the situation and were waste of time.
Stockfish 6 gets 500-100 nps on an intel 80486 with 4.3 MIPS.
Mephisto uses a 2.1 MIP CPU, so you could have just given Stockfish half time odds.
With 40/120 time control for Mephisto 22500 nodes per move would have been fair for Stockfish.
And what about more nps makes a program intrinsically less efficient?
Mephisto uses a computationally more expensive function, therefore it has lower nps.
Other programs use a function that computationally less expensive, therefore they have higher nps.
If Mephisto is so good, then how come it only gets so few nps compared to other programs on the same hardware?
Must be insanely inefficient code.
-
- Posts: 476
- Joined: Sun Mar 17, 2019 12:00 pm
- Full name: Henk Drost
Re: stockfish 10 vs. Mephisto III S Glasgow
It's possible with Stockfish 6, as I've already showed.koedem wrote: ↑Sun Apr 07, 2019 10:25 amThat was the point. That nodes are not a good comparison. And that instead one should test on (possibly simulated) equal hardware.Raphexon wrote: ↑Sun Apr 07, 2019 10:19 amWith Leela's network being 52.5MB and her client being 8 MB, it'd be already a challenge to load her up.jp wrote: ↑Sun Apr 07, 2019 1:24 amHow many nodes could Leela get on that hardware? It looks like you'd need to give it a long time per move even to get a 0 node "search".Raphexon wrote: ↑Sat Apr 06, 2019 10:40 pmLeela would probably smash Mephisto on a single node search.koedem wrote: ↑Sat Apr 06, 2019 5:49 pm If we're doing strange fixed nodes testing, then why not put Leela vs Mephisto at 1000 nodes. Surely that will be close. (I predict Leela will score 100-0 in a hundred game match) Obviously if your node contains much more calculations your engine will be stronger at fixed node count. Instead it should be tested at fixed (be it slow) hardware.
Either way, I tried calculating how many NPS could get on a CPU from 1984.
I think you'd be talking about maybe at most a node per day. But realistically it wouldn't work at all.
SF6 on 80486 gets 500-1000 NPS, if you give it half-time odds vs Mephisto it's a fair matchup.
And SF at 250 nps would easily dominate Mephisto.