MRL - The MEA Rating List

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

User avatar
Rebel
Posts: 6991
Joined: Thu Aug 18, 2011 12:04 pm

MRL - The MEA Rating List

Post by Rebel »

This new type of rating list is inspired by MEA a tool developed by Ferdinand Mosca processing the 1500 theme based STS positions created by Dann Corbit and Swaminathan Natarajan. As an experiment shows the results are reasonable in sync with the CCRL 4/40 rating list and insipired me to give the STS testsuite more attention in the form of 6 rating lists.

...

http://rebel13.nl/rebel13/mrl.html

-------

Feature updates in the tournament sub-forum.
90% of coding is debugging, the other 10% is writing bugs.
Albert Silver
Posts: 3019
Joined: Wed Mar 08, 2006 9:57 pm
Location: Rio de Janeiro, Brazil

Re: MRL - The MEA Rating List

Post by Albert Silver »

Rebel wrote: Thu Jun 07, 2018 8:14 pm This new type of rating list is inspired by MEA a tool developed by Ferdinand Mosca processing the 1500 theme based STS positions created by Dann Corbit and Swaminathan Natarajan. As an experiment shows the results are reasonable in sync with the CCRL 4/40 rating list and insipired me to give the STS testsuite more attention in the form of 6 rating lists.

...

http://rebel13.nl/rebel13/mrl.html

-------

Feature updates in the tournament sub-forum.
I'm quite curious to see whether Leela can fit into that system, being the bipolar gal she is. :-)
"Tactics are the bricks and sticks that make up a game, but positional play is the architectural blueprint."
User avatar
Rebel
Posts: 6991
Joined: Thu Aug 18, 2011 12:04 pm

Re: MRL - The MEA Rating List

Post by Rebel »

Albert Silver wrote: Thu Jun 07, 2018 9:20 pmI'm quite curious to see whether Leela can fit into that system, being the bipolar gal she is. :-)
I don't follow the zero news, what is the downoad location?
90% of coding is debugging, the other 10% is writing bugs.
Uri Blass
Posts: 10282
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: MRL - The MEA Rating List

Post by Uri Blass »

Rebel wrote: Thu Jun 07, 2018 8:14 pm This new type of rating list is inspired by MEA a tool developed by Ferdinand Mosca processing the 1500 theme based STS positions created by Dann Corbit and Swaminathan Natarajan. As an experiment shows the results are reasonable in sync with the CCRL 4/40 rating list and insipired me to give the STS testsuite more attention in the form of 6 rating lists.

...

http://rebel13.nl/rebel13/mrl.html

-------

Feature updates in the tournament sub-forum.

I am surprised to see old Critter leading at 10 seconds per move by a significant margin.

1 Critter 1.6a 13997 3481 1314 1500 0.876 15000 0.933 10000 128 1
2 asmFish 13886 3422 1288 1500 0.859 15000 0.926 10000 128 1
3 Komodo 9.02 13825 3392 1287 1500 0.858 15000 0.922 10000 128 1
4 Stockfish 9 13694 3326 1265 1500 0.843 15000 0.913 10000 128 1
Dann Corbit
Posts: 12540
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: MRL - The MEA Rating List

Post by Dann Corbit »

Uri Blass wrote: Thu Jun 07, 2018 11:03 pm
Rebel wrote: Thu Jun 07, 2018 8:14 pm This new type of rating list is inspired by MEA a tool developed by Ferdinand Mosca processing the 1500 theme based STS positions created by Dann Corbit and Swaminathan Natarajan. As an experiment shows the results are reasonable in sync with the CCRL 4/40 rating list and insipired me to give the STS testsuite more attention in the form of 6 rating lists.

...

http://rebel13.nl/rebel13/mrl.html

-------

Feature updates in the tournament sub-forum.

I am surprised to see old Critter leading at 10 seconds per move by a significant margin.

1 Critter 1.6a 13997 3481 1314 1500 0.876 15000 0.933 10000 128 1
2 asmFish 13886 3422 1288 1500 0.859 15000 0.926 10000 128 1
3 Komodo 9.02 13825 3392 1287 1500 0.858 15000 0.922 10000 128 1
4 Stockfish 9 13694 3326 1265 1500 0.843 15000 0.913 10000 128 1
Perhaps Critter had the evaluation tuned using STS.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
User avatar
Rebel
Posts: 6991
Joined: Thu Aug 18, 2011 12:04 pm

Re: MRL - The MEA Rating List

Post by Rebel »

Dann Corbit wrote: Thu Jun 07, 2018 11:33 pmPerhaps Critter had the evaluation tuned using STS.
That was my thought as well. I will look at it later. I have an EPD tool that mirrors the board position. We will see it will play the same (but mirrored) moves.
90% of coding is debugging, the other 10% is writing bugs.
Ferdy
Posts: 4833
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: MRL - The MEA Rating List

Post by Ferdy »

My experience with Fizbo 2 is that it supports movetime but is not optimal, if you let it search for 10s for example with,
go movetime 10000
it will only search less than 10s or exactly 16ms.
uci
id name Fizbo 2
id author Youri Matiounine
option name Threads type spin default 1 min 1 max 56
option name Hash type spin default 128 min 1 max 65536
option name Ponder type check default false
option name SyzygyPath type string default <empty>
option name UseEGTBInsideSearch type check default true
option name EGTBProbeLimit type spin default 5 min 0 max 6
uciok
isready
readyok
ucinewgame
position startpos
go movetime 10000
info depth 4 seldepth 7 score cp 25 time 0 nodes 735 nps 735000 tbhits 0 pv e2e4
d7d5 d1f3 d5e4
info depth 4 seldepth 7 score cp 30 time 0 nodes 924 nps 924000 tbhits 0 pv d2d4
d7d5 e2e3 c7c5
info depth 5 seldepth 12 score cp 30 time 16 nodes 1415 nps 88437 tbhits 0 pv d2
d4 d7d5 e2e3 c7c6 d1h5
info depth 5 seldepth 12 score cp 30 time 16 nodes 1416 nps 88500 tbhits 0 pv d2
d4 d7d5 e2e3 c7c6 d1h5
bestmove d2d4 ponder d7d5
You may check the log of mea on this engine for verification too.

At the end of each engine's log, there is a Time allocation rating, example

Code: Select all

[ 2017-08-06 23:15:28,696 -  WARNING ] Time allocation  : GOOD!!
[ 2017-08-06 23:15:28,696 -  WARNING ] ExpectedTime     : 64.0s
[ 2017-08-06 23:15:28,696 -  WARNING ] ActualTime       : 64.8s
[ 2017-08-06 23:15:28,696 -  WARNING ] TimeMargin/pos   : 0.2s
[ 2017-08-06 23:15:28,697 -  WARNING ] TimeMarginTotal  : 3.2s
It is better not to include this engine in the list.
User avatar
Rebel
Posts: 6991
Joined: Thu Aug 18, 2011 12:04 pm

Re: MRL - The MEA Rating List

Post by Rebel »

Ferdy wrote: Fri Jun 08, 2018 2:33 am My experience with Fizbo 2 is that it supports movetime but is not optimal, if you let it search for 10s for example with,
go movetime 10000
it will only search less than 10s or exactly 16ms.
I know, seen it in the log.
Ferdy wrote: Fri Jun 08, 2018 2:33 am It is better not to include this engine in the list.
There are more engines that not fully exploit the 10 seconds (among them my own) and move earlier. I leave them there to inspire the programmer to fix it. Consider that as a note to self also :D
90% of coding is debugging, the other 10% is writing bugs.
Ferdy
Posts: 4833
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: MRL - The MEA Rating List

Post by Ferdy »

Rating rate comparison as movetime increases. Higher is better. Arasan gained more going from 1s to 5s at r=1.172.

Image
User avatar
Rebel
Posts: 6991
Joined: Thu Aug 18, 2011 12:04 pm

Re: MRL - The MEA Rating List

Post by Rebel »

Nice stat....

In the meantime Critter strikes again, now also on top of the 8 CPU list.

http://rebel13.nl/mea4.html

Stockfish 9 now at place 7 :roll:

Pre-testing Bouquet 1.8 on another PC, scores about the same as Critter :lol:
90% of coding is debugging, the other 10% is writing bugs.