Move ordering contest

Discussion of chess software programming and technical issues.

Moderators: hgm, Dann Corbit, Harvey Williamson

User avatar
Rebel
Posts: 6946
Joined: Thu Aug 18, 2011 12:04 pm

Re: Move ordering contest

Post by Rebel »

petero2 wrote: I disabled razoring, futility pruning and reverse futility pruning in my program and repeated the test:

Code: Select all

pos  cutoffs firstMove    fh%  move  bm
1   12870889  12038268  93.53  Rxb4  Rxb4
2   11787485  11353698  96.32  O-O   O-O 
3   12082633  11727666  97.06  Bxe4  Bxe4
4   11200972  10538521  94.09  Bxg6  Bxg6
5   11282575  10794627  95.68  b4    b4	 
6   12586754  12090930  96.06  f3    f3	 
7   15946205  14852428  93.14  Kg3   Kg3 
8   15371962  14490620  94.27  Kg3   Kg3 
9   11617640  11104048  95.58  c5    c5	 
10  12167017  11742638  96.51  Nxe6  Nxe6
This supports the theory that examining uninteresting nodes improves the "fail high on first move" percentage, which could also explain why stronger programs had lower percentages in this test: http://www.top-5000.nl/moresu.htm
Wow...

I got some more entries, will publish tomorrow. Had little time the last days.
User avatar
Rebel
Posts: 6946
Joined: Thu Aug 18, 2011 12:04 pm

Re: Move ordering contest

Post by Rebel »

Final results are up: http://www.top-5000.nl/moresu.htm

Marcel wins the trophy :lol:

Don loses with a great distance :wink:

It seems I had it all backwards.

I will make a new one as suggested just for the fun of it and see where we will end up. But I won't call it a contest any longer !
User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 4:27 pm

Re: Move ordering contest

Post by Don »

Rebel wrote:Final results are up: http://www.top-5000.nl/moresu.htm

Marcel wins the trophy :lol:

Don loses with a great distance :wink:

It seems I had it all backwards.

I will make a new one as suggested just for the fun of it and see where we will end up. But I won't call it a contest any longer !
There is no doubt something here we are all missing.

Is there in general a reverse correlation between strength and score on this test or is it random?
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
Daniel Shawul
Posts: 4185
Joined: Tue Mar 14, 2006 11:34 am
Location: Ethiopia

Re: Move ordering contest

Post by Daniel Shawul »

Rebel wrote:Final results are up: http://www.top-5000.nl/moresu.htm

Marcel wins the trophy :lol:

Don loses with a great distance :wink:

It seems I had it all backwards.

I will make a new one as suggested just for the fun of it and see where we will end up. But I won't call it a contest any longer !
I will venture a guess that such contests will end up being an ego-trip for some after the recent 'counter move' buzz. I bet you expected Don to win because he suggested it to stockish and it got +20 elo. The eventual winner Rookie wins by a huge margin again because it has counter move heuristic. Hence my guess that this was a subtle counter moves ego-trip. So I will cut to the chase and say the idea is at least 10 years older than both as Gerd pointed out in that thread. For the record I got about 6-8 elos out of it, while something I did to improve move ordering through IID gave me 14 elos. Calling it a contest and assigning a winner shadows any scientific merrit it may have. It is a liscense to say 'my eval is best', 'my move ordering is best' etc.. and the automatic justification is 'my engine is not so strong for nothing fool'. Ok enough of this from me, but I would have liked it more if it was more a discussion of move ordering methods and how much you gain from it.
User avatar
Rebel
Posts: 6946
Joined: Thu Aug 18, 2011 12:04 pm

Re: Move ordering contest

Post by Rebel »

Don wrote:
Rebel wrote:Final results are up: http://www.top-5000.nl/moresu.htm

Marcel wins the trophy :lol:

Don loses with a great distance :wink:

It seems I had it all backwards.

I will make a new one as suggested just for the fun of it and see where we will end up. But I won't call it a contest any longer !
There is no doubt something here we are all missing.

Is there in general a reverse correlation between strength and score on this test or is it random?
You missed the irony of the outcome?
User avatar
Rebel
Posts: 6946
Joined: Thu Aug 18, 2011 12:04 pm

Re: Move ordering contest

Post by Rebel »

Daniel Shawul wrote:
Rebel wrote:Final results are up: http://www.top-5000.nl/moresu.htm

Marcel wins the trophy :lol:

Don loses with a great distance :wink:

It seems I had it all backwards.

I will make a new one as suggested just for the fun of it and see where we will end up. But I won't call it a contest any longer !
I will venture a guess that such contests will end up being an ego-trip for some after the recent 'counter move' buzz. I bet you expected Don to win
Exactly!

And look what happened.
User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 4:27 pm

Re: Move ordering contest

Post by Don »

Rebel wrote:
Daniel Shawul wrote:
Rebel wrote:Final results are up: http://www.top-5000.nl/moresu.htm

Marcel wins the trophy :lol:

Don loses with a great distance :wink:

It seems I had it all backwards.

I will make a new one as suggested just for the fun of it and see where we will end up. But I won't call it a contest any longer !
I will venture a guess that such contests will end up being an ego-trip for some after the recent 'counter move' buzz. I bet you expected Don to win
Exactly!

And look what happened.
My conclusion is that Komodo has horrible move ordering, the worst of all programs!
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
User avatar
Rebel
Posts: 6946
Joined: Thu Aug 18, 2011 12:04 pm

Re: Move ordering contest

Post by Rebel »

Don wrote:
Rebel wrote:
Daniel Shawul wrote:
Rebel wrote:Final results are up: http://www.top-5000.nl/moresu.htm

Marcel wins the trophy :lol:

Don loses with a great distance :wink:

It seems I had it all backwards.

I will make a new one as suggested just for the fun of it and see where we will end up. But I won't call it a contest any longer !
I will venture a guess that such contests will end up being an ego-trip for some after the recent 'counter move' buzz. I bet you expected Don to win
Exactly!

And look what happened.
My conclusion is that Komodo has horrible move ordering, the worst of all programs!
Mail me anyway, it seems to produce some good elo :wink:
Daniel Shawul
Posts: 4185
Joined: Tue Mar 14, 2006 11:34 am
Location: Ethiopia

Re: Move ordering contest

Post by Daniel Shawul »

I won't buy in to his 'appeal to humility' because I do not think being bad in move ordering indicates higher elo. It is absurd to try and justify negative correlation between move ordering and higher strength just because a strong engine performed bad. Strength is influenced by way too many factors to make any conclusions. Every one belived Rybka had super evaluation until it is proved not to be so. Therefore we need to make a proper test of each move ordering component for the test to be un-flawed by 'percieved elo of engine', otherwise it is no better than the Rybka PR.
User avatar
Rebel
Posts: 6946
Joined: Thu Aug 18, 2011 12:04 pm

Re: Move ordering contest

Post by Rebel »

Daniel Shawul wrote:I won't buy in to his 'appeal to humility' because I do not think being bad in move ordering indicates higher elo. It is absurd to try and justify negative correlation between move ordering and higher strength just because a strong engine performed bad. Strength is influenced by way too many factors to make any conclusions. Every one belived Rybka had super evaluation until it is proved not to be so. Therefore we need to make a proper test of each move ordering component for the test to be un-flawed by 'percieved elo of engine', otherwise it is no better than the Rybka PR.
Huh ?

Okay, carry on.