enough already with the rybka v houdini match?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

Should the current match between Rybka and Houdini in the tourney forum continue beyond 100 games?

no, I have seen enough; more games past 100 are not needed
12
30%
yes, I would like to see the match continue under the same conditions
24
60%
yes, I want more games but the conditions should change
4
10%
 
Total votes: 40

PawnStormZ
Posts: 880
Joined: Mon Feb 15, 2010 6:43 am

enough already with the rybka v houdini match?

Post by PawnStormZ »

   Hi All.
           I was wondering if people are tired of the match, and think it should end at 100 games.  Houdini has been extending its lead lately and there does not seem any reason to expect anything different in another 100 games.

           Maybe some change in conditions are advised before continuing.  I have read that there are some time-management and\or other settings that are supposed to make Rybka play stronger.  Is this even true, or worth trying?  Maybe using ponder = on for both.  
 
           I was thinking that if conditions were changed, the same 50 positions could be played again to see the effect, if any, on results.  Maybe you think it better to change settings but not reuse positions (but then we can not really be sure what effect they had).
 
           Of course, maybe you are tired of the whole thing already and think it should just go away!   :)
 
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: enough already with the rybka v houdini match?

Post by Laskos »

PawnStormZ wrote:    Hi All.
           I was wondering if people are tired of the match, and think it should end at 100 games.  Houdini has been extending its lead lately and there does not seem any reason to expect anything different in another 100 games.

           Maybe some change in conditions are advised before continuing.  I have read that there are some time-management and\or other settings that are supposed to make Rybka play stronger.  Is this even true, or worth trying?  Maybe using ponder = on for both.  
 
           I was thinking that if conditions were changed, the same 50 positions could be played again to see the effect, if any, on results.  Maybe you think it better to change settings but not reuse positions (but then we can not really be sure what effect they had).
 
           Of course, maybe you are tired of the whole thing already and think it should just go away!   :)
 
Pal, do not change anything, and continue as much as you can. 100 games are still few for error margins, they are enough only for LOS. If you change conditions, another 100 games would be a waste, as no one can decide what was a statistical fluke or a feature. To me, the most interesting part are the games themselves (half of the moves are simply uninteligible to me), the ratio win/loss at LTC on very strong hardware and the draw ratio. The problem is not that Houdini anyway wins, HOW it wins is the most important.

Continue to 200 games in identical conditions. I understand that it requires patience and passion, but you have to realize that your test would be one of the most reliable at LTC at this level, if not unique.

Kai
Dann Corbit
Posts: 12542
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: enough already with the rybka v houdini match?

Post by Dann Corbit »

Laskos wrote:
PawnStormZ wrote:    Hi All.
           I was wondering if people are tired of the match, and think it should end at 100 games.  Houdini has been extending its lead lately and there does not seem any reason to expect anything different in another 100 games.

           Maybe some change in conditions are advised before continuing.  I have read that there are some time-management and\or other settings that are supposed to make Rybka play stronger.  Is this even true, or worth trying?  Maybe using ponder = on for both.  
 
           I was thinking that if conditions were changed, the same 50 positions could be played again to see the effect, if any, on results.  Maybe you think it better to change settings but not reuse positions (but then we can not really be sure what effect they had).
 
           Of course, maybe you are tired of the whole thing already and think it should just go away!   :)
 
Pal, do not change anything, and continue as much as you can. 100 games are still few for error margins, they are enough only for LOS. If you change conditions, another 100 games would be a waste, as no one can decide what was a statistical fluke or a feature. To me, the most interesting part are the games themselves (half of the moves are simply uninteligible to me), the ratio win/loss at LTC on very strong hardware and the draw ratio. The problem is not that Houdini anyway wins, HOW it wins is the most important.

Continue to 200 games in identical conditions. I understand that it requires patience and passion, but you have to realize that your test would be one of the most reliable at LTC at this level, if not unique.

Kai
(+1) I echo this.
IOW, "Run the tournament as long as you can possibly stand it."
gerold
Posts: 10121
Joined: Thu Mar 09, 2006 12:57 am
Location: van buren,missouri

Re: enough already with the rybka v houdini match?

Post by gerold »

Dann Corbit wrote:
Laskos wrote:
PawnStormZ wrote:    Hi All.
           I was wondering if people are tired of the match, and think it should end at 100 games.  Houdini has been extending its lead lately and there does not seem any reason to expect anything different in another 100 games.

           Maybe some change in conditions are advised before continuing.  I have read that there are some time-management and\or other settings that are supposed to make Rybka play stronger.  Is this even true, or worth trying?  Maybe using ponder = on for both.  
 
           I was thinking that if conditions were changed, the same 50 positions could be played again to see the effect, if any, on results.  Maybe you think it better to change settings but not reuse positions (but then we can not really be sure what effect they had).
 
           Of course, maybe you are tired of the whole thing already and think it should just go away!   :)
 
Pal, do not change anything, and continue as much as you can. 100 games are still few for error margins, they are enough only for LOS. If you change conditions, another 100 games would be a waste, as no one can decide what was a statistical fluke or a feature. To me, the most interesting part are the games themselves (half of the moves are simply uninteligible to me), the ratio win/loss at LTC on very strong hardware and the draw ratio. The problem is not that Houdini anyway wins, HOW it wins is the most important.

Continue to 200 games in identical conditions. I understand that it requires patience and passion, but you have to realize that your test would be one of the most reliable at LTC at this level, if not unique.

Kai
(+1) I echo this.
IOW, "Run the tournament as long as you can possibly stand it."
Yes i would love to see another 100 games under the same conditions.

Thanks for your time and testing.Much appreciated.

Best,
Gerold.
gaard
Posts: 447
Joined: Mon Jun 07, 2010 3:13 am
Location: Holland, MI
Full name: Martin W

Re: enough already with the rybka v houdini match?

Post by gaard »

Thanks for all the high quality games.

I can't bear to watch any more, however. Let me throw in the towel for Rybka :) 4-sigma superiority is enough. I'd be interested in seeing more tournaments, but from endgame positions and with a faster time control like 40/20. My 2 cents.
User avatar
Dr.Wael Deeb
Posts: 9773
Joined: Wed Mar 08, 2006 8:44 pm
Location: Amman,Jordan

Re: enough already with the rybka v houdini match?

Post by Dr.Wael Deeb »

Sorry,I'm not interested in your match....
I have my own reasons regards,
Dr.D
_No one can hit as hard as life.But it ain’t about how hard you can hit.It’s about how hard you can get hit and keep moving forward.How much you can take and keep moving forward….
Dayffd
Posts: 424
Joined: Wed Sep 30, 2009 5:30 am

Re: enough already with the rybka v houdini match?

Post by Dayffd »

Dann Corbit wrote:
Laskos wrote:
PawnStormZ wrote:    Hi All.
           I was wondering if people are tired of the match, and think it should end at 100 games.  Houdini has been extending its lead lately and there does not seem any reason to expect anything different in another 100 games.

           Maybe some change in conditions are advised before continuing.  I have read that there are some time-management and\or other settings that are supposed to make Rybka play stronger.  Is this even true, or worth trying?  Maybe using ponder = on for both.  
 
           I was thinking that if conditions were changed, the same 50 positions could be played again to see the effect, if any, on results.  Maybe you think it better to change settings but not reuse positions (but then we can not really be sure what effect they had).
 
           Of course, maybe you are tired of the whole thing already and think it should just go away!   :)
 
Pal, do not change anything, and continue as much as you can. 100 games are still few for error margins, they are enough only for LOS. If you change conditions, another 100 games would be a waste, as no one can decide what was a statistical fluke or a feature. To me, the most interesting part are the games themselves (half of the moves are simply uninteligible to me), the ratio win/loss at LTC on very strong hardware and the draw ratio. The problem is not that Houdini anyway wins, HOW it wins is the most important.

Continue to 200 games in identical conditions. I understand that it requires patience and passion, but you have to realize that your test would be one of the most reliable at LTC at this level, if not unique.

Kai
(+1) I echo this.
IOW, "Run the tournament as long as you can possibly stand it."
(+2)
David S.
User avatar
hgm
Posts: 27811
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: enough already with the rybka v houdini match?

Post by hgm »

Playing two engines against each other is not really a reliable method to determine which one is better. At best you can say eventually: "It is almost certain that A beats B more often than B beats A", where "almost certain" can be driven up from 99% to 99.9% to 99.99999%, depending on your ptience. But even a 100% certainty that A beats B by a certain amount does not exclude that A can be weaker than B (i.e. has lower Elo).

To determine the Elo reliably you must play against a variety of opponents, distributed more or less homogeneously over an Elo range of +/- 300 around the engines.

So it would be much more meaningful to switch to other opponents after 100 games. Pit them both 100 games against an opponent 150 Elo weaker, and then each 100 games against an opponent 150 Elo stronger.
User avatar
Dr.Wael Deeb
Posts: 9773
Joined: Wed Mar 08, 2006 8:44 pm
Location: Amman,Jordan

Re: enough already with the rybka v houdini match?

Post by Dr.Wael Deeb »

hgm wrote:Playing two engines against each other is not really a reliable method to determine which one is better. At best you can say eventually: "It is almost certain that A beats B more often than B beats A", where "almost certain" can be driven up from 99% to 99.9% to 99.99999%, depending on your ptience. But even a 100% certainty that A beats B by a certain amount does not exclude that A can be weaker than B (i.e. has lower Elo).

To determine the Elo reliably you must play against a variety of opponents, distributed more or less homogeneously over an Elo range of +/- 300 around the engines.

So it would be much more meaningful to switch to other opponents after 100 games. Pit them both 100 games against an opponent 150 Elo weaker, and then each 100 games against an opponent 150 Elo stronger.
One of my reasons for my above answer....yes....
Dr.D
_No one can hit as hard as life.But it ain’t about how hard you can hit.It’s about how hard you can get hit and keep moving forward.How much you can take and keep moving forward….
Ant_Gugdin
Posts: 60
Joined: Sun Dec 26, 2010 9:13 pm

Re: enough already with the rybka v houdini match?

Post by Ant_Gugdin »

In the TCEC Division 1, Rybka 4 actually did better vs the other engines than Houdini did (although Houdini took first place by beating Rybka).