Robert question

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

kgburcham
Posts: 2016
Joined: Sun Feb 17, 2008 4:19 pm

Robert question

Post by kgburcham »

How do I know? I watched the output _carefully_. So that's an observed fact, not a guess.
Robert Hyatt


Doctor Hyatt, since Vas has tried to mask Rybka kns to fool you and other programmers from knowing the true kns, could Vas also do the same with Rybka output you were watching?

kgburcham
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Robert question

Post by bob »

kgburcham wrote:How do I know? I watched the output _carefully_. So that's an observed fact, not a guess.
Robert Hyatt


Doctor Hyatt, since Vas has tried to mask Rybka kns to fool you and other programmers from knowing the true kns, could Vas also do the same with Rybka output you were watching?

kgburcham
Vaguely possible. Here's the problem however. First, for the first couple of rounds they only kibitzed a single move, never a PV. They got complaints. So they tried to cobble something together to meet the requirements of the tournament more closely (rules required kibitzing the PV, not just the move played). When they did this, each cluster node kibitzed its own output which was way out of sync with the other nodes. The operator admitted, after I raised the question due to several people asking why the depth was jumping from 18 to 21 to 17 to ..., that the splitting at the root was what was going on. Apparently Vas admitted this on his Forum, but still claims that ridiculous +100 Elo improvement...

As far as I am concerned, this is a dead issue. He has claimed that his parallel search "scales better than anybody else's" but a couple of users have done some careful measurements and discovered this is not the truth, the whole truth, and nothing but the truth either (big surprise there...). You might find a single position where his program scales better, but not on average over many positions...

I currently have a ton of ideas to work on, so I don't plan on giving this any more thought... I only thought it interesting that he would try an idea that came from the late 1970's and was dismissed as unusable even back then...
User avatar
Ovyron
Posts: 4562
Joined: Tue Jul 03, 2007 4:30 am

Re: Robert question

Post by Ovyron »

bob wrote:Apparently Vas admitted this on his Forum, but still claims that ridiculous +100 Elo improvement...
This was Vas's last comment on the issue:
Vasik Rajlich wrote:Ok, when we have some time we'll do a controlled test. I ran a total of something like 1000 blitz games with several versions over the span of several weeks and the result was something like +100 Elo, this won't be too far off.

Vas
User avatar
Eelco de Groot
Posts: 4673
Joined: Sun Mar 12, 2006 2:40 am
Full name:   Eelco de Groot

Re: Robert question

Post by Eelco de Groot »

Ovyron wrote:
bob wrote:Apparently Vas admitted this on his Forum, but still claims that ridiculous +100 Elo improvement...
This was Vas's last comment on the issue:
Vasik Rajlich wrote:Ok, when we have some time we'll do a controlled test. I ran a total of something like 1000 blitz games with several versions over the span of several weeks and the result was something like +100 Elo, this won't be too far off.

Vas
It would certainly be interesting data! Lukas has a Core i7 Nehalem computer now. Maybe he could test it against the cluster! If he repeated the test with hyperthreading for Core i7 enabled, he could test two things at the same time. Another possibility for an interesting test against the cluster, or within the cluster, would be a Rybka 3 compiled on a recent Intel compiler running on the Nehalem. I'm not sure but Vas said in the past that he uses the Microsoft Visual C++ compiler. If that is still true, my guess would be that Intel will have compilers by now much better suited for its newest processor. But these are not free of course. If Rybka can be sped up for the Nehalem it might be worth it though for Vas to try to get a sample of the latest compilers that Intel must be using for its tests. No idea about prices.

Eelco
Debugging is twice as hard as writing the code in the first
place. Therefore, if you write the code as cleverly as possible, you
are, by definition, not smart enough to debug it.
-- Brian W. Kernighan
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Robert question

Post by bob »

Eelco de Groot wrote:
Ovyron wrote:
bob wrote:Apparently Vas admitted this on his Forum, but still claims that ridiculous +100 Elo improvement...
This was Vas's last comment on the issue:
Vasik Rajlich wrote:Ok, when we have some time we'll do a controlled test. I ran a total of something like 1000 blitz games with several versions over the span of several weeks and the result was something like +100 Elo, this won't be too far off.

Vas
It would certainly be interesting data! Lukas has a Core i7 Nehalem computer now. Maybe he could test it against the cluster! If he repeated the test with hyperthreading for Core i7 enabled, he could test two things at the same time. Another possibility for an interesting test against the cluster, or within the cluster, would be a Rybka 3 compiled on a recent Intel compiler running on the Nehalem. I'm not sure but Vas said in the past that he uses the Microsoft Visual C++ compiler. If that is still true, my guess would be that Intel will have compilers by now much better suited for its newest processor. But these are not free of course. If Rybka can be sped up for the Nehalem it might be worth it though for Vas to try to get a sample of the latest compilers that Intel must be using for its tests. No idea about prices.

Eelco
If he runs linux, he can at least test the intel compiler for free. But he can't sell the compiled version unless he buys the compiler from intel. It is not expensive either, compared to microsoft pricing... I don't think newer compilers are going to help a lot on the I7. The improvements are mostly internal and are pretty independent of how the compiler optimizes...
Nick C

Re: Robert question

Post by Nick C »

the operator admitted, after I raised the question due to several people asking why the depth was jumping from 18 to 21 to 17 to ..., that the splitting at the root was what was going on.
As the operator for Rybka on this occasion I can tell you that this statement is completely untrue. I said in Ch 64 that I thought this was Rybka kibbing its Sampled Search, as anyone listening there can confirm. The phrase "Splitting at the root" meant zero to me then, and not much more now.

Best regards,
Nick Carlin
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Robert question

Post by bob »

Nick C wrote:
the operator admitted, after I raised the question due to several people asking why the depth was jumping from 18 to 21 to 17 to ..., that the splitting at the root was what was going on.
As the operator for Rybka on this occasion I can tell you that this statement is completely untrue. I said in Ch 64 that I thought this was Rybka kibbing its Sampled Search, as anyone listening there can confirm. The phrase "Splitting at the root" meant zero to me then, and not much more now.

Best regards,
Nick Carlin
Then I was wrong in what I remembered. Perhaps this came from the Rybka forum instead... but whatever, that _is_ what was going on. There is no other explanation for the output at all...