My two cents on Rybka's disqualification

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

Dann Corbit
Posts: 12870
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: My two cents on Rybka's disqualification

Post by Dann Corbit »

Dann Corbit wrote:
Laskos wrote:
Dann Corbit wrote:
Laskos wrote:
Dann Corbit wrote:
Laskos wrote:
Dann Corbit wrote:
The strength increases exponentially due to hardware.
The strength increases exponentially due to software.
Logarithmically.
The Elo scale is logarithmic, but the increase in compute power and playing strength is exponential.

In the same way that a 9.0 earthquake is much larger than a 7.0 earthquake, an Elo of 2900 is tremendously larger than an Elo of 2600.

Taking the log of one million does not make the million become smaller.
It just turns it into a smaller number that is easier for some to understand.
Take the strength as % and compare it to Elo, the curve is even flatter than linear for % versus Elo, and pretty linear for small Elo differences. So, the strength in % (do you have a better definition?) is going even slower than logarithm. The strength change is at best (fastest) logarithmic in hardware and software.

Kai
For strength, I choose the measurement of number of chessboards intelligently condidered per second.

I don't know if it works for people. It works great for chess computers.
By 'intelligently' I also imply that we look at the important boards and discard examination of the stupid looking choices unless we really have to consider them.
Sorry, this hermeneutic evades me, I simply consider strength as % taken from the adversary, 0% minimum, 100% maximum, 50% equal, perfectly asymmetric at 50% and slower than logarithmic in hardware and software, with the limiting pure logarithm at 50%.

Kai
If you measure the effort with mini-max, I think you will find that the measure is quite faulty. It will take millions of times more effort to win 95% of the points.
Put another way, why is it that a computer that is 1000 times stronger does not win 1000 times more points? If we measure the compute power on a linear scale, we can use this scale to find out what it means in wins, losses, and draws.
Dann Corbit
Posts: 12870
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: My two cents on Rybka's disqualification

Post by Dann Corbit »

Dann Corbit wrote:
Dann Corbit wrote:
Laskos wrote:
Dann Corbit wrote:
Laskos wrote:
Dann Corbit wrote:
Laskos wrote:
Dann Corbit wrote:
The strength increases exponentially due to hardware.
The strength increases exponentially due to software.
Logarithmically.
The Elo scale is logarithmic, but the increase in compute power and playing strength is exponential.

In the same way that a 9.0 earthquake is much larger than a 7.0 earthquake, an Elo of 2900 is tremendously larger than an Elo of 2600.

Taking the log of one million does not make the million become smaller.
It just turns it into a smaller number that is easier for some to understand.
Take the strength as % and compare it to Elo, the curve is even flatter than linear for % versus Elo, and pretty linear for small Elo differences. So, the strength in % (do you have a better definition?) is going even slower than logarithm. The strength change is at best (fastest) logarithmic in hardware and software.

Kai
For strength, I choose the measurement of number of chessboards intelligently condidered per second.

I don't know if it works for people. It works great for chess computers.
By 'intelligently' I also imply that we look at the important boards and discard examination of the stupid looking choices unless we really have to consider them.
Sorry, this hermeneutic evades me, I simply consider strength as % taken from the adversary, 0% minimum, 100% maximum, 50% equal, perfectly asymmetric at 50% and slower than logarithmic in hardware and software, with the limiting pure logarithm at 50%.

Kai
If you measure the effort with mini-max, I think you will find that the measure is quite faulty. It will take millions of times more effort to win 95% of the points.
Put another way, why is it that a computer that is 1000 times stronger does not win 1000 times more points? If we measure the compute power on a linear scale, we can use this scale to find out what it means in wins, losses, and draws.
Put another way, this sort of thinking would mean that a 9.0 earthquake is only twice as strong as a 4.5
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: My two cents on Rybka's disqualification

Post by Laskos »

Dann Corbit wrote:
If you measure the effort with mini-max, I think you will find that the measure is quite faulty. It will take millions of times more effort to win 95% of the points.
This is precisely the "strength" according to hardware and software. In fact exactly this is faster than exponential versus strength given by %, which is the palpable strength. Usually we consider the strength to be represented by the _score_, means %.

Kai
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: My two cents on Rybka's disqualification

Post by Laskos »

Dann Corbit wrote:
Dann Corbit wrote:
Dann Corbit wrote:
Laskos wrote:
Dann Corbit wrote:
Laskos wrote:
Dann Corbit wrote:
Laskos wrote:
Dann Corbit wrote:
The strength increases exponentially due to hardware.
The strength increases exponentially due to software.
Logarithmically.
The Elo scale is logarithmic, but the increase in compute power and playing strength is exponential.

In the same way that a 9.0 earthquake is much larger than a 7.0 earthquake, an Elo of 2900 is tremendously larger than an Elo of 2600.

Taking the log of one million does not make the million become smaller.
It just turns it into a smaller number that is easier for some to understand.
Take the strength as % and compare it to Elo, the curve is even flatter than linear for % versus Elo, and pretty linear for small Elo differences. So, the strength in % (do you have a better definition?) is going even slower than logarithm. The strength change is at best (fastest) logarithmic in hardware and software.

Kai
For strength, I choose the measurement of number of chessboards intelligently condidered per second.

I don't know if it works for people. It works great for chess computers.
By 'intelligently' I also imply that we look at the important boards and discard examination of the stupid looking choices unless we really have to consider them.
Sorry, this hermeneutic evades me, I simply consider strength as % taken from the adversary, 0% minimum, 100% maximum, 50% equal, perfectly asymmetric at 50% and slower than logarithmic in hardware and software, with the limiting pure logarithm at 50%.

Kai
If you measure the effort with mini-max, I think you will find that the measure is quite faulty. It will take millions of times more effort to win 95% of the points.
Put another way, why is it that a computer that is 1000 times stronger does not win 1000 times more points? If we measure the compute power on a linear scale, we can use this scale to find out what it means in wins, losses, and draws.
Put another way, this sort of thinking would mean that a 9.0 earthquake is only twice as strong as a 4.5
No, no, no, it's exactly due to computers that we are used to such exponentials, due to the tragic fact that an increase by a factor of 2 is always giving almost the same Elo increase (or even smaller % increase). Before that we had only the _score_, therefore only %.

Kai
Dann Corbit
Posts: 12870
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: My two cents on Rybka's disqualification

Post by Dann Corbit »

Laskos wrote:
Dann Corbit wrote:
If you measure the effort with mini-max, I think you will find that the measure is quite faulty. It will take millions of times more effort to win 95% of the points.
This is precisely the "strength" according to hardware and software. In fact exactly this is faster than exponential versus strength given by %, which is the palpable strength. Usually we consider the strength to be represented by the _score_, means %.

Kai
Well, I agree that you are right by your measure and I am right by my measure. I do not think it matters at all which measure you choose. Both measurements will make accurate predictions based upon the data so both are equally good.
Dann Corbit
Posts: 12870
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: My two cents on Rybka's disqualification

Post by Dann Corbit »

Dann Corbit wrote:
Laskos wrote:
Dann Corbit wrote:
If you measure the effort with mini-max, I think you will find that the measure is quite faulty. It will take millions of times more effort to win 95% of the points.
This is precisely the "strength" according to hardware and software. In fact exactly this is faster than exponential versus strength given by %, which is the palpable strength. Usually we consider the strength to be represented by the _score_, means %.

Kai
Well, I agree that you are right by your measure and I am right by my measure. I do not think it matters at all which measure you choose. Both measurements will make accurate predictions based upon the data so both are equally good.
P.S.
A chess program that wins 100% is probably impossible for the near future so every measure of strength should show that.
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: My two cents on Rybka's disqualification

Post by Laskos »

Dann Corbit wrote:
Dann Corbit wrote:
Laskos wrote:
Dann Corbit wrote:
If you measure the effort with mini-max, I think you will find that the measure is quite faulty. It will take millions of times more effort to win 95% of the points.
This is precisely the "strength" according to hardware and software. In fact exactly this is faster than exponential versus strength given by %, which is the palpable strength. Usually we consider the strength to be represented by the _score_, means %.

Kai
Well, I agree that you are right by your measure and I am right by my measure. I do not think it matters at all which measure you choose. Both measurements will make accurate predictions based upon the data so both are equally good.
P.S.
A chess program that wins 100% is probably impossible for the near future so every measure of strength should show that.
Well, Crafty 19 will very probably have 100:0 against me on my hardware. Such things are not rare (to me at least).

Kai
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: My two cents on Rybka's disqualification

Post by bob »

garybelton wrote:
then purports the product they produce to be "100% original"
If Vasik said this then that would be ridiculous! Please provide reference.
Read the report. :) Look at communication between Vas and David. It has been stated by Vas multiple times. And it _is_ ridiculous.
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: My two cents on Rybka's disqualification

Post by bob »

garybelton wrote:If he said that (and I trust you that he did) then I think he is stretching it a lot. Basically NO chess engine author can say that at all. It may be 100% fresh codes but that doesn't mean a god damned thing, the ideas are well known and well worn. As I have said, the ideas are the gold.
But the ideas are a "concept" while the code is an "implementation". We are concerned with implementation here, which is where the problem lies. Too much identical code. Not just identical ideas. But identical code.
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: My two cents on Rybka's disqualification

Post by bob »

Dann Corbit wrote:
Harvey Williamson wrote:
Dann Corbit wrote:I think it is very likely that Vas has done something wrong. My big problem is with the process.

1. Vas was singled out. Where is the reverse engineering of Shredder, Hiarcs, Junior, etc.? Why has Vas been singled out for this scrutiny and the other commercial vendors have not?
2. The process to find similarity is someone's recent invention. Has it been tested on other similar systems (meaning a test of similarity between TSCP and Brutus is absurd, a test between Junior and Shredder and other high end programs that are binary only against high-end open source programs makes sense)
If the experiment is run without any controls, then what have we really shown?
To my way of thinking, it would also be a good idea to perform the following exercise:
A. Take a strong open source program and copy it.
B. Make a bunch of changes and form a binary.
C. Study the program and use the algorithms in a new program.
D. Compare the results of the tool for case B and C
I doubt if anybody wants to put the work in to do this, but it seems a good way to model the problem and test the results of analysis.
3. The prosecution was formed by asking "Hey, anybody who wants to throw stones at Vas, gather over here in the barn."
Imagine if a jury were formed in that manner.
4. Look into your own source code at the following:
A. PVS search
B. Null move reductions
C. LMR reductions
Now, do these look a whole lot like those found in some other programs? If so, why are you not prosecuting yourself? If it is OK, then where is the line to draw in the sand that says "This amount of similarity is OK, but this amount is cheating"?
The panel was open to all that had an interest prosecution and defence. Why team rybka stayed away is a question for them. vas was invited many times to take part, he refused. i am sure if you had applied to join you would have been accepted.

Fabien came back to this board as a result of his posts other programmers and him sent a letter to the ICGA asking them to investigate.

if there is evidence against other programs then make a case and I am sure they will be investigated.
I do hereby charge every single program which has ever entered the ICGA with possible copying.
Fine, now offer some reasonable evidence to get things started, such as we had with the strelka release...