What happend to TCEC?

[Dann Corbit]

What should have happened:
1. If the ICGA wanted to throw Vas out of the contest, they should have asked for his code at the time of the contest, like the contest rules state.
2. If they decided to reverse engineer his code, they should have reverse engineered the code of the program that played in the contest, not some other version of the program.
3. If they found that his program violated their club rules, they should have done what their rule said, and disqualified him from the contest. But the rule does not give them the right to go beyond that.

They should NOT have sent press releases all over the planet calling Vas a liar and a thief. This was the worst and most absurd thing that they did.

So, once again, did Vas do something I don't like? Yes, he used an awful lot of Fruit's ideas in his Rybka 1.0 program. Note that this is not the program that played in the ICGA contest. Should we also (for instance) issue mean press releases because the Baron was originally based on TSCP? Of course not. If we wanted to question some of the programs in the contest, it should be programs that actually played in the contest.

So there is my beef with the whole thing. And we get to rehash it once again. And yet we are right back at square one, and nobody is any the smarter and we did not learn anything from our mistakes. How typically human of us.

[Dann Corbit]

specific legal process.
https://en.wikipedia.org/wiki/Abstracti ... rison_test
This process was not performed.

I should mention, even if it had been performed, it would have to be performed in a court of law and then decided upon by either a judge or a jury to be something you can publicly use against them.

Joe Schmedlap does not get to perform this test and then call someone a criminal. It has to be done legally.

The ICGA sounds important, because they have "International" in their name. But they do not carry the weight of law.

[hgm]

You understand well enough. The odds of flipping ten heads in a row is 1/(2^10) but ten heads in a row happens. A string of 7 observations has no statistical significance. Even a win by the weaker program would mean almost nothing.

So you have no clue. Having 10 heads out of 10 allows you to conclude with 99.9% confidence that the coin was not fair. And yes, it happens with a fair coin. In 0.1% of the cases. This is why the confidence is 'only' 99.9%, and not 100.0%. For 7 heads out of 7 the confidence in would still be >99%. And a draw by a 300-Elo weaker program has even less probability than 50%. It is supposed to score less than 15%, perhaps 5% wins and 20% draws. So only 25% chance for a non-loss. 7 non-losses out of 7 would reject the hypothesis that it is 300 Elo weaker with > 99.99% confidence.

Still trembling with fear?

TCEC did not do anything illegal or immoral.

Uh? They used stolen software, not? That is a crime.

[Dann Corbit]

The Jonny system with the zillion cores system it used for WCCC would be literally pulverized by:

Stockfish
or
Houdini
or
Komodo

on TCEC hardware.

I guess in a thousand games Jonny would be lucky to win one.

I wonder how you can be so confident on this. You certainly did not try it. (That is a friendly way of saying: you are just dreaming.) In reality Komodo had great trouble beating Johnny at WCCC, even though it was using a 60 core system. All the regular games ended draw. It took 3 playoffs at successively shorter TC for Komodo to finally win one.

That hardly sounds like 'being pulverized'.

And this is fact, rather than purely wishful thinking...

On your part.

I don't even know what "fact on my part" means... It seems you are completely detached from reality.

Look, I know you are a smart fellow. But any reasonable person can see that both of us have valid viewpoints on this.

You see things in absolutes, and that is OK. My big sister is like that too, and I love her.

[Dann Corbit]

You understand well enough. The odds of flipping ten heads in a row is 1/(2^10) but ten heads in a row happens. A string of 7 observations has no statistical significance. Even a win by the weaker program would mean almost nothing.

So you have no clue. Having 10 heads out of 10 allows you to conclude with 99.9% confidence that the coin was not fair. And yes, it happens with a fair coin. In 0.1% of the cases. This is why the confidence is 'only' 99.9%, and not 100.0%. For 7 heads out of 7 the confidence in would still be >99%. And a draw by a 300-Elo weaker program has even less probability than 50%. It is supposed to score less than 15%, perhaps 5% wins and 20% draws. So only 25% chance for a non-loss. 7 non-losses out of 7 would reject the hypothesis that it is 300 Elo weaker with > 99.99% confidence.

Still trembling with fear?

TCEC did not do anything illegal or immoral.

Uh? They used stolen software, not? That is a crime.

So your conclusion about 10 heads or 10 tails in a row is that the coin is broken?

Brilliant.

The strength of Jonny has been accurately measured by CEGT, and it is miles below the strongest programs.

But enjoy your fantasy, based on 7 games, including the fact that it did lose as expected.

[Dann Corbit]

P.S.
If we include all 9 games from their ICGA faceoff, we get error bars 217 Elo wide.

Code: Select all

  Program   Elo    +   -   Games   Score   Av.Op.  Draws
1 Komodo  : 3019  174  43     9    55.6 %   2981   88.9 %
2 Jonny   : 2981   43 174     9    44.4 %   3019   88.9 %

[hgm]

So your conclusion about 10 heads or 10 tails in a row is that the coin is broken?

That conclusion would be correct 99.8% of the time. Only 0.2% of the fair coins would be falsely rejected with that standard.

Brilliant.

Hardly. Just not completely stupid...

The strength of Jonny has been accurately measured by CEGT, and it is miles below the strongest programs.

In your dreams. They did not measure the version that participated in the WCCC. They did not measure it running on 2048 cores.

But enjoy your fantasy, based on 7 games, including the fact that it did lose as expected.

When you discard hard math as 'fantasy'... I just point out the facts. That they are not convenient for your beliefs, is purely your problem. It basically means you can no longer be taken seriously.

P.S.
If we include all 9 games from their ICGA faceoff, we get error bars 217 Elo wide.

Code: Select all

  Program   Elo    +   -   Games   Score   Av.Op.  Draws
1 Komodo  : 3019  174  43     9    55.6 %   2981   88.9 %
2 Jonny   : 2981   43 174     9    44.4 %   3019   88.9 %

Indeed. And 2981 - 217 = 2764. Which is larger than 3019 - 300 = 2719. As these are 95%-confidence error bars, and the suspected rating difference lies outside it, the hypothesis that the ratig difference is 300 is rejected with >95% confidence.

[Dann Corbit]

So your conclusion about 10 heads or 10 tails in a row is that the coin is broken?

That conclusion would be correct 99.8% of the time. Only 0.2% of the fair coins would be falsely rejected with that standard.

Brilliant.

Hardly. Just not completely stupid...

The strength of Jonny has been accurately measured by CEGT, and it is miles below the strongest programs.

In your dreams. They did not measure the version that participated in the WCCC. They did not measure it running on 2048 cores.

But enjoy your fantasy, based on 7 games, including the fact that it did lose as expected.

When you discard hard math as 'fantasy'... I just point out the facts. That they are not convenient for your beliefs, is purely your problem. It basically means you can no longer be taken seriously.

Well then, the Stockfish testing people can rejoice because now instead of running thousands of tests, they can stop at 7.

You seem to misunderstand the difference between an observation and the probability of that observation. But that is OK with me.
Just because it is unlikely does not mean you won't see it.

In fact if you do it often enough, you will see lots and lots of incredibly unlikely things.

Maybe even Jonny beating Komodo.

[Dann Corbit]

So your conclusion about 10 heads or 10 tails in a row is that the coin is broken?

That conclusion would be correct 99.8% of the time. Only 0.2% of the fair coins would be falsely rejected with that standard.

Brilliant.

Hardly. Just not completely stupid...

The strength of Jonny has been accurately measured by CEGT, and it is miles below the strongest programs.

In your dreams. They did not measure the version that participated in the WCCC. They did not measure it running on 2048 cores.

But enjoy your fantasy, based on 7 games, including the fact that it did lose as expected.

When you discard hard math as 'fantasy'... I just point out the facts. That they are not convenient for your beliefs, is purely your problem. It basically means you can no longer be taken seriously.

Well then, the Stockfish testing people can rejoice because now instead of running thousands of tests, they can stop at 7.

You seem to misunderstand the difference between an observation and the probability of that observation. But that is OK with me.
Just because it is unlikely does not mean you won't see it.

In fact if you do it often enough, you will see lots and lots of incredibly unlikely things.

Maybe even Jonny beating Komodo.

P.S.
Did you know that:
HHHHHHHHHH
and
TTTTTTTTTT
and
HHTHTHTTTT
and
HTTHTTHHHH
all have equal probability?
It's a fact.

[hgm]

Well then, the Stockfish testing people can rejoice because now instead of running thousands of tests, they can stop at 7.

Stupid, stupid, stupid... The Stockfish testing people want to reject 3-Elo regressions with confidence, not 300-Elo regressions. Hundred times smaller error bars requires 10,000 times as many games, because the error bars decrease as the square root of the number of games.

You seem to misunderstand the difference between an observation and the probability of that observation. But that is OK with me.
Just because it is unlikely does not mean you won't see it.

I really must advise you to educate yourself in the most elementary aspects of statistics, as anything you say here makes you look more and more ignorant....

P.S.
Did you know that:
HHHHHHHHHH
and
TTTTTTTTTT
and
HHTHTHTTTT
and
HTTHTTHHHH
all have equal probability?
It's a fact.

Yes, everyone knows that, and even friends of mine that had never had any formal math training in their life came to that conclusion after some thinking. The fact that you think it is a discovery important enough to dedicate a whole posting to it says it all.