What happend to TCEC?

[hgm]

Johannes might have explained to you, but you obviously didn't get much of it. I don't need to speak to him to know the best way of implementing speculative pondering and I've already posted the full calculation of possible gain and showed it is minuscule at best.
But ofc you didn't bother to read it, and I doubt you'd understand it. It's actually too optimistic to expect from someone who doesn't understand simple parallel programming concepts such as Amdahl's law to actually understand why any high gain in extreme parallelization of a serial algorithm is plain ridiculous.

That calculation is demonstrably flawed: with an infinite number of cores it is easy to get an arbitrarily large extra depth (corresponding to an arbitrarily large speedup, if you equate depth to the logarithm of speed). Just start with the desired number of extra ply as an (easily parallellizable) full-width minimax search, and in each node of that put a 24-core cluster node at work for a normal search.

Amdahls law only applies to parallellizing a given algorithm, not necessarily to parallellizing a given task. It doesn't rule out that there are other algorithms, more suitable to parallellization, that might be inferior on a single core, but gain that back by scaling better.

Of course we know that alpha-beta reaches double the number of ply from minimax, and that the tree size is exponential with depth, so that the number of nodes required by minimax is about the square of that in an alpha-beta tree of the same depth. So the expected speedup factor can only be the square root of of what you would have for a 'magically parallellized' alpha-beta. But if you have a factor of 100 more nodes than can be efficiently used to search a single position, that is still a sizable factor. Of course it heps that the cluster nodes in Johnny's case are not truly independent, but exchange their important hash entries at a rate carefully designed not to hinder the searches themselves.

[Uri Blass]

Johannes might have explained to you, but you obviously didn't get much of it. I don't need to speak to him to know the best way of implementing speculative pondering and I've already posted the full calculation of possible gain and showed it is minuscule at best.
But ofc you didn't bother to read it, and I doubt you'd understand it. It's actually too optimistic to expect from someone who doesn't understand simple parallel programming concepts such as Amdahl's law to actually understand why any high gain in extreme parallelization of a serial algorithm is plain ridiculous.

That calculation is demonstrably flawed: with an infinite number of cores it is easy to get an arbitrarily large extra depth (corresponding to an arbitrarily large speedup, if you equate depth to the logarithm of speed). Just start with the desired number of extra ply as an (easily parallellizable) full-width minimax search, and in each node of that put a 24-core cluster node at work for a normal search.

Amdahls law only applies to parallellizing a given algorithm, not necessarily to parallellizing a given task. It doesn't rule out that there are other algorithms, more suitable to parallellization, that might be inferior on a single core, but gain that back by scaling better.

Of course we know that alpha-beta reaches double the number of ply from minimax, and that the tree size is exponential with depth, so that the number of nodes required by minimax is about the square of that in an alpha-beta tree of the same depth. So the expected speedup factor can only be the square root of of what you would have for a 'magically parallellized' alpha-beta. But if you have a factor of 100 more nodes than can be efficiently used to search a single position, that is still a sizable factor. Of course it heps that the cluster nodes in Johnny's case are not truly independent, but exchange their important hash entries at a rate carefully designed not to hinder the searches themselves.

Practically alpha beta with LMR and null move pruning reach clearly more than double the number of plies from minimax and I guess it may be better to build some tree of logical moves for a short time and let the cores search only nodes in the tree.

[hgm]

Sure. LMR would also help minimax to reach larger depth (on the PV, at the expense of the side branches). That you get exact scores helps to apply the reductions with more confidence, and thus more aggressively. (Thus defining the 'tree of logical moves' that you mention.) And when you use aspiration windows, even the minimax part would already have a large number of beta cutoffs, without having to wait for any branch to complete before you can set the search window: it is known before the minimax search, from the previous iteration. You can assign moves that seem deservant based on the current root score a separate cluster node, moving the search of the other moves on another cluster node with the main move excluded from search (after copying the most important part of the hash table there). There is plenty of room to be clever here, and I am sure Johannes is very clever.

If you have an infinite number of cores, there is no need to be clever, however. The example was just a 'proof of principle', to show that the 'ultimate limit' posed by Amdahl's law only apply as long as you keep the same algorithm, and can be surpassed by switching to other algorithms.

[Dann Corbit]

For games on the 'ultimate hardware' you have to watch Johnny play at WCCC. TCEC is a far cry from 2048 cores.

That's just BS. Jonny on 2048 cores is weaker than SF on any i7 laptop. So what were you trying to tell us?
2048 cores is anyway totally pointless, almost zero difference in strength (certainly less than 5 Elo) between 2048 of 256 cores.

How did you come to those conclusions? With what evidence?

I have spoken a bit with Johannes Zwanzger, The Jonny author, so I know a few of the things it does. One is multiple pondering. Having so much hardware available lets you ponder on pretty much any move your opponent might make. This would effectively double the amount of time a program would have to come up with a move. And a doubling of time, even at this number of cores, is certainly worth more than 5 elo. And he does a lot of other very clever things (but I will leave it up to him to mention them). On this hardware, Jonny is very strong, and I would consider it to be our strongest opponent this year (since I do not know much about this year's Shredder which also has probably gained a lot of elo).

Mark

I guess that none of these showed up then:

1 Stockfish 8.0 x64 4CPU 3414
{ N/A 2 Komodo 11.01 x64 4CPU 3402}
4 Houdini 5.0 x64 4CPU 3380
6 Deep Shredder 13 x64 4CPU 3225
7 Fizbo 1.9 x64 4CPU 3207
8 Gull 3.0 x64 4CPU 3186
9 Ginkgo 2.0 x64 4CPU 3174
10 Andscacs 0.90 x64 4CPU 3174

Since Jonny is below them.
11 Jonny 8.00 x64 4CPU 3157

Where do the numbers come from and how old is Jonny 8.00?
I see no reference and no details for your numbers.

Guenther

That looks like the CEGT 40/20 MP list.

Correct.
And for the curious, Jonny 8 is both the version tested by CEGT and also the version that played in TCEC 9.

[Dann Corbit]

I cower in fear at the statistical significance of those 7 games.

[Dann Corbit]

If it were 300 Elo stronger, in addition to having a time-odds advantage of a factor 10, it should have crushed Komodo 7-3 or so. But that did not happen.

I missed this pearl.
Haha and again haha. You are hilarious man, hilarious. You should become a stand-up comedian.

What has happened, obviously, is that Jonny somehow picked up 300 Elo since the TCEC 9 contest, and the other engines like Stockfish and Komodo picked up 0 Elo, therefore they are caught by Jonny.

Otherwise, the WCCC contest would be a sham and a waste of time, played by one strong engine and a few far weaker stragglers.

The main issue, as I see it, is that the ICGA (like Australia) is entirely peopled with criminals.

Hence, most of the strong engines won't show up. After all, who wants to have your software illegally reverse engineered and posted on the internet. And then have some yahoo send a post to all the major news services calling you a liar and a cheat.

Much better to have a sensible contest like TCEC, but unfortunately, Thoresen seems to have run out of gas and nobody is willing to pick up the baton.

[hgm]

I cower in fear at the statistical significance of those 7 games.

Have you calculated the probability that a 300-Elo weaker program scores 7 draws in a row? Or are you just an easily scared person?

The main issue, as I see it, is that the ICGA (like Australia) is entirely peopled with criminals.

Hmm, seems to get dangerously close to libel. (A criminal offense...)

You don't seem very concerned that Vas' work was stolen, forcing him out of business, and others pass off his work as their own in TCEC. So your crusade against 'criminals' seems a bit lopsided.

[Dann Corbit]

I cower in fear at the statistical significance of those 7 games.

Have you calculated the probability that a 300-Elo weaker program scores 7 draws in a row? Or are you just an easily scared person?

The error bars on 7 games is enough.

The main issue, as I see it, is that the ICGA (like Australia) is entirely peopled with criminals.

Hmm, seems to get dangerously close to libel. (A criminal offense...)

You don't seem very concerned that Vas' work was stolen, forcing him out of business, and others pass off his work as their own in TCEC. So your crusade against 'criminals' seems a bit lopsided.

[/quote]
Apparently, you are not a "Princess Bride" fan.

In any case, I honestly do feel that their actions were criminal.

No wait, they don't need subpoenas or warrants to obtain and publish illegally reverse engineered program details. They found a "Captain Planet" decoder ring in their box of cereal, which gives them "Phenomenal Cosmic Power" do do whatever they want.

If you imagine that I am not upset at the treatment of Vas, then you have not read anything I have said on the subject.

Let me reiterate:
Vas did something I did not like, which is use a great deal of information from Fruit. What he did was not illegal. In fact, nearly everyone did this that was in the contest, but not nearly to so great a degree as what Vas did. In the case of Rybka 1.0, I think it would have been reasonable to even disqualify Vas on the basis of their "too similar" rule. But that is neither here, nor there.

The ICGA did lots of illegal things in their investigation. And their punishment was beyond absurd.

OK, that is my statement on it in a nutshell.

[hgm]

The error bars on 7 games is enough.

So what were the error bars?

If you imagine that I am not upset at the treatment of Vas, then you have not read anything I have said on the subject.

So, according to you, TCEC is "peopled with criminals" as well, for shamelessly promoting the Rybka rip-offs?

[Dann Corbit]

The error bars on 7 games is enough.

So what were the error bars?

You understand well enough. The odds of flipping ten heads in a row is 1/(2^10) but ten heads in a row happens. A string of 7 observations has no statistical significance. Even a win by the weaker program would mean almost nothing.

If you imagine that I am not upset at the treatment of Vas, then you have not read anything I have said on the subject.

So, according to you, TCEC is "peopled with criminals" as well, for shamelessly promoting the Rybka rip-offs?

TCEC did not do anything illegal or immoral. They did not espouse or analyze engines at all. They simply held a contest. Formerly, the CCRL and CEGT groups tried to make moral judgements about the programs and that was a mistake.

If TCEC had (for instance) reverse engineered Junior, published the reverse engineering details, and called the Junior programer Amir Ban a criminal in the press, then that would have been something illegal and bad.

As far as the ICGA goes, it is a little club with their own little rules. As long as people agree to these rules, I think they can do a lot of things that might be questionable outside of the club, if they keep things inside of the club.

But when they publish the details outside of the club and when they send out an international press release calling one of the program authors a liar and a cheat, they crossed the sensible boundaries.

In order to say that someone has stolen someone else's code, there is a specific legal process.
https://en.wikipedia.org/wiki/Abstracti ... rison_test
This process was not performed.

Now, let me clarify something else. I don't fault the ICGA as much as you might imagine. They are not a body of software lawyers who knew how to legally arrive at a verdict or who understood the ramifications of what they were doing. Similarly for the board of programmers who went over the evidence. They were all smart programmers who know about programming but they are not legal experts who know what is legal and what is illegal and how to go about proving it.

If you are going to publicly say that someone stole something, it is a very bad idea to make up your own standards and then go international with that, instead of actually using legal channels.

Now, the ICGA probably did not want to get involved in a big legal struggle about what was really right and wrong in this instance. And for club purposes, they probably would not have to. But they went way beyond these bounds.

So who are the victims here? My answer is everyone.
Fabian is a victim.
Vas is a victim.
The ICGA board are all victims.
The ICGA itself is a victim.

The really, really big problem was the procedures used and the decisions made based on the outcome.

When it is a group of nerds who like to write chess programs and who have agreed to some internal rules then I see no problems with enforcing the rules as long as the punishments are those that are proscribed by the initial agreements.

Let's look at some history. There were many cases in precedent where someone literally stole someone elses' program and made a tiny tweak and entered it in the ICGA contest. That is not what Vas did. What sort of punishment did they receive? They got kicked out of the contest, just like they agreed to.

That is not what happened to Vas (although it is ONE of the things that happened to Vas).

His punishment was some kind of giant snowball full of the anger of every programmer who was ever wronged and then multiplied by a few billion.

I guess maybe the members got tired of the clones and so they made an example out of Vas. But Vas was not a thief like the others who only got a little slap on the wrist (which is what they all agreed to). Instead, he was bludgeoned to death. I don't think he agreed to that.