TCEC 10
Moderators: hgm, Rebel, chrisw
-
- Posts: 1346
- Joined: Sat Apr 19, 2014 1:47 pm
Re: TCEC 10
20% Slower means Komodo might loose something like 15 to 20 elo I guess ? Can someone estimate it ?
-
- Posts: 5566
- Joined: Tue Feb 28, 2012 11:56 pm
Re: TCEC 10
In a match with two equally strong engines, there is a 1 in 16 probability that the first 5 wins are by the same engine. Are you seriously claiming that there is no such thing as a 1 in 16 event?Milos wrote:Not such thing as bad luck. It is currently +5=20-0 which translates tosyzygy wrote:Indeed. Bad luck probably plays a bigger role.Geonerd wrote:It looks like Komodo is now running ~20% slower than in prior rounds.
How many ELO is that supposed to be worth?
I can't imagine K's NUMA bug is entirely responsible for the thrashing it's receiving.
70+/-56Elo (2 sigma error bars).
Houdini is simply a stronger engine.
But I was not even suggesting that they are equally strong. I just pointed out that there is no reason to suspect that the version of Komodo now playing is seriously bugged, i.e. beyond having lower nps than the earlier versions.
I recently ran a test that had one engine lead the other 10-1-19 before it got trashed. It just got lucky in those first 30 games. Such things happen all the time.
-
- Posts: 5566
- Joined: Tue Feb 28, 2012 11:56 pm
Re: TCEC 10
If a doubling in speed wins 50 Elo, so a loss of 50% speed loses 50 Elo, then a loss of 20% speed corresponds to a loss of 50 * log(0.8) / log(0.5) = 16 Elo.JJJ wrote:20% Slower means Komodo might loose something like 15 to 20 elo I guess ? Can someone estimate it ?
-
- Posts: 3657
- Joined: Wed Nov 18, 2015 11:41 am
- Location: hungary
Re: TCEC 10
I think in this case there are about more things.
During previous Stages Komodo team could play with variations of Contempt. Against weaker engines this gave some advantage for Komodo to avoid draws. But in general Contempt causes some weakening in chess power. Against Houdini Komodo needs its full power so playing with Contempt does not give any help for it.
I think it should be more correct to play the whole competition with the same engines without any modifications.
During previous Stages Komodo team could play with variations of Contempt. Against weaker engines this gave some advantage for Komodo to avoid draws. But in general Contempt causes some weakening in chess power. Against Houdini Komodo needs its full power so playing with Contempt does not give any help for it.
I think it should be more correct to play the whole competition with the same engines without any modifications.
-
- Posts: 3657
- Joined: Wed Nov 18, 2015 11:41 am
- Location: hungary
Re: TCEC 10
[quote="syzygy"]
If a doubling in speed wins 50 Elo, so a loss of 50% speed loses 50 Elo, then a loss of 20% speed corresponds to a loss of 50 * log(0.8) / log(0.5) = 16 Elo.
[/quote]
You are right if number of cores are few.
But such a high number of cores the difference is about 20 Elo in the case of doubling the speed. So weakening of Komodo is about 6 Elo merely.
If a doubling in speed wins 50 Elo, so a loss of 50% speed loses 50 Elo, then a loss of 20% speed corresponds to a loss of 50 * log(0.8) / log(0.5) = 16 Elo.
[/quote]
You are right if number of cores are few.
But such a high number of cores the difference is about 20 Elo in the case of doubling the speed. So weakening of Komodo is about 6 Elo merely.
-
- Posts: 4190
- Joined: Wed Nov 25, 2009 1:47 am
Re: TCEC 10
I am suggesting you don't understand much about statistics. Probability is trinomial not binomial, draw probability plays a serious role. +1=99-0 is few orders of magnitude more reliable statistics than +1=0-0. Therefore, your simplified comparison with coin toss is just a wrong straw man. Chance that K is not worse than H is cdf for x>2.5sigma which is around 1% not 6.25% as your oversimplified "calculation" suggests.syzygy wrote:In a match with two equally strong engines, there is a 1 in 16 probability that the first 5 wins are by the same engine. Are you seriously claiming that there is no such thing as a 1 in 16 event?
20% lower nps at TCEC TC (obvious sore looser excuse btw.) and that strong hardware is at best 10Elo. Houdini's advantage is clearly over 10Elo in the worst case (best for Komodo).But I was not even suggesting that they are equally strong. I just pointed out that there is no reason to suspect that the version of Komodo now playing is seriously bugged, i.e. beyond having lower nps than the earlier versions.
Again you are just mixing apples and oranges. Draw probability is not even remotely similar as to the one in TCEC. In addition TC plays a role. It's a proven fact that extremely long TC's reduce error bars compared to extremely short. Your test is most probably from bullet. So again totally incomparable. Moreover, quoting once in a blue moon event as some example against solid statistics is really the basic fallacy that ppl who don't know much about statistics so often do.I recently ran a test that had one engine lead the other 10-1-19 before it got trashed. It just got lucky in those first 30 games. Such things happen all the time.
-
- Posts: 5566
- Joined: Tue Feb 28, 2012 11:56 pm
Re: TCEC 10
People are freaking out about H winning 5 out of 5 decided games. I am pointing out that, if we assume H and K to be equally strong, the probability of either H or K winning the first 5 decided game is 1 in 16. This is not a rare event.Milos wrote:I am suggesting you don't understand much about statistics. Probability is trinomial not binomial, draw probability plays a serious role. +1=99-0 is few orders of magnitude more reliable statistics than +1=0-0. Therefore, your simplified comparison with coin toss is just a wrong straw man.syzygy wrote:In a match with two equally strong engines, there is a 1 in 16 probability that the first 5 wins are by the same engine. Are you seriously claiming that there is no such thing as a 1 in 16 event?
I neither mentioned nor suggested any probability of K not being weaker than H.Chance that K is not worse than H is cdf for x>2.5sigma which is around 1% not 6.25% as your oversimplified "calculation" suggests.
As to the "low" draw rate, that is influenced very strongly by the openings.
-
- Posts: 1339
- Joined: Fri Nov 02, 2012 9:43 am
- Location: New Delhi, India
Re: TCEC 10
+1.Milos wrote:Moreover, quoting once in a blue moon event as some example against solid statistics is really the basic fallacy that ppl who don't know much about statistics so often do.
I must say I agree.
Some lawyers I know could prove with their arguments that Night is actually Day and Day is actually Night !
Ronald de man seems to fall in the same category, juggling around figures to prove anything.
He may be a great Programmer or whatever, but seems to lack basic Common Sense.
Its perfectly obvious to the average Joe watching TCEC, that Houdini is playing much stronger and brilliantly than Komodo; whether it is because of Contempt or any other reason, is another matter.
i7 5960X @ 4.1 Ghz, 64 GB G.Skill RipJaws RAM, Twin Asus ROG Strix OC 11 GB Geforce 2080 Tis
-
- Posts: 5566
- Joined: Tue Feb 28, 2012 11:56 pm
Re: TCEC 10
A large number of cores will not lower the Elo gain of a doubling in speed. (A doubling in speed is certainly much better than a doubling in number of cores. We are talking about a difference in speed here, not a difference in number of cores.)corres wrote:You are right if number of cores are few.syzygy wrote:
If a doubling in speed wins 50 Elo, so a loss of 50% speed loses 50 Elo, then a loss of 20% speed corresponds to a loss of 50 * log(0.8) / log(0.5) = 16 Elo.
But such a high number of cores the difference is about 20 Elo in the case of doubling the speed. So weakening of Komodo is about 6 Elo merely.
A large time control might lower the Elo gain of a doubling in speed, but I think mostly through the increased draw rate. Since TCEC openings are deliberately somewhat unbalanced, the draw rate in TCEC is likely to be lower than one would otherwise expect at the time control used.
-
- Posts: 5566
- Joined: Tue Feb 28, 2012 11:56 pm
Re: TCEC 10
If the average Joe carefully reads what I wrote, he will see that I never suggested that Houdini is not stronger than Komodo.shrapnel wrote:+1.Milos wrote:Moreover, quoting once in a blue moon event as some example against solid statistics is really the basic fallacy that ppl who don't know much about statistics so often do.
I must say I agree.
Some lawyers I know could prove with their arguments that Night is actually Day and Day is actually Night !
Ronald de man seems to fall in the same category, juggling around figures to prove anything.
He may be a great Programmer or whatever, but seems to lack basic Common Sense.
Its perfectly obvious to the average Joe watching TCEC, that Houdini is playing much stronger and brilliantly than Komodo; whether it is because of Contempt or any other reason, is another matter.
Someone suggested that Komodo's supposed NUMA bug cannot be entirely responsible for the "trashing" Komodo is receiving. I responded to point out that with a bit of bad luck, even an equally strong engine may well end up losing the first five decisive games.
So the Komodo from stages 1 and 2, possibly affected by a bug that slows it down by 20% to lose about 16 Elo, can easily lose the first 5 decisive games against the Houdini from stages 1 and 2. There is no real reason to suspect that Komodo 1970.00 is weaker than the previous version (beyond the possible slow down).
It is clear that Houdini has so far greatly outperformed Komodo. It is leading 5-0 with 22 draws.