TCEC 10

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Milos
Posts: 2905
Joined: Wed Nov 25, 2009 12:47 am

Re: TCEC 10

Post by Milos » Sat Nov 25, 2017 2:38 am

syzygy wrote:
Geonerd wrote:It looks like Komodo is now running ~20% slower than in prior rounds.

How many ELO is that supposed to be worth?
I can't imagine K's NUMA bug is entirely responsible for the thrashing it's receiving.
Indeed. Bad luck probably plays a bigger role.
Not such thing as bad luck. It is currently +5=20-0 which translates to
70+/-56Elo (2 sigma error bars).
Houdini is simply a stronger engine.

shrapnel
Posts: 1089
Joined: Fri Nov 02, 2012 8:43 am
Location: New Delhi, India

Re: TCEC 10

Post by shrapnel » Sat Nov 25, 2017 3:50 am

I read from TCEC chat, that ML confirmed Komodo 1970 has NUMA bug causing slow speed
That is just loser whining, nothing else.
Komodo is a good Engine and in some ways even better than Houdini, but Lefler is simply paying the price of being over-cautious with his ridiculous Contempt 0.
In Chess, as in Life, sometimes chances have to be taken to achieve something worthwhile.
He who Dares, Wins !
About Houdini overall, there is something I like about its piece sacrifices and willingness to take calculated chances.
That's EXACTLY what positive Contempt is about !
i7 5960X @ 4.1 Ghz, 64 GB G.Skill RipJaws RAM, Asus ROG Strix 11 GB Geforce 1080 Ti and AMD Ryzen 7 1800X @4.0 GHz, 32 GB DDR4-2400 G.Skill RAM, ASUS Prime x370-PRO, Noctua NH-D15 SE-AM4 Cooler.

Modern Times
Posts: 2126
Joined: Thu Jun 07, 2012 9:02 pm

Re: TCEC 10

Post by Modern Times » Sat Nov 25, 2017 4:43 am

Damir wrote:No adjustments. Komodo must suffer. The version Komodo is playing with should have been tested before the super final, and found out if it was buggy or not... Komodo should not have any kind of special treatment :) :)
Agree. The engine isn't crashing, it isn't causing problems with GUI or hardware, and isn't losing on time.
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

JJJ
Posts: 1174
Joined: Sat Apr 19, 2014 11:47 am

Re: TCEC 10

Post by JJJ » Sat Nov 25, 2017 6:45 am

20% Slower means Komodo might loose something like 15 to 20 elo I guess ? Can someone estimate it ?

syzygy
Posts: 4181
Joined: Tue Feb 28, 2012 10:56 pm

Re: TCEC 10

Post by syzygy » Sat Nov 25, 2017 9:37 am

Milos wrote:
syzygy wrote:
Geonerd wrote:It looks like Komodo is now running ~20% slower than in prior rounds.

How many ELO is that supposed to be worth?
I can't imagine K's NUMA bug is entirely responsible for the thrashing it's receiving.
Indeed. Bad luck probably plays a bigger role.
Not such thing as bad luck. It is currently +5=20-0 which translates to
70+/-56Elo (2 sigma error bars).
Houdini is simply a stronger engine.
In a match with two equally strong engines, there is a 1 in 16 probability that the first 5 wins are by the same engine. Are you seriously claiming that there is no such thing as a 1 in 16 event?

But I was not even suggesting that they are equally strong. I just pointed out that there is no reason to suspect that the version of Komodo now playing is seriously bugged, i.e. beyond having lower nps than the earlier versions.

I recently ran a test that had one engine lead the other 10-1-19 before it got trashed. It just got lucky in those first 30 games. Such things happen all the time.

syzygy
Posts: 4181
Joined: Tue Feb 28, 2012 10:56 pm

Re: TCEC 10

Post by syzygy » Sat Nov 25, 2017 9:44 am

JJJ wrote:20% Slower means Komodo might loose something like 15 to 20 elo I guess ? Can someone estimate it ?
If a doubling in speed wins 50 Elo, so a loss of 50% speed loses 50 Elo, then a loss of 20% speed corresponds to a loss of 50 * log(0.8) / log(0.5) = 16 Elo.

corres
Posts: 374
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: TCEC 10

Post by corres » Sat Nov 25, 2017 9:59 am

I think in this case there are about more things.
During previous Stages Komodo team could play with variations of Contempt. Against weaker engines this gave some advantage for Komodo to avoid draws. But in general Contempt causes some weakening in chess power. Against Houdini Komodo needs its full power so playing with Contempt does not give any help for it.
I think it should be more correct to play the whole competition with the same engines without any modifications.

corres
Posts: 374
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: TCEC 10

Post by corres » Sat Nov 25, 2017 11:53 am

[quote="syzygy"]

If a doubling in speed wins 50 Elo, so a loss of 50% speed loses 50 Elo, then a loss of 20% speed corresponds to a loss of 50 * log(0.8) / log(0.5) = 16 Elo.

[/quote]

You are right if number of cores are few.
But such a high number of cores the difference is about 20 Elo in the case of doubling the speed. So weakening of Komodo is about 6 Elo merely.

Milos
Posts: 2905
Joined: Wed Nov 25, 2009 12:47 am

Re: TCEC 10

Post by Milos » Sat Nov 25, 2017 12:07 pm

syzygy wrote:In a match with two equally strong engines, there is a 1 in 16 probability that the first 5 wins are by the same engine. Are you seriously claiming that there is no such thing as a 1 in 16 event?
I am suggesting you don't understand much about statistics. Probability is trinomial not binomial, draw probability plays a serious role. +1=99-0 is few orders of magnitude more reliable statistics than +1=0-0. Therefore, your simplified comparison with coin toss is just a wrong straw man. Chance that K is not worse than H is cdf for x>2.5sigma which is around 1% not 6.25% as your oversimplified "calculation" suggests.
But I was not even suggesting that they are equally strong. I just pointed out that there is no reason to suspect that the version of Komodo now playing is seriously bugged, i.e. beyond having lower nps than the earlier versions.
20% lower nps at TCEC TC (obvious sore looser excuse btw.) and that strong hardware is at best 10Elo. Houdini's advantage is clearly over 10Elo in the worst case (best for Komodo).
I recently ran a test that had one engine lead the other 10-1-19 before it got trashed. It just got lucky in those first 30 games. Such things happen all the time.
Again you are just mixing apples and oranges. Draw probability is not even remotely similar as to the one in TCEC. In addition TC plays a role. It's a proven fact that extremely long TC's reduce error bars compared to extremely short. Your test is most probably from bullet. So again totally incomparable. Moreover, quoting once in a blue moon event as some example against solid statistics is really the basic fallacy that ppl who don't know much about statistics so often do.

syzygy
Posts: 4181
Joined: Tue Feb 28, 2012 10:56 pm

Re: TCEC 10

Post by syzygy » Sat Nov 25, 2017 12:43 pm

Milos wrote:
syzygy wrote:In a match with two equally strong engines, there is a 1 in 16 probability that the first 5 wins are by the same engine. Are you seriously claiming that there is no such thing as a 1 in 16 event?
I am suggesting you don't understand much about statistics. Probability is trinomial not binomial, draw probability plays a serious role. +1=99-0 is few orders of magnitude more reliable statistics than +1=0-0. Therefore, your simplified comparison with coin toss is just a wrong straw man.
People are freaking out about H winning 5 out of 5 decided games. I am pointing out that, if we assume H and K to be equally strong, the probability of either H or K winning the first 5 decided game is 1 in 16. This is not a rare event.
Chance that K is not worse than H is cdf for x>2.5sigma which is around 1% not 6.25% as your oversimplified "calculation" suggests.
I neither mentioned nor suggested any probability of K not being weaker than H.

As to the "low" draw rate, that is influenced very strongly by the openings.

Post Reply