Discussion of anything and everything relating to chess playing software and machines.
Moderators: hgm, Dann Corbit, Harvey Williamson
Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
-
Javier Ros
- Posts: 185
- Joined: Fri Oct 12, 2012 10:48 am
- Location: Seville (SPAIN)
- Full name: Javier Ros
Post
by Javier Ros » Wed Sep 26, 2018 3:44 pm
chrisw wrote: ↑Wed Sep 26, 2018 9:32 am
Guenther wrote: ↑Wed Sep 26, 2018 6:14 am
This is the table after some cleaning up (not only the unterminated, but also the wrong game headers)
Game 1 which was still testing and was played with no opening moves at all still counts...
Andscacs instead of
1. e4 c5 2. Nf3 h6 3. c3 vs. SF played from the start position and the testing game
was elevated to a real stage 2 game later.
The normally reversed opening pair for that game:
[Event "TESTING for Stage 2"]
[Site "Chess.com"]
[Date "2018.09.17"]
[Round "?"]
[White "Andscacs 0.94"]
[Black "Stockfish 220818"]
[Result "0-1"]
[WhiteElo "3244"]
[BlackElo "3300"]
[ECO "C01"]
[TimeControl "900+5"]
[Time "10:23:16"]
[Termination "normal"]
[PlyCount "118"]
1. e4 {+0.20/26 21}
1... e6 {-0.15/33 50}
2. d4 {+0.29/29 29}
2... d5 {-0.11/32 8}
3. exd5 {+0.27/29 67}
[Event "CCCC 1: Rapid Rumble Stage 2"]
[Site "Chess.com"]
[Date "2018.09.18"]
[Round "?"]
[White "Stockfish 220818"]
[Black "Andscacs 0.94"]
[Result "1-0"]
[WhiteElo "3439"]
[BlackElo "3300"]
[ECO "B27"]
[TimeControl "900+5"]
[Time "08:39:57"]
[Termination "normal"]
[PlyCount "147"]
1. e4 c5 2. Nf3 h6 3. c3 {+0.77/32 50}
Code: Select all
CCCC 1: Rapid Rumble (15|5) Stage 2
Chess.com, 2018.09.17 - 2018.09.25
Average Rating: 3430 (Category 48)
Rtng Score 1 2 3 4 5 6 7 8 Perf Chg
------------------------------------------------------------------------------------------------------------------------------------------
1: Stockfish 220818 3439 51.0 / 70 XXXXXXXXXX =1=1=1=1=1 ===1=====1 ====1===== 1=1=1=1=11 11=11=1=1= =1=111===1 11=11111== 3596 +140 (+32 -0 =38)
2: Houdini 6.03 3439 42.0 / 69 =0=0=0=0=0 XXXXXXXXXX 1==1==01=. ========1= 1==1=1==== 11=11=1==1 =1===1=1=1 1=1=1===1= 3501 +55 (+21 -6 =42)
3: Komodo 2118.00 3439 38.5 / 69 ===0=====0 0==0==10=. XXXXXXXXXX ======0=01 1====1===1 1==101==== 1==1==1==1 1==1=1=1== 3465 +21 (+16 -8 =45)
4: Lc0 17.11089 3439 37.5 / 70 ====0===== ========0= ======1=10 XXXXXXXXXX ========10 =111=0==== 11====1=== 10====101= 3451 +7 (+12 -7 =51)
5: Ethereal 10.88 3404 31.0 / 70 0=0=0=0=00 0==0=0==== 0====0===0 ========01 XXXXXXXXXX ==0=0====1 ======1=== 1=1=10=11= 3386 -14 (+8 -16 =46)
6: Fire 7.1 3404 29.0 / 70 00=00=0=0= 00=00=0==0 0==010==== =000=1==== ==1=1====0 XXXXXXXXXX ===1=1=1== =====0=1== 3364 -35 (+8 -20 =42)
7: Booot 6.3.1 3439 25.5 / 70 =0=000===0 =0===0=0=0 0==0==0==0 00====0=== ======0=== ===0=0=0== XXXXXXXXXX ====1===== 3321 -119 (+1 -20 =49)
8: Andscacs 0.94 3439 24.5 / 70 00=00000== 0=0=0===0= 0==0=0=0== 01====010= 0=0=01=00= =====1=0== ====0===== XXXXXXXXXX 3310 -126 (+4 -25 =41)
------------------------------------------------------------------------------------------------------------------------------------------
279 games: +79 -23 =177
I want to suggest the elo of LC0 somehow manages to match the elo of whatever it is playing against. Well, that's one "explanation" of its rather curious behaviour:
If one looks at the loss count of the second program, Houdini against the final ranking of its opponents, we get, as would be expected, an decreasing gradient: 5,1,0,0,0,0,0
Komodo gets: 2,3,2,0,1,0,0
Ethereal: 6,3,3,1,2,0,1
Fire: 6,6,3,3,1,0,1
Booot: 5,4,4,3,1,3,0
Andsacs: 7,4,4,3,5,1,1
Lc0 is different: 1,1,1,1,1,0,2, almost irrelevent who the opponent is, the loss rate remains almost constant.
Obviously, "non-losses", counting wins and draws together, shows the same pattern in reverse. Which suggests, well, to me, that LC0 doesn't really have an elo that can be mapped onto any particular opponent. It's not behaving itself properly according to the laws of elo ratings.
This constant loss rate is mainly due to the tactical weakness of lc0 that, when it appears against any alpha-beta program always leads to defeat.
On the other hand, this tactical weakness is practically irrelevant against humans, unable to detect it and take advantage of it in most cases.
So, perhaps we should say that lc0 is not behaving itself according to the laws of elo of the classical alpha-beta chess programs.
The love relationship between a chess engine tester and his computer can be summarized in one sentence:
Until heat do us part.
-
Guenther
- Posts: 3668
- Joined: Wed Oct 01, 2008 4:33 am
- Location: Regensburg, Germany
- Full name: Guenther Simon
-
Contact:
Post
by Guenther » Fri Sep 28, 2018 6:51 am
Guenther wrote: ↑Wed Sep 26, 2018 8:28 am
Code: Select all
# PLAYER : RATING ERROR POINTS PLAYED (%)
1 Stockfish 220818 : 3663.79 52.89 90.0 116 77.6
2 Houdini 6.03 : 3584.63 41.68 79.0 115 68.7
3 Komodo 2118.00 : 3565.40 46.24 76.5 115 66.5
4 Lc0 17.11089 : 3513.30 47.34 68.5 115 59.6
5 Fire 7.1 : 3465.80 43.57 62.0 116 53.4
6 Ethereal 10.88 : 3459.26 41.68 61.0 116 52.6
7 Booot 6.3.1 : 3416.64 44.68 54.5 116 47.0
8 Andscacs 0.94 : 3400.08 44.39 52.0 116 44.8
9 Shredder 13 : 3370.30 62.61 26.5 46 57.6
10 Xiphos 0.3.17 : 3324.13 63.92 24.0 46 52.2
11 Fritz 16.10 : 3324.13 60.07 24.0 46 52.2
12 Texel 1.07 : 3305.80 63.57 23.0 46 50.0
13 Laser 1.6 : 3289.81 60.37 21.0 44 47.7
14 Pedone 1.8 : 3269.06 64.72 21.0 46 45.7
15 Vajolet 2.6 : 3269.06 62.75 21.0 46 45.7
16 Arasan CCCC-2018 : 3250.55 61.77 20.0 46 43.5
17 Fizbo 1.9 : 3241.23 68.62 19.5 46 42.4
18 Gull 3.syz : 3222.42 70.70 18.5 46 40.2
19 Wasp 3.25 : 3164.08 72.15 15.5 46 33.7
20 Nemorino 5.00 : 3143.72 70.10 14.5 46 31.5
21 Ivanhoe 999946h : 3112.00 72.30 13.0 46 28.3
22 Senpai 2.0 : 3042.52 80.28 10.0 46 21.7
23 Nirvana 2.4 : 2989.46 90.39 8.0 46 17.4
24 Crafty 25.2 : 2932.85 104.85 6.0 45 13.3
White advantage = 61.91 +/- 8.17
Draw rate (equal opponents) = 72.56 % +/- 2.60
(based on average rating of 3305 see above)
Actualised after 72 games in the final, with the same calculation method and assumptions:
Code: Select all
# PLAYER : RATING ERROR POINTS PLAYED (%)
1 Stockfish 220818 : 3657.20 41.36 132.0 188 70.2
2 Houdini 6.03 : 3589.18 40.50 109.0 187 58.3
3 Komodo 2118.00 : 3564.86 42.28 76.5 115 66.5
4 Lc0 17.11089 : 3512.94 45.25 68.5 115 59.6
5 Fire 7.1 : 3465.54 37.90 62.0 116 53.4
6 Ethereal 10.88 : 3459.02 42.84 61.0 116 52.6
7 Booot 6.3.1 : 3416.49 40.92 54.5 116 47.0
8 Andscacs 0.94 : 3399.97 40.10 52.0 116 44.8
9 Shredder 13 : 3370.24 62.11 26.5 46 57.6
10 Xiphos 0.3.17 : 3324.15 63.11 24.0 46 52.2
11 Fritz 16.10 : 3324.15 62.58 24.0 46 52.2
12 Texel 1.07 : 3305.86 65.03 23.0 46 50.0
13 Laser 1.6 : 3289.88 59.45 21.0 44 47.7
14 Vajolet 2.6 : 3269.19 62.39 21.0 46 45.7
15 Pedone 1.8 : 3269.19 60.36 21.0 46 45.7
16 Arasan CCCC-2018 : 3250.71 65.01 20.0 46 43.5
17 Fizbo 1.9 : 3241.40 63.69 19.5 46 42.4
18 Gull 3.syz : 3222.63 64.67 18.5 46 40.2
19 Wasp 3.25 : 3164.39 67.80 15.5 46 33.7
20 Nemorino 5.00 : 3144.07 65.87 14.5 46 31.5
21 Ivanhoe 999946h : 3112.39 77.37 13.0 46 28.3
22 Senpai 2.0 : 3043.02 74.59 10.0 46 21.7
23 Nirvana 2.4 : 2990.04 83.16 8.0 46 17.4
24 Crafty 25.2 : 2933.47 102.07 6.0 45 13.3
White advantage = 59.74 +/- 7.87
Draw rate (equal opponents) = 74.87 % +/- 2.32
-
corres
- Posts: 3657
- Joined: Wed Nov 18, 2015 10:41 am
- Location: hungary
Post
by corres » Fri Sep 28, 2018 7:56 am
Robert Pope wrote: ↑Wed Sep 26, 2018 3:15 pm
I think the issue is that LC0 plays very well, except for a glaring hole that almost any opponent can capitalize on. Other engines have their own holes, but they are subtle, so weaker engines are less likely to discover them.
So we can say LC0 capitalize the weaker positional knowledge of AB engines and AB engines capitalize the weaker tactical knowledge of AB engines.
Where is the engine with positional knowledge of LC0 and tactical knowledge of AB engines?
-
yanquis1972
- Posts: 1766
- Joined: Tue Jun 02, 2009 10:14 pm
Post
by yanquis1972 » Fri Sep 28, 2018 8:03 am
Not far off. Will continue to be shocked there isn’t an adequate if not necessarily legal jerryrigged hybrid floating around. If lc0 + 3rd party doesn’t do it my money would be on Komodo or Houdini making a splash commercially.
-
ernest
- Posts: 1927
- Joined: Wed Mar 08, 2006 7:30 pm
Post
by ernest » Thu Oct 04, 2018 5:41 pm
What was the final result of the CCCC final : 120-80, 119-81 ?
Could find it nowhere...
-
ernest
- Posts: 1927
- Joined: Wed Mar 08, 2006 7:30 pm
Post
by ernest » Thu Oct 04, 2018 8:22 pm
Thanks ! How did you find that link ?
I had an older link, which never got updated...
-
jp
- Posts: 1438
- Joined: Mon Apr 23, 2018 5:54 am
Post
by jp » Mon Oct 08, 2018 6:42 am
On chess.com: "CCC2 is on hold until we can figure out the Lc0 timeout issue, and why some engines are going to 1 thread."
What is the Lc0 timeout issue??