The latest CCRL Rating Lists and Statistics are available for viewing from the following links:
http://computerchess.org.uk/ccrl/4040/ (40/40)
http://www.computerchess.org.uk/ccrl/404/ (40/4)
http://www.computerchess.org.uk/ccrl/404FRC/ (FRC 40/4)
Please note that the three lists are often updated separately to each other. The FRC list is only updated when a new engine or engine version is being/has been tested.
40/40 testing this week that I'm aware of will include (with live broadcast port for TLCV noted where applicable):
The Brawl Tournament (continuing 16001)
66th Amateur Series Division 7 Tournament (continuing 16002)
66th Amateur Series Division 8 Tournament (continuing 16053)
Monolith 0.3 64-bit Gauntlet (continuing 16063)
Isa 2.0.48 64-bit Gauntlet (continuing 16064)
RomiChess P3n 64-bit Gauntlet (starting Monday 16065)
Texel 1.07 64-bit Gauntlet (finishing Monday 16066)
Devel 1.8090 Gauntlet (continuing)
Amoeba 2.6 64-bit Gauntlet (finishing)
Andscacs 0.92 64-bit Gauntlet (finishing)
SugaR XPrO 1.3 64-bit Gauntlet (finishing)
The Baron 3.41 64-bit Gauntlet (starting)
Marvin 2.2.0 64-bit Gauntlet (starting)
Jumbo 0.5.3 64-bit Gauntlet (starting)
Houdini 6 64-bit 4CPU Gauntlet (continuing 16083)
Fire 6.1 64-bit 4CPU Gauntlet (continuing 16084)
Texel 1.07 64-bit 4CPU Gauntlet (being run by Charles)
Andscacs 0.92 64-bit 4CPU Gauntlet (being run by Charles)
40/4 testing since the last update has included:
Little Wing 0.3.0 64-bit
Texel 1.07 64-bit
Demolito 2017-08-26 64-bit
Magnum 4.0
Devel 1.8090
Arasan 20.2 64-bit
Monolith 0.3 64-bit
ChessBrainVB 3.50
Anatoli 0.35k
Fire 6.1 64-bit
tomitankChess 1.4 64-bit
Orion 0.4.0 64-bit
Counter 2.0.2 64-bit
Amoeba 2.6 64-bit
Zurichess Neuchatel 64-bit
RomiChess P3n 64-bit
ECE X3 64-bit
Galjoen 0.36.0.1 64-bit
Hoichess 0.21.0 64-bit
K2 0.87
Zevra 1.6 r512 64-bit
The Baron 3.41 64-bit
CT800 1.12 64-bit
Soldat 4.0 64-bit
Pupsi2 0.09
Drosophila 1.4 64-bit
RuyDos 1.0.27 64-bit
Oberon 0.04
FireFly 2.7.2 64-bit
Dimitri 3.81
Sergio's Swiss Tournaments
Fire 6.1 64-bit 4CPU
40/4 FRC testing since the last update has included:
Nemorino 3.04 64-bit
CCRL 40/40, 40/4 and FRC lists updated (28th October 2017)
Moderators: hgm, Rebel, chrisw
-
- Posts: 41428
- Joined: Sun Feb 26, 2006 10:52 am
- Location: Auckland, NZ
CCRL 40/40, 40/4 and FRC lists updated (28th October 2017)
gbanksnz at gmail.com
-
- Posts: 2487
- Joined: Tue Aug 30, 2016 8:19 pm
- Full name: Rasmus Althoff
Re: CCRL 40/40, 40/4 and FRC lists updated (28th October 201
I'm surprised to see a drop of more than 40 Elo from CT800 1.11 to 1.12, that didn't show up during testing. Some questions:
1) do you use a standardised hash size (via the appropriate WB/UCI commands)? If no hash size is given, 1.12 will use the default of only 8 MB.
2) is the opening book handled in a different way for WB and UCI engines?
1) do you use a standardised hash size (via the appropriate WB/UCI commands)? If no hash size is given, 1.12 will use the default of only 8 MB.
2) is the opening book handled in a different way for WB and UCI engines?
-
- Posts: 41428
- Joined: Sun Feb 26, 2006 10:52 am
- Location: Auckland, NZ
Re: CCRL 40/40, 40/4 and FRC lists updated (28th October 201
Ras wrote:I'm surprised to see a drop of more than 40 Elo from CT800 1.11 to 1.12, that didn't show up during testing. Some questions:
1) do you use a standardised hash size (via the appropriate WB/UCI commands)? If no hash size is given, 1.12 will use the default of only 8 MB.
Hash size is set in the UCI options when creating the engine.
2) is the opening book handled in a different way for WB and UCI engines?
I use UCI, WB and Java engines and they all use the book similarly.
We use generic books.
gbanksnz at gmail.com
-
- Posts: 2487
- Joined: Tue Aug 30, 2016 8:19 pm
- Full name: Rasmus Althoff
Re: CCRL 40/40, 40/4 and FRC lists updated (28th October 201
I see, but what is the value of the hash size (in MB)?Graham Banks wrote:Hash size is set in the UCI options when creating the engine.
Ok, then I guess WB engines get the GUI opening book moves transmitted in forced mode so that it's pretty much the same as for UCI, logically seen.I use UCI, WB and Java engines and they all use the book similarly.
We use generic books.
-
- Posts: 4606
- Joined: Wed Oct 01, 2008 6:33 am
- Location: Regensburg, Germany
- Full name: Guenther Simon
Re: CCRL 40/40, 40/4 and FRC lists updated (28th October 201
from CCRL 40/4 full list:Ras wrote:I'm surprised to see a drop of more than 40 Elo from CT800 1.11 to 1.12, that didn't show up during testing. Some questions:
Code: Select all
CT800 1.11 64-bit 2240 +26 -26 45.8% +32.7 19.7% 532 54.7%
CT800 1.12 64-bit 2194 +33 -33 43.2% +49.8 19.7% 330 49.4%
NG-Play 9.86 64-bit 2187 +18 -18 50.5% -3.1 22.2% 1079 53.5%
above NG-Play? May be 1.11 was just a bit lucky so far and 1.12 will rise.
Of course a check of the games won't hurt.
Guenther
Edit: It seems the good result of 1.11 is primarily based on an extreme outsider result vs. Devel? (which produced strange outsider results so far en masse, if I see it right)
-
- Posts: 4606
- Joined: Wed Oct 01, 2008 6:33 am
- Location: Regensburg, Germany
- Full name: Guenther Simon
Re: CCRL 40/40, 40/4 and FRC lists updated (Devel problem?)
http://www.computerchess.org.uk/ccrl/40 ... vel_1_8090
There seem to be extreme fluctuations in the results of Devel 1.8090? (especially very bad performances which seem beyond statistical noise?)
Also the author estimated it above 2600?
http://www.develchessengine.no/
A pity that the games download per player for 40/4 only contains stripped off pgn.
I checked one game against CT800 1.11 and Devel seemed to play from a certain point on only depth 5 moves (according to my analysis) and lost in 26 moves.
CT800 1.11 also won several other games up to move 30 against Devel!
[pgn][Event "CCRL 40/4"]
[Site "CCRL"]
[Date "2017.10.10"]
[Round "395.5.611"]
[White "Devel 1.8090"]
[Black "CT800 1.11 64-bit"]
[Result "0-1"]
[ECO "D43"]
[Opening "QGD semi-Slav"]
[PlyCount "51"]
[WhiteElo "2457"]
[BlackElo "2240"]
1. d4 Nf6 2. c4 c6 3. Nf3 d5 4. Nc3 e6 5. Bg5 h6 6. Bxf6 Qxf6 7. e3 Nd7 8. cxd5
exd5 9. Bd3 Be7 10. O-O O-O 11. Qe2 Re8 12. e4 dxe4 13. Qxe4 Nf8 14. Ne5 Bd6
15. Rae1 Be6 16. a3 Rad8 17. Re3 Bc7 18. Rf3 Qe7 19. Qe3 c5 20. Nb5 cxd4 21.
Qf4 Bb8 22. Rg3 f6 23. Qxh6 fxe5 24. Qh5 a6 25. Nxd4 exd4 26. f4 0-1[/pgn]
Guenther
There seem to be extreme fluctuations in the results of Devel 1.8090? (especially very bad performances which seem beyond statistical noise?)
Also the author estimated it above 2600?
http://www.develchessengine.no/
A pity that the games download per player for 40/4 only contains stripped off pgn.
I checked one game against CT800 1.11 and Devel seemed to play from a certain point on only depth 5 moves (according to my analysis) and lost in 26 moves.
CT800 1.11 also won several other games up to move 30 against Devel!
[pgn][Event "CCRL 40/4"]
[Site "CCRL"]
[Date "2017.10.10"]
[Round "395.5.611"]
[White "Devel 1.8090"]
[Black "CT800 1.11 64-bit"]
[Result "0-1"]
[ECO "D43"]
[Opening "QGD semi-Slav"]
[PlyCount "51"]
[WhiteElo "2457"]
[BlackElo "2240"]
1. d4 Nf6 2. c4 c6 3. Nf3 d5 4. Nc3 e6 5. Bg5 h6 6. Bxf6 Qxf6 7. e3 Nd7 8. cxd5
exd5 9. Bd3 Be7 10. O-O O-O 11. Qe2 Re8 12. e4 dxe4 13. Qxe4 Nf8 14. Ne5 Bd6
15. Rae1 Be6 16. a3 Rad8 17. Re3 Bc7 18. Rf3 Qe7 19. Qe3 c5 20. Nb5 cxd4 21.
Qf4 Bb8 22. Rg3 f6 23. Qxh6 fxe5 24. Qh5 a6 25. Nxd4 exd4 26. f4 0-1[/pgn]
Guenther
-
- Posts: 3546
- Joined: Thu Jun 07, 2012 11:02 pm
Re: CCRL 40/40, 40/4 and FRC lists updated (Devel problem?)
Guenther wrote:http://www.computerchess.org.uk/ccrl/40 ... vel_1_8090
There seem to be extreme fluctuations in the results of Devel 1.8090? (especially very bad performances which seem beyond statistical noise?)
Yes that looks odd.
-
- Posts: 2487
- Joined: Tue Aug 30, 2016 8:19 pm
- Full name: Rasmus Althoff
Re: CCRL 40/40, 40/4 and FRC lists updated (28th October 201
That's also in there, yes.Guenther wrote:The error bars look still very high?
No, and you're right, that's also astonishing. I remember 1.11 at 2228, and somehow, it has risen to 2240. The playing style has changed considerably from NG-Play to CT800, but not the raw strength. At least not against engines as CT800 is designed to play humans.Did you expect version 1.11 that much above NG-Play?
I took a look at the current Amateur Division 8 where 1.12 only scored 0.5/4 in the first 4 rounds. Against Dorpsgek Dillinger, I've run extensive testing, and the usual score is around 50%. My direct 100 game match between 1.11 and 1.12 also ended up at around 50%.Of course a check of the games won't hurt.
The other spectacular loss with a king attack is to be expected, that has always been a weak side already in NG-Play, and if the CT doesn't get enough counterplay in the centre, it will be toast. Usually, the opening book avoids lines with blocked centre to cover this weakness, but if the internal book isn't used, that's the problem. The embedded version doesn't have this issue unless the opening book is disabled.
That would be a comforting explanation.. there's no big drop because there had been no big rise either.It seems the good result of 1.11 is primarily based on an extreme outsider result vs. Devel? (which produced strange outsider results so far en masse, if I see it right)
-
- Posts: 2283
- Joined: Sat Jun 02, 2012 2:13 am
Re: CCRL 40/40, 40/4 and FRC lists updated (28th October 201
Hi Rasmus,Ras wrote:
The other spectacular loss with a king attack is to be expected, that has always been a weak side already in NG-Play, and if the CT doesn't get enough counterplay in the centre, it will be toast. Usually, the opening book avoids lines with blocked centre to cover this weakness, but if the internal book isn't used, that's the problem. The embedded version doesn't have this issue unless the opening book is disabled.
Very interesting project you have there, developing an engine with anti-human play in mind. Have you considered making changes (such as king safety, pawn storms, etc) that improve its play in flank attacks and closed positions (even though that may cost Elo - after all it is the style that matters here)?
Avoiding closed games only skirts the issue and does not resolve it. A human player would greatly appreciate an engine opponent that can put up a good fight in closed positions, too.
Most developers stumble over the issue of Elo gains, leaving stylistic considerations on the back burner, with the end result being an engine that plays like a machine. If the goal is NOT to produce the strongest engine possible, then Elo should matter less than style, as long as the level of play remains adequately high (at least candidate master, let's say).
Interested to know your thoughts on this matter.
Cheers,
CL
-
- Posts: 2487
- Joined: Tue Aug 30, 2016 8:19 pm
- Full name: Rasmus Althoff
Re: CCRL 40/40, 40/4 and FRC lists updated (28th October 201
The cornerstones are: 1) preventing closed centres, both via the opening book and via the asymmetric pawn eval that scores blocked centres negative for the computer side. 2) in the root position, the material balance is recorded, and if piece exchanges happen somewhere down the tree, it is scored slightly negative for the engine. The idea is to keep the board full so that tactical complications arise, and of course this isn't helpful against other engines.carldaman wrote:Very interesting project you have there, developing an engine with anti-human play in mind.
Yes, I've tried that for opposite castling, but I couldn't make it work. The only real effect was slowing down the search. Well, from what I've read here in the forum, even the top-engines don't perform too well in these positions, and e.g. Stockfish is made by much more capable chess programmers than I will ever be. That's how I got the idea to cover the weaknesses instead of resolving them. The engine knows what positions it cannot handle and tries to avoid them.Have you considered making changes (such as king safety, pawn storms, etc) that improve its play in flank attacks and closed positions
One thing that I think I might still add is better king safety. Things like how many squares around the king are attacked/defended by how many pieces, that could work.
The PC version is mostly a testbed and also a nice goodie, but the "real" core is the embedded version running with a Cortex-M4 microcontroller and no operating system. Candidate master level is not quite in range with that hardware, but it should be a challenge for amateur players.then Elo should matter less than style, as long as the level of play remains adequately high (at least candidate master, let's say).
I think the good old Mephisto Vancouver is a bit stronger on average, but the CT800 is a dangerous opponent. Here a game at 30 minutes per side:
[pgn][White "CT800"]
[Black "Mephisto Vancouver 020"]
[Result "1-0"]
1. d4 d5 2. Nf3 Nf6 3. c4 dxc4 4. e3 e6 5. Bxc4 c5 6. O-O
a6 7. Qe2 b5 8. Bd3 cxd4 9. exd4 Bb7 10. a4 bxa4 11. Rxa4
Be7 12. Nc3 O-O 13. Be3 Qd6 14. Rfa1 Ng4 15. h3 Nf6 16. Bc4
Rc8 17. Ne5 Nfd7 18. Nxd7 Qxd7 19. Qg4 Rd8 20. Ne4 Qc6
21. Nc5 g6 22. Nxe6 fxe6 23. Bxe6+ Kg7 24. d5 Qd6 25. Qd4+
Bf6 26. Bh6+ Kxh6 27. Qxf6 Qh2+ 28. Kxh2 Rxd5 29. Bxd5 Bxd5
30. Rh4# 1-0[/pgn]