1000 game match result

Discussion of chess software programming and technical issues.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
smcracraft
Posts: 645
Joined: Wed Mar 08, 2006 7:08 pm
Location: Orange County California
Full name: Stuart Cracraft
Contact:

1000 game match result

Post by smcracraft » Fri Dec 07, 2018 5:04 am


Joost Buijs
Posts: 772
Joined: Thu Jul 16, 2009 8:47 am
Location: Almere, The Netherlands

Re: 1000 game match result

Post by Joost Buijs » Fri Dec 07, 2018 11:00 am

When you look at these match results the AlphaZero surpasses Stockfish8 by approx. 52 Elo points, so it is probably in the same league as Stockfish9. There is no doubt that AlphaZero is stronger positionally, but it probably misses a lot in the realm of tactics and endgames. When you look at how simple Stockfish's evaluation function actually is, there is still a lot of improvement to be expected from that side as well.

I have the feeling that everything regarding ANN's and machine learning is a bit overhyped at the moment. I also wonder why DeepMind only allows you to download a selected subset of these 1000 games and not all of them. Probably because they not want you to look at games where the ANN didn't do so well.

User avatar
Ozymandias
Posts: 978
Joined: Sun Oct 25, 2009 12:30 am

Re: 1000 game match result

Post by Ozymandias » Fri Dec 07, 2018 6:33 pm

There was a lot of hype a while back, but these past few months were Lc0 has made no progress has somewhat toned down expectations.

In any case, I think this is were we can expect engines to make more of a headway. They're still in their infancy, and already SF looks like the only real challenge to overcome. That's quite the accomplishment.

Joost Buijs
Posts: 772
Joined: Thu Jul 16, 2009 8:47 am
Location: Almere, The Netherlands

Re: 1000 game match result

Post by Joost Buijs » Sat Dec 08, 2018 10:26 am

Ozymandias wrote:
Fri Dec 07, 2018 6:33 pm
There was a lot of hype a while back, but these past few months were Lc0 has made no progress has somewhat toned down expectations.

In any case, I think this is were we can expect engines to make more of a headway. They're still in their infancy, and already SF looks like the only real challenge to overcome. That's quite the accomplishment.
It is an accomplishment indeed that LC0 reached the level of Stockfish v9 after just 1 year of development while it took Stockfish 10 years. A lot of improvement is also due to better and faster hardware that exists nowadays, 10 years ago this just wasn't feasible.

Of course, both classic a-b engines and NN engines will continue to improve, and it is interesting to see which type of engine will get the upper hand in the future. It is my feeling that a hybrid approach, a combination of old and new tech, is needed to reach the highest level of play possible.

User avatar
Ozymandias
Posts: 978
Joined: Sun Oct 25, 2009 12:30 am

Re: 1000 game match result

Post by Ozymandias » Sat Dec 08, 2018 5:06 pm

Joost Buijs wrote:
Sat Dec 08, 2018 10:26 am
it is interesting to see which type of engine will get the upper hand in the future
Stockfish doesn't give much indication of faltering. This is the progress achieved (on a daily basis) over the past 6 years and change:

Image

Based on data collected from the CCRL 40/40 rating list:

Code: Select all

Stockfish 10 64-bit 4CPU  3485	+141  −107	301 days	53 points
Stockfish  9 64-bit 4CPU  3432	 +15   −15	457 days	42 points
Stockfish  8 64-bit 4CPU  3390   +19   −18	304 days	53 points
Stockfish  7 64-bit 4CPU  3337   +15   −15	340 days	35 points
Stockfish  6 64-bit 4CPU  3302   +14   −14	241 days	23 points
Stockfish  5 64-bit 4CPU  3279   +15   −15	183 days	35 points
Stockfish DD 64-bit 4CPU  3244   +17   −17	101 days	29 points
Stockfish  4 64-bit 4CPU  3215   +17   −17	112 days	40 points
Stockfish  3 64-bit 4CPU  3175   +21   −21	220 days	24 points

User avatar
Ozymandias
Posts: 978
Joined: Sun Oct 25, 2009 12:30 am

Re: 1000 game match result

Post by Ozymandias » Mon Dec 17, 2018 5:39 pm

Ozymandias wrote:
Sat Dec 08, 2018 5:06 pm

Based on data collected from the CCRL 40/40 rating list:

Code: Select all

Stockfish 10 64-bit 4CPU  3466		301 days	34 points
Stockfish  9 64-bit 4CPU  3432		457 days	54 points
Stockfish  8 64-bit 4CPU  3378   	304 days	51 points
Stockfish  7 64-bit 4CPU  3327   	340 days	35 points
Stockfish  6 64-bit 4CPU  3292   	241 days	23 points
Stockfish  5 64-bit 4CPU  3269   	183 days	34 points
Stockfish DD 64-bit 4CPU  3235   	101 days	29 points
Stockfish  4 64-bit 4CPU  3206   	112 days	39 points
Stockfish  3 64-bit 4CPU  3167   	220 days	24 points
With new data, comes a new graph, and now it looks like there's some stagnation after SF5. Only version 8 made good progress:

Image

Post Reply