AlphaZero vs Stockfish

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

Rodolfo Leoni
Posts: 545
Joined: Tue Jun 06, 2017 4:49 pm
Location: Italy

Re: AlphaZero vs Stockfish

Post by Rodolfo Leoni »

Tobber wrote:
Rodolfo Leoni wrote:
Leo wrote:
Milos wrote:
Uri Blass wrote:This is at 1 minute per move on 1 core.

The difference is going to be clearly smaller with 64 cores and
I see no evidence that book at that time control is additional +50 elo.
Maybe book does not give much at long time control because the program can often find better moves by itself.
Smaller yes, but not 1/5 of it, maybe 1/2 of it.
How I see it on 64 cores (we also don't know if these are cores or threads, since they don't say which machine they used, if it is cores Numa was not used, Large Pages were also not used):
SFdev vs SF8 at least 30Elo, hash up to 10ELO, EGTBs up to 10 ELO, large pages 5-10Elo, better SMP implementation of SFdev compared to SF8 for large number of cores (>32) 5-10Elo, Cerebellum book 30Elo against conventional engine, but against A0 most probably more, coz A0 is well trained for only small amount of openings, and good book would help SF a lot to get much more draws with black.
If you add all this together it is easy 100Elo if not even more.
I am beginning to think the Alpha team was afraid the results would have been a lot less impressive if they had beefed things up for SF.
The graph here show difference between SF8 and latest SF dev is more than 100 ELO.

https://nextchessmove.com/dev-builds

No it doesn't, read again.

/John
You're right... It scales 50 points per square and it confused me a bit. It could be around 60 ELOs. Thanks.
F.S.I. Chess Teacher
User avatar
hgm
Posts: 27789
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: AlphaZero vs Stockfish

Post by hgm »

Milos wrote:
hgm wrote:It is rather funny to see how the notion had Stockfish would be better if it was using a book seems to prevale. Imagine how strong a Chess player I would be, if I could make Kasparov do the moves for me...
Wouldn't help much if Kasparov would only make first 10 moves, you'd almost equally suck at it. :lol:
Only 10 moves? :shock: He must of course play many more. What good would it be if he played only so few moves that it made no difference?

How can it possibly be a fair test of my Chess abilities if I have to play my own moves? Or if Kasparov only plays so few moves that it makes me suck equally? A differece that makes no difference is no differece!
Uri Blass
Posts: 10269
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: AlphaZero vs Stockfish

Post by Uri Blass »

FWCC wrote:Alpha Zero can prob give a GM two minor odds and do favorably.
The perfect player probably is going to lose even against me with two minor odds(at least at 45+15 time control).

2 minors are a lot and
I believe that today even 1700 players can beat top programs with 2 minors odd with 45+15 time control
Milos
Posts: 4190
Joined: Wed Nov 25, 2009 1:47 am

Re: AlphaZero vs Stockfish

Post by Milos »

hgm wrote:
Milos wrote:
hgm wrote:It is rather funny to see how the notion had Stockfish would be better if it was using a book seems to prevale. Imagine how strong a Chess player I would be, if I could make Kasparov do the moves for me...
Wouldn't help much if Kasparov would only make first 10 moves, you'd almost equally suck at it. :lol:
Only 10 moves? :shock: He must of course play many more. What good would it be if he played only so few moves that it made no difference?

How can it possibly be a fair test of my Chess abilities if I have to play my own moves? Or if Kasparov only plays so few moves that it makes me suck equally? A differece that makes no difference is no differece!
That comparison is so ridiculously wrong that it is nothing but trolling, I mean you comparing yourself to SF and Kasparov to strong opening book.
But ofc, whenever you don't have any useful argument for the discussion you come up with some kind of "wittiness" that probably only you could call funny.
duncan
Posts: 12038
Joined: Mon Jul 07, 2008 10:50 pm

Re: AlphaZero vs Stockfish

Post by duncan »

what google's neural networks can achieve outside games. hope not too off topic.

https://www.theguardian.com/science/201 ... ler-google


For all their success with Kepler, Nasa scientists knew that more planets lay hidden in the telescope’s observations, but the signals were so weak they were difficult to spot. This is where Google’s AI researchers came in. By training a neural network to learn what bona fide signals of distant planets looked like, Christopher Shallue, a Google researcher, helped Nasa to scour Kepler’s observations of 670 stars for planets that had previously been missed.

The search turned up two new planets around different stars, Kepler 90i, and another world named Kepler 80g, the sixth planet now known to orbit its star. The scientists now plan to search Kepler’s data on all 150,000 stars for other missed planets. A research paper on the findings will be published by the Astronomical Journal.

..


The search turned up two new planets around different stars, Kepler 90i, and another world named Kepler 80g, the sixth planet now known to orbit its star. The scientists now plan to search Kepler’s data on all 150,000 stars for other missed planets. A research paper on the findings will be published by the Astronomical Journal.

Suzanne Aigrain, an astrophysicist at Oxford University who was not involved with the research, said: “What is perhaps most exciting is that they are able to find planets that were previously missed, suggesting there are more yet to be found using this approach.”

Earlier this year, Kepler scientists announced the discovery of 219 more candidate planets, of which 10 appeared to be about the same size and temperature as Earth.
FWCC
Posts: 117
Joined: Wed Aug 22, 2007 4:39 pm

Re: AlphaZero vs Stockfish

Post by FWCC »

What odds do YOU suggest? I was simply trying to state AlphaZero's uniqueness.
User avatar
hgm
Posts: 27789
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: AlphaZero vs Stockfish

Post by hgm »

Milos wrote:
hgm wrote:How can it possibly be a fair test of my Chess abilities if I have to play my own moves? Or if Kasparov only plays so few moves that it makes me suck equally? A differece that makes no difference is no differece!
That comparison is so ridiculously wrong that it is nothing but trolling, I mean you comparing yourself to SF and Kasparov to strong opening book.
But ofc, whenever you don't have any useful argument for the discussion you come up with some kind of "wittiness" that probably only you could call funny.
It is exactly as ridiculous/funny as this entire book discussion. It is claimed that one Chess-playig entity was significantly 'handicapped' because it had to play some of the moves itself, rather than having them played on its behalf by a completely different entity that is supposed to be better at them.
Spliffjiffer
Posts: 416
Joined: Thu Aug 02, 2012 7:48 pm
Location: Germany

Re: AlphaZero vs Stockfish

Post by Spliffjiffer »

Spliffjiffer wrote: Tue Dec 12, 2017 11:34 pm like everytime there is a "point of no return" and as far as i can tell 27...bxe4!?(instead of 27...bg6? ) was the last chance to head towards a draw imo:
[d]rn3r2/p3b1kp/2p5/1p3bp1/4B3/q1P1B1P1/P4P2/3RR1KQ b
bm 27...bxe4
...at least my efforts were leading nowhere.

my mainline:
1. Qxe4 Kg8 2. Bd4 Rf7 3. Qg4 Na6 4. Re5 Qd6 5. Re6 Qd5 6. Rde1 Nc7 7. R1e5 Qxe6 8. Rxe6 Nxe6 9. Qxe6 c5 10. Be3 Rd8 11. Qa6 b4 12. cxb4 cxb4 13. Qxa7 Rd6...its a draw imo
Today i read an article about the Alphazero vs SF8 match and the author considered this game as the best chessgame ever played in chesshistory...so i gave this game another analysis while beeing especially interested in the progress that the NNUE-era offers and i think i must revise my statement from above a bit: while im still thinking that 27...bxe4 !? is still a draw, i do not believe anymore that it was the last chance for SF to draw this game !?
the last mistake that threw away the draw was probably 30...Qxa2??! while 30...Rh8!? would have been probably still enough to draw:
[d]rn3r2/p5k1/2p2bp1/1p4p1/8/q1P1B1PQ/P4PK1/3RR3 b - - 3 30
bm Rh8!?...dont be too greedy :-)
for example: 30...Rh8!? 31.Qg4 Rf8 32.Bd4 Qd6 33. Be5 Qxe5 34.Rxe5 Bxe5 35.Rh1 Rf5 36.Qh3 Kf8 37.Rd1 Na6 38.Qh6+ Bg7 (maybe 38.Rd7 might be worth a try, i did not investigate here) 39. Qxg6 Rf6 40.Qxg5 Nc7 41.Qc5+ Kg8 42.Rd6 Raf8 43.f4 Rxd6 44.Qxd6 Nd5 45.Qxc6 Rd8 (Nxc3 should draw as well !?) 46.Qxb5 Bxc3 we have this position
[d]3r2k1/p7/8/1Q1n4/5P2/2b3P1/P5K1/8 w - - 0 47
eventually white will push his g-pawn to g8 and black will sac his knight against him, meanwhile black gets his pawn to the 6th and the resulting 8men endgame is pretty much sure completely drawn by letting the rook on the 6th and protect f6 together with the bishop !!?

as always im very pleased when you will prove me wrong and give some improvement for the white side :-)
greetings, Peer
Wahrheiten sind Illusionen von denen wir aber vergessen haben dass sie welche sind.
OneTrickPony
Posts: 157
Joined: Tue Apr 30, 2013 1:29 am

Re: AlphaZero vs Stockfish

Post by OneTrickPony »

It is exactly as ridiculous/funny as this entire book discussion. It is claimed that one Chess-playig entity was significantly 'handicapped' because it had to play some of the moves itself, rather than having them played on its behalf by a completely different entity that is supposed to be better at them.
There was a common expectation for SF to have an opening book to be strong from starting position. All the tournament/rating list/playchess engine room/correspondence w/e games were played with knowledge that you have to have an opening book if you want to be competitive. The common accepted view was that it's better for the book to come from 3rd parties instead of SF team trying to come up with one. SF testing itself is done with an opening book.
The whole situation is akin to someone claiming they can beat an F1 car and then running the tests on a muddy rally track. Yeah, they won but they didn't get what F1 car is good at. It was never optimized for it the same way chess engines were never optimized to be as strong as possible from the starting position.

It's funny how this completely obvious point is discussed why the topic of time control is rarely considered. Time is part of the game and the expectation for chess engines was that they do time management themselves. The tournament/rating list/playchess etc. games were played in conditions where the chess engines had to manage their time and that's why SF team spent some time to implement it unlike the opening book.

It's sad that the publicity stunt Google pulled seems to work. Defending them you are contributing to the terrible academic and corporate culture where deceiving the wide public is more profitable than intellectual honesty. Just ask yourself this: if I claimed I can beat your engine in a match you would want to establish the terms. You would ask about opening books and tablebases and we would establish something. Being honest people we wouldn't modify our engines to hide opening book in them. If we didn't trust each other we would agree on executable size for example and then do w/e we could within those constraints.

Btw, Deepmind recently claimed to solve (for practical purposes) protein folding. Unlike chess I don't know anything about that. Trying to widen my understanding of the world I would like to know if they really solved it, if their progress is really meaningful and how big a deal it is. Remembering how intellectually dishonest PR stunt they pulled with one of the oldest computer science challenges I am very skeptical to accept their claims so should be everyone else. The stakes with this one are actually quite high unlike chess and we all know people who only deceive when it doesn't matter are a rare breed.

What's even worse I can't even trust what experts in the field say seeing how supposed experts in chess programming field are so high on DeepMind's koolaid even 2-3 years after the event.
Madeleine Birchfield
Posts: 512
Joined: Tue Sep 29, 2020 4:29 pm
Location: Dublin, Ireland
Full name: Madeleine Birchfield

Re: AlphaZero vs Stockfish

Post by Madeleine Birchfield »

OneTrickPony wrote: Tue Dec 08, 2020 6:46 am There was a common expectation for SF to have an opening book to be strong from starting position. All the tournament/rating list/playchess engine room/correspondence w/e games were played with knowledge that you have to have an opening book if you want to be competitive. The common accepted view was that it's better for the book to come from 3rd parties instead of SF team trying to come up with one. SF testing itself is done with an opening book.

The whole situation is akin to someone claiming they can beat an F1 car and then running the tests on a muddy rally track. Yeah, they won but they didn't get what F1 car is good at. It was never optimized for it the same way chess engines were never optimized to be as strong as possible from the starting position.

It's funny how this completely obvious point is discussed why the topic of time control is rarely considered. Time is part of the game and the expectation for chess engines was that they do time management themselves. The tournament/rating list/playchess etc. games were played in conditions where the chess engines had to manage their time and that's why SF team spent some time to implement it unlike the opening book.

It's sad that the publicity stunt Google pulled seems to work. Defending them you are contributing to the terrible academic and corporate culture where deceiving the wide public is more profitable than intellectual honesty. Just ask yourself this: if I claimed I can beat your engine in a match you would want to establish the terms. You would ask about opening books and tablebases and we would establish something. Being honest people we wouldn't modify our engines to hide opening book in them. If we didn't trust each other we would agree on executable size for example and then do w/e we could within those constraints.

Btw, Deepmind recently claimed to solve (for practical purposes) protein folding. Unlike chess I don't know anything about that. Trying to widen my understanding of the world I would like to know if they really solved it, if their progress is really meaningful and how big a deal it is. Remembering how intellectually dishonest PR stunt they pulled with one of the oldest computer science challenges I am very skeptical to accept their claims so should be everyone else. The stakes with this one are actually quite high unlike chess and we all know people who only deceive when it doesn't matter are a rare breed.

What's even worse I can't even trust what experts in the field say seeing how supposed experts in chess programming field are so high on DeepMind's koolaid even 2-3 years after the event.
You are responding to a 3 year old comment.