Alphazero news
Moderators: hgm, Rebel, chrisw
-
- Posts: 27796
- Joined: Fri Mar 10, 2006 10:06 am
- Location: Amsterdam
- Full name: H G Muller
Re: Alphazero news
Any engine has an opening repertoire by definition. Stockfish also prefers some opening moves over others, and would considtently play these.
-
- Posts: 560
- Joined: Sun Nov 08, 2015 11:10 pm
Re: Alphazero news
I thought people are over with exploiting deterministic characteristics of the engines to measure their performance, but since yesterday it became a feature and was exaggerated to some extent.
-
- Posts: 1470
- Joined: Mon Apr 23, 2018 7:54 am
Re: Alphazero news
If you define "having an opening repertoire" as "playing deterministically", then yeah.
But that sort of "opening repertoire" may be a very bad one.
-
- Posts: 10282
- Joined: Thu Mar 09, 2006 12:37 am
- Location: Tel-Aviv Israel
Re: Alphazero news
Stockfish without book is not deterministic and may play different moves in the same position in the opening dependent on luck because of SMP
-
- Posts: 1470
- Joined: Mon Apr 23, 2018 7:54 am
Re: Alphazero news
Not 100% deterministic, but largely deterministic.
-
- Posts: 1766
- Joined: Wed Jun 03, 2009 12:14 am
Re: Alphazero news
not sure, mostly curious about the strength & depth of it. i have no concrete idea about the test i want to see, just some nebulous theories.jp wrote: ↑Sun Dec 16, 2018 8:18 pmYeah, I'm not sure if we can make a good guess for what 'x' might be, but there's no way that SF+bookX is simply outbooking its opponent. If bookX were so great an advantage, Lc0+bookX would not have done so badly against SF no book, even if Lc0 did not like those openings.yanquis1972 wrote: ↑Sun Dec 16, 2018 7:54 pm depending on what 'x' is, that the book(or resultant TC advantage) is overly strong. as i've said, there's a massive difference between opening variety & simply outbooking the opponent. if you want to test the former it's vital to make sure you're not falling into the latter. so low-ply & with relatively high randomness as to moves selected, imo, is sensible.
Maybe some info on how many plies bookX goes would be good.
yanquis, if Kai gives the url to download bookX, could you run the test you want to see?
Yeah, you could say that the NN engine is essentially outbooking its bookless opponent. The Deepmind argument for not giving its opponent a book is that neither has a book, but no one here is disagreeing that the NN engine obviously has an opening repertoire.yanquis1972 wrote: ↑Sun Dec 16, 2018 7:54 pm i don't really understand the LTC argument w/r/t lc0, provided the book is sufficiently shallow. a thoroughly trained NN has what is essentially an opening repertoire, so i agree it's essential to test it against a large variety. (much larger than deepmind did; as i said, i believe those openings are primarily important for benchmarking against A0)
the reason i can't hold any special weight to the Lc0+book result is that, again, it goes against principle. you're starting Lc0 in a 'foreign' position, even if it's believed to be (or objectively) superior.
i'd be curious how leela does against SF with a broad, shallow array of essentially random openings, &, should Lc0 match or outperform elo vs bookless SF, a solid (but still wide & similarly shallow) set.
rn i'm testing SF+perfect2017 vs rolling test30 nets, but at a fairly long TC, so results aren't significant yet.
-
- Posts: 1470
- Joined: Mon Apr 23, 2018 7:54 am
Re: Alphazero news
Didn't you want e.g. SF+book vs bookless SF?yanquis1972 wrote: ↑Mon Dec 17, 2018 12:14 am not sure, mostly curious about the strength & depth of it. i have no concrete idea about the test i want to see, just some nebulous theories.
the reason i can't hold any special weight to the Lc0+book result is that, again, it goes against principle. you're starting Lc0 in a 'foreign' position, even if it's believed to be (or objectively) superior.
I don't know why bookless SF is so important to so many here, though.
If even normal, very superior positions were "foreign" to a NN engine, that would be a bit damning of the NN engine, wouldn't it?
-
- Posts: 1766
- Joined: Wed Jun 03, 2009 12:14 am
Re: Alphazero news
I don’t think so, necessarily. Given their purpose, and their nature, the first of which (to self-learn and excel at chess with limited resources) tends toward a machine of relatively limited scope but ultimately greater pure strength. With greater resources and refinement of techniques we should see more practical iterations.
For me Sf+book vs Sf is critical to understanding the basic significance of the elo difference. Ie Is stockfish +100 elo with it?
For me Sf+book vs Sf is critical to understanding the basic significance of the elo difference. Ie Is stockfish +100 elo with it?
-
- Posts: 27796
- Joined: Fri Mar 10, 2006 10:06 am
- Location: Amsterdam
- Full name: H G Muller
Re: Alphazero news
So the meaningful thing to say is not that "AlphaZero has an opening repertoire", but "in contrast to AlphaZero, Stockfish' built-in opening repertoire sucks". Whether the line of reasoning "this is a weak spot of Stockfish, so it would not be fair to include it in any strength test" makes any sense... Well, I suppose anyone can judge that for himself.
-
- Posts: 560
- Joined: Sun Nov 08, 2015 11:10 pm
Re: Alphazero news
What do you think is the "proper" way to perform a strength test? Let's just say now I create a book, have it compiled into the engine, how can this kind of cheating get justified and the other engines cannot beat it from the starting position is because their "opening repertoire sucks"?hgm wrote: ↑Mon Dec 17, 2018 9:29 amSo the meaningful thing to say is not that "AlphaZero has an opening repertoire", but "in contrast to AlphaZero, Stockfish' built-in opening repertoire sucks". Whether the line of reasoning "this is a weak spot of Stockfish, so it would not be fair to include it in any strength test" makes any sense... Well, I suppose anyone can judge that for himself.
Maybe there is no bottom line, it's only the on-the-board performance you care after all.