Alphazero news

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
hgm
Posts: 23004
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: Alphazero news

Post by hgm » Sun Dec 16, 2018 8:05 pm

Any engine has an opening repertoire by definition. Stockfish also prefers some opening moves over others, and would considtently play these.

noobpwnftw
Posts: 287
Joined: Sun Nov 08, 2015 10:10 pm

Re: Alphazero news

Post by noobpwnftw » Sun Dec 16, 2018 9:39 pm

I thought people are over with exploiting deterministic characteristics of the engines to measure their performance, but since yesterday it became a feature and was exaggerated to some extent.

jp
Posts: 320
Joined: Mon Apr 23, 2018 5:54 am

Re: Alphazero news

Post by jp » Sun Dec 16, 2018 10:06 pm

hgm wrote:
Sun Dec 16, 2018 8:05 pm
Any engine has an opening repertoire by definition. Stockfish also prefers some opening moves over others, and would considtently play these.
If you define "having an opening repertoire" as "playing deterministically", then yeah.
But that sort of "opening repertoire" may be a very bad one.

Uri Blass
Posts: 8334
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: Alphazero news

Post by Uri Blass » Sun Dec 16, 2018 10:25 pm

jp wrote:
Sun Dec 16, 2018 10:06 pm
hgm wrote:
Sun Dec 16, 2018 8:05 pm
Any engine has an opening repertoire by definition. Stockfish also prefers some opening moves over others, and would considtently play these.
If you define "having an opening repertoire" as "playing deterministically", then yeah.
But that sort of "opening repertoire" may be a very bad one.
Stockfish without book is not deterministic and may play different moves in the same position in the opening dependent on luck because of SMP

jp
Posts: 320
Joined: Mon Apr 23, 2018 5:54 am

Re: Alphazero news

Post by jp » Sun Dec 16, 2018 10:28 pm

Not 100% deterministic, but largely deterministic.

yanquis1972
Posts: 1762
Joined: Tue Jun 02, 2009 10:14 pm

Re: Alphazero news

Post by yanquis1972 » Sun Dec 16, 2018 11:14 pm

jp wrote:
Sun Dec 16, 2018 7:18 pm
yanquis1972 wrote:
Sun Dec 16, 2018 6:54 pm
depending on what 'x' is, that the book(or resultant TC advantage) is overly strong. as i've said, there's a massive difference between opening variety & simply outbooking the opponent. if you want to test the former it's vital to make sure you're not falling into the latter. so low-ply & with relatively high randomness as to moves selected, imo, is sensible.
Yeah, I'm not sure if we can make a good guess for what 'x' might be, but there's no way that SF+bookX is simply outbooking its opponent. If bookX were so great an advantage, Lc0+bookX would not have done so badly against SF no book, even if Lc0 did not like those openings.

Maybe some info on how many plies bookX goes would be good.

yanquis, if Kai gives the url to download bookX, could you run the test you want to see?

yanquis1972 wrote:
Sun Dec 16, 2018 6:54 pm
i don't really understand the LTC argument w/r/t lc0, provided the book is sufficiently shallow. a thoroughly trained NN has what is essentially an opening repertoire, so i agree it's essential to test it against a large variety. (much larger than deepmind did; as i said, i believe those openings are primarily important for benchmarking against A0)
Yeah, you could say that the NN engine is essentially outbooking its bookless opponent. The Deepmind argument for not giving its opponent a book is that neither has a book, but no one here is disagreeing that the NN engine obviously has an opening repertoire.
not sure, mostly curious about the strength & depth of it. i have no concrete idea about the test i want to see, just some nebulous theories.

the reason i can't hold any special weight to the Lc0+book result is that, again, it goes against principle. you're starting Lc0 in a 'foreign' position, even if it's believed to be (or objectively) superior.

i'd be curious how leela does against SF with a broad, shallow array of essentially random openings, &, should Lc0 match or outperform elo vs bookless SF, a solid (but still wide & similarly shallow) set.

rn i'm testing SF+perfect2017 vs rolling test30 nets, but at a fairly long TC, so results aren't significant yet.

jp
Posts: 320
Joined: Mon Apr 23, 2018 5:54 am

Re: Alphazero news

Post by jp » Sun Dec 16, 2018 11:25 pm

yanquis1972 wrote:
Sun Dec 16, 2018 11:14 pm
not sure, mostly curious about the strength & depth of it. i have no concrete idea about the test i want to see, just some nebulous theories.

the reason i can't hold any special weight to the Lc0+book result is that, again, it goes against principle. you're starting Lc0 in a 'foreign' position, even if it's believed to be (or objectively) superior.
Didn't you want e.g. SF+book vs bookless SF?
I don't know why bookless SF is so important to so many here, though.

If even normal, very superior positions were "foreign" to a NN engine, that would be a bit damning of the NN engine, wouldn't it?

yanquis1972
Posts: 1762
Joined: Tue Jun 02, 2009 10:14 pm

Re: Alphazero news

Post by yanquis1972 » Mon Dec 17, 2018 4:35 am

I don’t think so, necessarily. Given their purpose, and their nature, the first of which (to self-learn and excel at chess with limited resources) tends toward a machine of relatively limited scope but ultimately greater pure strength. With greater resources and refinement of techniques we should see more practical iterations.

For me Sf+book vs Sf is critical to understanding the basic significance of the elo difference. Ie Is stockfish +100 elo with it?

User avatar
hgm
Posts: 23004
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: Alphazero news

Post by hgm » Mon Dec 17, 2018 8:29 am

jp wrote:
Sun Dec 16, 2018 10:06 pm
If you define "having an opening repertoire" as "playing deterministically", then yeah.
But that sort of "opening repertoire" may be a very bad one.
So the meaningful thing to say is not that "AlphaZero has an opening repertoire", but "in contrast to AlphaZero, Stockfish' built-in opening repertoire sucks". Whether the line of reasoning "this is a weak spot of Stockfish, so it would not be fair to include it in any strength test" makes any sense... Well, I suppose anyone can judge that for himself.

noobpwnftw
Posts: 287
Joined: Sun Nov 08, 2015 10:10 pm

Re: Alphazero news

Post by noobpwnftw » Mon Dec 17, 2018 9:56 am

hgm wrote:
Mon Dec 17, 2018 8:29 am
jp wrote:
Sun Dec 16, 2018 10:06 pm
If you define "having an opening repertoire" as "playing deterministically", then yeah.
But that sort of "opening repertoire" may be a very bad one.
So the meaningful thing to say is not that "AlphaZero has an opening repertoire", but "in contrast to AlphaZero, Stockfish' built-in opening repertoire sucks". Whether the line of reasoning "this is a weak spot of Stockfish, so it would not be fair to include it in any strength test" makes any sense... Well, I suppose anyone can judge that for himself.
What do you think is the "proper" way to perform a strength test? Let's just say now I create a book, have it compiled into the engine, how can this kind of cheating get justified and the other engines cannot beat it from the starting position is because their "opening repertoire sucks"?

Maybe there is no bottom line, it's only the on-the-board performance you care after all.

Post Reply