Alphazero news

[hgm]

Any engine has an opening repertoire by definition. Stockfish also prefers some opening moves over others, and would considtently play these.

[noobpwnftw]

I thought people are over with exploiting deterministic characteristics of the engines to measure their performance, but since yesterday it became a feature and was exaggerated to some extent.

[jp]

Any engine has an opening repertoire by definition. Stockfish also prefers some opening moves over others, and would considtently play these.

If you define "having an opening repertoire" as "playing deterministically", then yeah.
But that sort of "opening repertoire" may be a very bad one.

[Uri Blass]

Any engine has an opening repertoire by definition. Stockfish also prefers some opening moves over others, and would considtently play these.

If you define "having an opening repertoire" as "playing deterministically", then yeah.
But that sort of "opening repertoire" may be a very bad one.

Stockfish without book is not deterministic and may play different moves in the same position in the opening dependent on luck because of SMP

[jp]

Not 100% deterministic, but largely deterministic.

[yanquis1972]

depending on what 'x' is, that the book(or resultant TC advantage) is overly strong. as i've said, there's a massive difference between opening variety & simply outbooking the opponent. if you want to test the former it's vital to make sure you're not falling into the latter. so low-ply & with relatively high randomness as to moves selected, imo, is sensible.

Yeah, I'm not sure if we can make a good guess for what 'x' might be, but there's no way that SF+bookX is simply outbooking its opponent. If bookX were so great an advantage, Lc0+bookX would not have done so badly against SF no book, even if Lc0 did not like those openings.

Maybe some info on how many plies bookX goes would be good.

yanquis, if Kai gives the url to download bookX, could you run the test you want to see?

i don't really understand the LTC argument w/r/t lc0, provided the book is sufficiently shallow. a thoroughly trained NN has what is essentially an opening repertoire, so i agree it's essential to test it against a large variety. (much larger than deepmind did; as i said, i believe those openings are primarily important for benchmarking against A0)

Yeah, you could say that the NN engine is essentially outbooking its bookless opponent. The Deepmind argument for not giving its opponent a book is that neither has a book, but no one here is disagreeing that the NN engine obviously has an opening repertoire.

not sure, mostly curious about the strength & depth of it. i have no concrete idea about the test i want to see, just some nebulous theories.

the reason i can't hold any special weight to the Lc0+book result is that, again, it goes against principle. you're starting Lc0 in a 'foreign' position, even if it's believed to be (or objectively) superior.

i'd be curious how leela does against SF with a broad, shallow array of essentially random openings, &, should Lc0 match or outperform elo vs bookless SF, a solid (but still wide & similarly shallow) set.

rn i'm testing SF+perfect2017 vs rolling test30 nets, but at a fairly long TC, so results aren't significant yet.

[jp]

not sure, mostly curious about the strength & depth of it. i have no concrete idea about the test i want to see, just some nebulous theories.

the reason i can't hold any special weight to the Lc0+book result is that, again, it goes against principle. you're starting Lc0 in a 'foreign' position, even if it's believed to be (or objectively) superior.

Didn't you want e.g. SF+book vs bookless SF?
I don't know why bookless SF is so important to so many here, though.

If even normal, very superior positions were "foreign" to a NN engine, that would be a bit damning of the NN engine, wouldn't it?

[yanquis1972]

I don’t think so, necessarily. Given their purpose, and their nature, the first of which (to self-learn and excel at chess with limited resources) tends toward a machine of relatively limited scope but ultimately greater pure strength. With greater resources and refinement of techniques we should see more practical iterations.

For me Sf+book vs Sf is critical to understanding the basic significance of the elo difference. Ie Is stockfish +100 elo with it?

[hgm]

If you define "having an opening repertoire" as "playing deterministically", then yeah.
But that sort of "opening repertoire" may be a very bad one.

So the meaningful thing to say is not that "AlphaZero has an opening repertoire", but "in contrast to AlphaZero, Stockfish' built-in opening repertoire sucks". Whether the line of reasoning "this is a weak spot of Stockfish, so it would not be fair to include it in any strength test" makes any sense... Well, I suppose anyone can judge that for himself.

[noobpwnftw]

If you define "having an opening repertoire" as "playing deterministically", then yeah.
But that sort of "opening repertoire" may be a very bad one.

So the meaningful thing to say is not that "AlphaZero has an opening repertoire", but "in contrast to AlphaZero, Stockfish' built-in opening repertoire sucks". Whether the line of reasoning "this is a weak spot of Stockfish, so it would not be fair to include it in any strength test" makes any sense... Well, I suppose anyone can judge that for himself.

What do you think is the "proper" way to perform a strength test? Let's just say now I create a book, have it compiled into the engine, how can this kind of cheating get justified and the other engines cannot beat it from the starting position is because their "opening repertoire sucks"?

Maybe there is no bottom line, it's only the on-the-board performance you care after all.