CCRL 40/40, 40/4 and FRC lists updated (16th February 2019)

Discussion of computer chess matches and engine tournaments.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
hgm
Posts: 24109
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: CCRL 40/40, 40/4 and FRC lists updated (16th February 2019)

Post by hgm » Sun Feb 24, 2019 8:50 am

Who cares if you read it? The reason I post here is to prevent you from misleading others by all your false claims.

As I said, you really have no clue at all how AZ/LZ work, and the totally wrong interpretation of the paragraphs of the AZ paper you quote confirm that. What the quoted text really says is that AlphaZero uses a neural network as evaluation function, but that initially the parameters in this function are set to random values. The subsequent tuning (with self-play games in case of AZ/LZ, with GM games in case of Deus X) of the eval parameters then makes this a very good evaluation.

This is not different from Texel tuning of eval parameters, something that many conventional alpha-beta engines do. Any engine would play like a random mover if you set all its eval parameters to zero or a random value. That doesn't mean it had no evaluation, just that the evaluation is crap before it gets tuned.

chrisw
Posts: 2783
Joined: Tue Apr 03, 2012 2:28 pm

Re: CCRL 40/40, 40/4 and FRC lists updated (16th February 2019)

Post by chrisw » Sun Feb 24, 2019 9:37 am

hgm wrote:
Sun Feb 24, 2019 8:50 am
Who cares if you read it? The reason I post here is to prevent you from misleading others by all your false claims.

As I said, you really have no clue at all how AZ/LZ work, and the totally wrong interpretation of the paragraphs of the AZ paper you quote confirm that. What the quoted text really says is that AlphaZero uses a neural network as evaluation function, but that initially the parameters in this function are set to random values. The subsequent tuning (with self-play games in case of AZ/LZ, with GM games in case of Deus X) of the eval parameters then makes this a very good evaluation.

This is not different from Texel tuning of eval parameters, something that many conventional alpha-beta engines do. Any engine would play like a random mover if you set all its eval parameters to zero or a random value. That doesn't mean it had no evaluation, just that the evaluation is crap before it gets tuned.
I'm afraid you are the one in error here, Ed is correct.

You call it an "evaluation" but that is an anthropomorphisation, an attempt to understand using human and classical chess program parallels.

Ed is correct when he says the output is a probability. AZ gives a probability measure of how many times this situation worked out as opposed to didn't work out, and for its policy a measure of how many times this move got tried as opposed to didn't.

And you are wrong to say it is just like Texel tuning. Texel tuning is done on an array of pre-computed, hand crafted features.

Anyway, call it an "evaluation" if that helps your brain (that actually doesn't understand how it works, a phrase you are fond of hurling at people) if that helps you "understand". But calling it an "evaluation" actually means that it's you that doesn't "understand".

User avatar
Kotlov
Posts: 215
Joined: Fri Jul 10, 2015 7:23 pm
Location: Russia

Re: CCRL 40/40, 40/4 and FRC lists updated (16th February 2019)

Post by Kotlov » Sun Feb 24, 2019 10:07 am

HGM are absolutely right.
Eugene Kotlov
Hedgehog 2.0 64-bit coming soon...

User avatar
hgm
Posts: 24109
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: CCRL 40/40, 40/4 and FRC lists updated (16th February 2019)

Post by hgm » Sun Feb 24, 2019 12:32 pm

Evaluation is BY DEFINITION what you do in the leaves of a tree search where the game has not yet ended by rule (such as mate), to heuristically calculate a number that expresses the relative desirability of the position (assuming the goal of both players is to win). A unique (i.e. single-valued) mapping into the set of numbers is the mathematical definition of the concept 'function'. The unique value delivered by the NN's 'value head' in response to the board position and a few predecessors (together forming the current game state) thus is an evaluation function.

One would think even an idiot could understand this...

BTW, (and not that this matters...), the terms in AZ's NN are also hand-crafted: the topology of the NN (number of layers; which cells to connect with which, what non-linear transformation to apply to the sum of inputs in each layer) was consciously designed. Only difference is that this design did not take any game-specific knowledge into account, but is 'brute force', in the sence that you use such an overwhelmingly large number of terms that those terms that would be useful for a particular game are amongst those.

Any conventional evaluation can in fact be implemented as a neural net, typically with some 10-1000 neurons. But if you want such small networks to be any good you should indeed very carefully design the topology of their connections, to match the concepts that are relevant for the game. But throw in a couple of million neurons, interconnected as a multi-layer stack with an enormously high connectivity, and each topology you could desire will be a subset of that. Which can be 'instantiated' (through training) by giving large weights to the desired connections, and near-zero weights to the connections with cells used for other, independent purposes
chrisw wrote:
Sun Feb 24, 2019 9:37 am
Ed is correct when he says the output is a probability. AZ gives a probability measure of how many times this situation worked out as opposed to didn't work out, and for its policy a measure of how many times this move got tried as opposed to didn't.
Yes, it is a probability, so what? No one ever claimed anything else. It doesn't mater how you distort the numerical scale in which you express the desirability of the position, as long as you preserve the order. The rest of your claim is pure nonsense; in virtually all cases AZ would have never seen that particular position before, so how many times it worked out or not, or which moves could try would all be at a big fat zero. The probability it outputs is just an expectation on how well it thinks this position is going to do, based on features of it (such as material balance, pawn structure, king safety) it recognizes, and was tuned to guage their importance. Just as conventional evaluations do.

chrisw
Posts: 2783
Joined: Tue Apr 03, 2012 2:28 pm

Re: CCRL 40/40, 40/4 and FRC lists updated (16th February 2019)

Post by chrisw » Sun Feb 24, 2019 4:24 pm

hgm wrote:
Sun Feb 24, 2019 12:32 pm
Evaluation is BY DEFINITION what you do in the leaves of a tree search where the game has not yet ended by rule (such as mate), to heuristically calculate a number that expresses the relative desirability of the position (assuming the goal of both players is to win). A unique (i.e. single-valued) mapping into the set of numbers is the mathematical definition of the concept 'function'. The unique value delivered by the NN's 'value head' in response to the board position and a few predecessors (together forming the current game state) thus is an evaluation function.

One would think even an idiot could understand this...

BTW, (and not that this matters...), the terms in AZ's NN are also hand-crafted: the topology of the NN (number of layers; which cells to connect with which, what non-linear transformation to apply to the sum of inputs in each layer) was consciously designed. Only difference is that this design did not take any game-specific knowledge into account, but is 'brute force', in the sence that you use such an overwhelmingly large number of terms that those terms that would be useful for a particular game are amongst those.

Any conventional evaluation can in fact be implemented as a neural net, typically with some 10-1000 neurons. But if you want such small networks to be any good you should indeed very carefully design the topology of their connections, to match the concepts that are relevant for the game. But throw in a couple of million neurons, interconnected as a multi-layer stack with an enormously high connectivity, and each topology you could desire will be a subset of that. Which can be 'instantiated' (through training) by giving large weights to the desired connections, and near-zero weights to the connections with cells used for other, independent purposes
chrisw wrote:
Sun Feb 24, 2019 9:37 am
Ed is correct when he says the output is a probability. AZ gives a probability measure of how many times this situation worked out as opposed to didn't work out, and for its policy a measure of how many times this move got tried as opposed to didn't.
Yes, it is a probability, so what?
Haha. The "so what" is in the part you snipped away, so as to be able to post "so what?".

Look, I actually posted in objection to your gratuitous outburst of rudeness (although it's to be noted Ed has now blocked you, again. Not the first from you, is it?). There's no reason to be calling people idiots or telling them they know nothing, or that you are posting to "correct their nonsense", in particular when you are marked as here as "moderator". I'm posting in objection to your repeated sudden personal rudenesses in what were purely technical threads with no personal stuff involved, prior.
Needless to say, you didn't get that, so carried on being "technical" with rudeness attached. So here it is in black and white. Quit with the gratuitous rude comments. Okay? You can't "moderate" yourself, I'll do it for you. Goodbye. Have a nice day.

No one ever claimed anything else. It doesn't mater how you distort the numerical scale in which you express the desirability of the position, as long as you preserve the order. The rest of your claim is pure nonsense; in virtually all cases AZ would have never seen that particular position before, so how many times it worked out or not, or which moves could try would all be at a big fat zero. The probability it outputs is just an expectation on how well it thinks this position is going to do, based on features of it (such as material balance, pawn structure, king safety) it recognizes, and was tuned to guage their importance. Just as conventional evaluations do.

User avatar
xr_a_y
Posts: 984
Joined: Sat Nov 25, 2017 1:28 pm
Location: France

Re: CCRL 40/40, 40/4 and FRC lists updated (16th February 2019)

Post by xr_a_y » Sun Feb 24, 2019 5:22 pm

I may have one question on this subject, maybe so that what was said before is now summarized.

- Learning during the test is disable. This is fine, fair, and make the testing process able to "converge". Got it.
- Own books are disable. OK.
- NN engines can use some static data on the file system. OK.

What if a not-NN engine want to use a file (that will not evolve during the test of course) with some interesting data in it, that is not a so-called book, or EGtable, but something that "helps" its evaluation ? And what if this thing is activated even in the opening ? will it be allowed ? Of course this file would probably be created by a learning process but would be read-only during the testing process. Is that OK ? Will it have a limited size ?

User avatar
hgm
Posts: 24109
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: CCRL 40/40, 40/4 and FRC lists updated (16th February 2019)

Post by hgm » Sun Feb 24, 2019 5:40 pm

If its not a table with information on individual positions (i.e. a book), it would be allowed. Whether it is in a separate file, or included as initialized data in the engine binary is completely immaterial.

User avatar
hgm
Posts: 24109
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: CCRL 40/40, 40/4 and FRC lists updated (16th February 2019)

Post by hgm » Sun Feb 24, 2019 5:43 pm

chrisw wrote:
Sun Feb 24, 2019 4:24 pm
Haha. The "so what" is in the part you snipped away, so as to be able to post "so what?".
As usually you have nothing of substance to contribute, and are just trolling to stir up trouble. It is not even clear why you would think you would qualify for an idiot, as you obviously are denying what I claim any idiot would understand... There is only meaningless babble in the parts I snipped away...

chrisw
Posts: 2783
Joined: Tue Apr 03, 2012 2:28 pm

Re: CCRL 40/40, 40/4 and FRC lists updated (16th February 2019)

Post by chrisw » Sun Feb 24, 2019 7:02 pm

hgm wrote:
Sun Feb 24, 2019 5:43 pm
chrisw wrote:
Sun Feb 24, 2019 4:24 pm
Haha. The "so what" is in the part you snipped away, so as to be able to post "so what?".
As usually you have nothing of substance to contribute, and are just trolling to stir up trouble. It is not even clear why you would think you would qualify for an idiot, as you obviously are denying what I claim any idiot would understand... There is only meaningless babble in the parts I snipped away...
No. And there is no reason why should be allowed to get away with this.

Ed regards you as a stalker. He doesn't know what impels you to do it, but you do it - on him. He quit these forums some months ago, precisely because of your stalking, as you well know. Ed returned to posting recently. He is a mild-mannered guy and he posts quiet technical, often helpful things. He posted here, and you came again with the old, old stuff that you do, that he regards as rude. It is rude. He has put you on block, as you also know.

I'm not so mild mannered, as you know. You drove him off the forum once. And now you are doing it again. If you don't get the message to stop stalking the guy, stop being rude to him with absolutely no provocation, who is going to stop you? Ed has made his limits very clear, but you continue to breach them. It's not useful complaining to moderators, you are a moderator. So I'm going to stop you. Okay? Just pack it in. Stop with the gratuitous insults, or take them to CTF or something, but when you are moderator on a forum, you need to behave, you've had enough signals that you're being a problem. Stop. Goodbye and have a nice day.

User avatar
Rebel
Posts: 4986
Joined: Thu Aug 18, 2011 10:04 am

Re: CCRL 40/40, 40/4 and FRC lists updated (16th February 2019)

Post by Rebel » Sun Feb 24, 2019 9:00 pm

Thanks Chris.

HGM, what happened in this thread was not so bad, it just was the (well known) last drop. During the last years you (hardly) without any exception seek the confrontation with me and I am tired of it. Look, we are about the same age and really 2 old men fighting is just pathetic! Nevertheless, be well.
90% of coding is debugging, the other 10% is writing bugs.

Post Reply