Comparative nodes per second

[mcostalba]

But as I said, I don't know what to do about that! If you modify the score to account for that you in a feedback loop, you would change the root score. If a system could be worked out I believe there would be many opportunities based on things going on in the tree.

I have thought about this at lenght, for instance it is well known that starting from a winning position evaluation increases with depth in an almost monotonic way simply because, being a winning position if both players play "well" the winning one accumulates advantage with each move, this translates in the evaluation to go up after ID loop. Unfortunatly I am (still) not able to code this in something useful, but I think that a kind of feedback on root eval based not only on evaluation at ply n, but at ply n-1, ..., n-m could be really a source of big potential in ELO terms for who is able to exploit it.

[BubbaTough]

But as I said, I don't know what to do about that! If you modify the score to account for that you in a feedback loop, you would change the root score. If a system could be worked out I believe there would be many opportunities based on things going on in the tree.

I have thought about this at lenght, for instance it is well known that starting from a winning position evaluation increases with depth in an almost monotonic way simply because, being a winning position if both players play "well" the winning one accumulates advantage with each move, this translates in the evaluation to go up after ID loop. Unfortunatly I am (still) not able to code this in something useful, but I think that a kind of feedback on root eval based not only on evaluation at ply n, but at ply n-1, ..., n-m could be really a source of big potential in ELO terms for who is able to exploit it.

A friend suggested something similar to me maybe 8 years ago. I have always been extremely doubtful of its potential, and had some great arguments against it at the time, that I have since forgotten .

-Sam

[michiguel]

But as I said, I don't know what to do about that! If you modify the score to account for that you in a feedback loop, you would change the root score. If a system could be worked out I believe there would be many opportunities based on things going on in the tree.

I have thought about this at lenght, for instance it is well known that starting from a winning position evaluation increases with depth in an almost monotonic way simply because, being a winning position if both players play "well" the winning one accumulates advantage with each move, this translates in the evaluation to go up after ID loop. Unfortunatly I am (still) not able to code this in something useful, but I think that a kind of feedback on root eval based not only on evaluation at ply n, but at ply n-1, ..., n-m could be really a source of big potential in ELO terms for who is able to exploit it.

A friend suggested something similar to me maybe 8 years ago. I have always been extremely doubtful of its potential, and had some great arguments against it at the time, that I have since forgotten .

-Sam

That is what I do with checkmate scores, but HGM does it more generally in one of his engines. Since I do this with checkmates, I do not need to do any correction with checkmate scores with they are saved in the hashtable. I just save the checkmate score.

Miguel

[Don]

But as I said, I don't know what to do about that! If you modify the score to account for that you in a feedback loop, you would change the root score. If a system could be worked out I believe there would be many opportunities based on things going on in the tree.

I have thought about this at lenght, for instance it is well known that starting from a winning position evaluation increases with depth in an almost monotonic way simply because, being a winning position if both players play "well" the winning one accumulates advantage with each move, this translates in the evaluation to go up after ID loop. Unfortunatly I am (still) not able to code this in something useful, but I think that a kind of feedback on root eval based not only on evaluation at ply n, but at ply n-1, ..., n-m could be really a source of big potential in ELO terms for who is able to exploit it.

A friend suggested something similar to me maybe 8 years ago. I have always been extremely doubtful of its potential, and had some great arguments against it at the time, that I have since forgotten .

-Sam

Like I say, I would like to do something to exploit that concept, but it's not obvious what it is I should do.

It could turn out that the solution is a real head-slapper. One of those, "that was just too obvious" moments. There have been several times (just in computer chess) that I have stumbled upon great ideas, not recognized them for what they were worth and thus not pursued them, only to find out later that someone else did and now everyone uses that idea. I hope this is not one of those things

My biggest blunder was with LMR - I experimented with it many years ago very much in hte same spirit it is widely used toady. I noticed that it was "almost worth it" and came to the conclusion eventually that it "was obviously unsound." It was just one of those foolish experiments that wasted my time and didn't pay off. Today Komodo gets over 100 ELO for LMR.

[Dan Andersson]

Some of the problems you perceive with evaluation might be down to a basic property of chess and the game theoretical result you are seeking. When you have a set and a graph there is no guarantee there is a total order but a partial order only.

[Don]

Some of the problems you perceive with evaluation might be down to a basic property of chess and the game theoretical result you are seeking. When you have a set and a graph there is no guarantee there is a total order but a partial order only.

All positions can be assigned the value -1, 0 or 1 for lost, drawn or won. If the position is a win a natural order can be imposed based on how many moves is required to checkmate. The inverse of this of course applies to games that are lost. Draws can simply always be assigned the score zero.

In this way you can eliminate partial ordering. With a realistic evaluation of course everything is a guess - and you are probably correct, any ordering you impose is rather arbitrary. So there is no realistic sense in which you can say that a score of "0.6" is more correct that a score of "0.5" unless the 06. is a win and the 0.5 is not - in that case 0.6 is in some sense closer to the truth.

So what we require from a practical evaluation function for chess programs is a way to estimate the probability of winning - we want to always try to move towards positions where a win for us is more likely and a loss less likely and that is always a guess anyway, unless we can actually see mate or prove it's a win.

[Don]

But as I said, I don't know what to do about that! If you modify the score to account for that you in a feedback loop, you would change the root score. If a system could be worked out I believe there would be many opportunities based on things going on in the tree.

I have thought about this at lenght, for instance it is well known that starting from a winning position evaluation increases with depth in an almost monotonic way simply because, being a winning position if both players play "well" the winning one accumulates advantage with each move, this translates in the evaluation to go up after ID loop. Unfortunatly I am (still) not able to code this in something useful, but I think that a kind of feedback on root eval based not only on evaluation at ply n, but at ply n-1, ..., n-m could be really a source of big potential in ELO terms for who is able to exploit it.

Let me ask you this. Would you rather play a position where stockfish with a zero ply search (just a quies) says is 1.2 pawns up, or one where a 30 ply search says is 0.8 pawns up? The 30 ply search is more likely to be a sure win for you, the quies only search could very likely be in error. In fact the quies search already assumes that you cannot lose as you may already be 1.2 pawns up and it is just standing pat (assuming it is your turn to move.)

We could attach actual statistics to this. Take several thousands games with know results played by equally strong computer players, pick out random positions from those games, assign (perhaps) 2 scores to each of our sample positions - one based on a 1 ply search the other based on a very deep search - and then see if it's better to have a score of X centi-pawns from a deep search or a shallow search. I don't know what answer you would get - because the shallow searches would have more error, but the error might work to your advantage in many cases - in other words you might actually "really" be 2 pawns up when the search thinks you are only 1 pawn up - which would be revealed by the deeper search. I would like to see that study.

[Uri Blass]

I am not sure
If you get only 0.8 pawns advantage after 30 ply search it may be with a significant probability only a draw.

If it is a real win then very deep search is going to give a bigger advantage for the winner and I suspect that at 99 ply maybe most of the +3 evaluations are draws because in most winning positions the computer is going to see more than +3 pawns by 99 ply search and the fact that the computer can only see +3 pawns suggest that there may be some fortress.

[mcostalba]

Let me ask you this. Would you rather play a position where stockfish with a zero ply search (just a quies) says is 1.2 pawns up, or one where a 30 ply search says is 0.8 pawns up? The 30 ply search is more likely to be a sure win for you

I would say the 30 ply search is more likely not to be a lose for me (and yes, I think that bounding eval scores to winning probabilities is stretching the eval concept too much).

Eval and search are much more strictly interconnected than one would like, they are not independent concepts. For instance, if in qsearch we would not stand pat, we wouldn't need to perform threat/hanging pieces evaluation at all.

[Rebel]

But as I said, I don't know what to do about that! If you modify the score to account for that you in a feedback loop, you would change the root score. If a system could be worked out I believe there would be many opportunities based on things going on in the tree.

I have thought about this at lenght, for instance it is well known that starting from a winning position evaluation increases with depth in an almost monotonic way simply because, being a winning position if both players play "well" the winning one accumulates advantage with each move, this translates in the evaluation to go up after ID loop. Unfortunatly I am (still) not able to code this in something useful, but I think that a kind of feedback on root eval based not only on evaluation at ply n, but at ply n-1, ..., n-m could be really a source of big potential in ELO terms for who is able to exploit it.

One idea I have experimented is counting "clear advantages" and then add a bonus to score based on the number and weights.

The basic idea, if you have a good passer in the midgame, your mobility is superior, your king is safe while having some pressure on your opponent king then you have all the ingredients for a won game. While one clear advantage does not justify an add score bonus, 2 and 3, 4, 5 certainly do. Something like:

int table[] = { 0, 0, 50, 100, 300, 500, 900, ........ };
score=score + table [counter] * weight;