Aleks Peshkov wrote:I do not believe that Diep's eval is badly tuned.
But I think that complex evaluation hurts branching factor and worsen gains of any forward pruning tricks because of larger random factor of minor relevant positional moves that leads to search instabilities.
I suspect that Rybka 3 has worse branching factor then Rybka 2 because of more complex evaluation and not because Vas had greatly redesigned his search from Rybka 1 beta times.
I believe that rybka3 has worse branching factor because rybka3 extends more than Rybka2.
I do not believe that more complex evaluation reduce the branching factor unless you start with something like only material evaluation.
With only material evaluation every move cause a cutoff but if you start with something like piece square table evaluation then I see no reason to have worse branching factor with better evaluation.
It is the opposite because with better evaluation you can use your evaluation better for pruning tricks based on evaluation so you get smaller branching factor.
Good evaluation do not give you a single number but also probabilities of win,draw,loss.
With good complicated evaluation you know that white is not worse in the following position with 100% confidence so if the evaluation is not positive for white you can safely prune in this position regardless of the remaining depth(the reason is that your complex evaluation can detect that white has material advantage and black has no type of positional advantage and no threats).
With bad evaluation you do not know it so you need to search and waste time.
[D]6rk/5ppp/8/8/8/8/4PPPP/5R1K w - - 0 1
Aleks Peshkov wrote:I do not believe that Diep's eval is badly tuned.
But I think that complex evaluation hurts branching factor and worsen gains of any forward pruning tricks because of larger random factor of minor relevant positional moves that leads to search instabilities.
I suspect that Rybka 3 has worse branching factor then Rybka 2 because of more complex evaluation and not because Vas had greatly redesigned his search from Rybka 1 beta times.
I believe that rybka3 has worse branching factor because rybka3 extends more than Rybka2.
I don't believe that. Tons of testing has shown most extensions actually hurt, not help, in real games. I've noticed from others reporting here that R3 is worse on tactical positions than previous versions, suggesting that extensions were reduced, not increased.
I do not believe that more complex evaluation reduce the branching factor unless you start with something like only material evaluation.
Neither does he. He said "complex evaluation _hurts_ branching factor, not "helps".
With only material evaluation every move cause a cutoff but if you start with something like piece square table evaluation then I see no reason to have worse branching factor with better evaluation.
It is the opposite because with better evaluation you can use your evaluation better for pruning tricks based on evaluation so you get smaller branching factor.
Good evaluation do not give you a single number but also probabilities of win,draw,loss.
With good complicated evaluation you know that white is not worse in the following position with 100% confidence so if the evaluation is not positive for white you can safely prune in this position regardless of the remaining depth(the reason is that your complex evaluation can detect that white has material advantage and black has no type of positional advantage and no threats).
With bad evaluation you do not know it so you need to search and waste time.
[D]6rk/5ppp/8/8/8/8/4PPPP/5R1K w - - 0 1
Uri
That's all well and good. But if you try the test, you will discover that as you add eval terms, branching factor goes up. Material-only has the lowest branching factor of all.
I've noticed from others reporting here that R3 is worse on tactical positions than previous versions, suggesting that extensions were reduced, not increased.
Ah, I've seen the exact opposite claim. Must've been very scientific tests whose reports we've both been reading.
bob wrote:
That's all well and good. But if you try the test, you will discover that as you add eval terms, branching factor goes up. Material-only has the lowest branching factor of all.
Material-only is special because the granularity is so big.
I don't think this is true in general.
A better evaluation would be expected to IMPROVE your branching factor, because you will have less "flipnodes" (to use a Vincent-term).
I've noticed from others reporting here that R3 is worse on tactical positions than previous versions, suggesting that extensions were reduced, not increased.
Ah, I've seen the exact opposite claim. Must've been very scientific tests whose reports we've both been reading.
bob wrote:
That's all well and good. But if you try the test, you will discover that as you add eval terms, branching factor goes up. Material-only has the lowest branching factor of all.
Material-only is special because the granularity is so big.
I don't think this is true in general.
A better evaluation would be expected to IMPROVE your branching factor, because you will have less "flipnodes" (to use a Vincent-term).
I noticed an improvement going back to centipawns, from millipawns, because there are fewer "gaps" in the scores. "flipnodes" only require one unit of evaluation to happen. The more stuff you put in, the greater the probability one of those terms will cause a "flip". Unless the evaluation is perfect.
Aleks Peshkov wrote:I do not believe that Diep's eval is badly tuned.
But I think that complex evaluation hurts branching factor and worsen gains of any forward pruning tricks because of larger random factor of minor relevant positional moves that leads to search instabilities.
I suspect that Rybka 3 has worse branching factor then Rybka 2 because of more complex evaluation and not because Vas had greatly redesigned his search from Rybka 1 beta times.
I believe that rybka3 has worse branching factor because rybka3 extends more than Rybka2.
I don't believe that. Tons of testing has shown most extensions actually hurt, not help, in real games. I've noticed from others reporting here that R3 is worse on tactical positions than previous versions, suggesting that extensions were reduced, not increased.
I do not believe that more complex evaluation reduce the branching factor unless you start with something like only material evaluation.
Neither does he. He said "complex evaluation _hurts_ branching factor, not "helps".
With only material evaluation every move cause a cutoff but if you start with something like piece square table evaluation then I see no reason to have worse branching factor with better evaluation.
It is the opposite because with better evaluation you can use your evaluation better for pruning tricks based on evaluation so you get smaller branching factor.
Good evaluation do not give you a single number but also probabilities of win,draw,loss.
With good complicated evaluation you know that white is not worse in the following position with 100% confidence so if the evaluation is not positive for white you can safely prune in this position regardless of the remaining depth(the reason is that your complex evaluation can detect that white has material advantage and black has no type of positional advantage and no threats).
With bad evaluation you do not know it so you need to search and waste time.
[D]6rk/5ppp/8/8/8/8/4PPPP/5R1K w - - 0 1
Uri
That's all well and good. But if you try the test, you will discover that as you add eval terms, branching factor goes up. Material-only has the lowest branching factor of all.
My comments:
1)I believe that you confuse between rybka3 and older versions of rybka
The first improvement in rybka from rybka1 beta to rybka1.2 and rybka2.x was by reducing rybka's results in test positions but this process stopped and Rybka3 is clearly the best version of rybka in test positions.
2)I did a mistake when I said that I do not believe that complex evaluation reduce the branching factor.
I meant the opposite.
I agree that with only material evaluation you can get better branching factor relative to more complicated evaluation because almost all scores are the same but when you have a simple evaluation that scores are different then I believe that more complex evaluation is not going to increase the branching factor and of course rybka never had only material evaluation.
I've noticed from others reporting here that R3 is worse on tactical positions than previous versions, suggesting that extensions were reduced, not increased.
Ah, I've seen the exact opposite claim. Must've been very scientific tests whose reports we've both been reading.
bob wrote:
That's all well and good. But if you try the test, you will discover that as you add eval terms, branching factor goes up. Material-only has the lowest branching factor of all.
Material-only is special because the granularity is so big.
I don't think this is true in general.
A better evaluation would be expected to IMPROVE your branching factor, because you will have less "flipnodes" (to use a Vincent-term).
I noticed an improvement going back to centipawns, from millipawns, because there are fewer "gaps" in the scores. "flipnodes" only require one unit of evaluation to happen. The more stuff you put in, the greater the probability one of those terms will cause a "flip". Unless the evaluation is perfect.
I agree that it is possible that you may get an improvement when you go from millipawns to centipawns but it is not about the complexity of the evaluation.
complexity of the evaluation is how much code you need and you sometimes need a lot of code for evaluating some 0.2 score.
I think that the importance of better evaluation is not to be correct about small 0.01 pawns but to be correct about the big factors.
bob wrote:
I noticed an improvement going back to centipawns, from millipawns, because there are fewer "gaps" in the scores. "flipnodes" only require one unit of evaluation to happen. The more stuff you put in, the greater the probability one of those terms will cause a "flip". Unless the evaluation is perfect.
So the statement is that more coarse granularity improves the branching factor.
I'm saying that all things being equal (including effective granularity) more evaluation helps you, because less things will flip (both in eval and in search).
bob wrote:
All I can say is "where is the +40 Elo coming from?" It is not imaginary, it is based on tens of thousands of test games against multiple opponents.
Bob you raise a valid point. However now i'm lurking around at ICC daily again, 99% of all games you get are against 3200+ rated rybka's, crafty being 2700 rated or so.
So basically you lose about every game. If you then build in a feature that has a BESTCASE, besides a worst case, then of course you no longer feel the worstcase, because worse than it is going right now for us there it can't go.
We gotta improve some stuff Bob!
What i'm seeing in testgames against 2000 elo rated engines is that LMR shows its worst case, not its bestcase. I understand your big desire to sometimes beat Rybka 3.0, but let's first kick who's below us i'd say. I'd hate to see that posting of Uri saying: "my never parallellized program beated Diep".
Vincent
I don't pay any attention to ICC games. I see cooked books, really fast hardware, most are running Rybka, etc. My testing is done in a controlled environment on equal hardware, equal time controls, selected starting positions, no pondering, etc... I trust these results far more than anything from ICC where you can't even tell what hardware your opponent is using.
As far as Rybka goes, I am not even looking at that at the moment. I don't run it on my cluster and it isn't a part of any testing I do.
Can you email me those games played in the controlled environment?
Thanks,
Vincent
How many millions of games do you want and what can they be used for???
Complex evaluation means not "a lot of code", but many chess terms.
Fiction example. It is difficult to compare the value of bishop pair to the value of doubled pawn. If their values are similar, then program would flip PV and make costly researches trying to decide whether it is good or bad to trade bishop on C6 square. I do not claim that these terms are complex, just an example.
When the number of chess term increases chances that a last minor positional move at the end of variation will change the PV. Lazy evaluation and futility pruning will hurt more interferencing to too sensitive evaluation.
The effect of last move is great even in material-only evaluation, that is why all strong chess programs from wild ages use quiescent search.
Aleks Peshkov wrote:Complex evaluation means not "a lot of code", but many chess terms.
Fiction example. It is difficult to compare the value of bishop pair to the value of doubled pawn. If their values are similar, then program would flip PV and make costly researches trying to decide whether it is good or bad to trade bishop on C6 square. I do not claim that these terms are complex, just an example.
When the number of chess term increases chances that a last minor positional move at the end of variation will change the PV. Lazy evaluation and futility pruning will hurt more interferencing to too sensitive evaluation.
The effect of last move is great even in material-only evaluation, that is why all strong chess programs from wild ages use quiescent search.
Last move matters a lot indeed. That's why in Diep i used to play more moves in qsearch than just captures and checks. Some attacked pieces were allowed to move away, though later i limited it. In 2006 i kicked it out,
it hurts branching factor bigtime.
Not sure how much elo it won or lost though. Would be interesting to retest now.
What i DID figure out is that capturing a pawn you can capture is not always changing score a lot. Maybe 0.5 pawn or if you're lucky 0.8 pawn the eval rises. This where a black knight in opening that moves from a6 to c5 soon can change score like 2.5 pawn or so. Yet i'm doing the pawn capture but not the knight move.
A new 21th century implementation of what to do in quiescencesearch is really interesting to take a look at.
