TCEC openings

[corres]

Draw is essence of chess, I think. But a draw gotten in battle is valuable only. So in material equilibrium a symmetrical pawn structure with tempo advantage or an asymmetrical pawn standing without blocked centre is a good choice for testing positional knowledge.
A handicapped game is very unnatural one. Between AI and Human is acceptable (mainly for demonstration), but between engines it is an ugly thing, I think. The games of gambit are more attractive: King`s Gambit, some line of Queen`s Gambit, Sicilian Gambit and so an. Gambits give a chance to take the measure of the engines` knowledge of attack/defense, too.

[lkaufman]

Draw is essence of chess, I think. But a draw gotten in battle is valuable only. So in material equilibrium a symmetrical pawn structure with tempo advantage or an asymmetrical pawn standing without blocked centre is a good choice for testing positional knowledge.
A handicapped game is very unnatural one. Between AI and Human is acceptable (mainly for demonstration), but between engines it is an ugly thing, I think. The games of gambit are more attractive: King`s Gambit, some line of Queen`s Gambit, Sicilian Gambit and so an. Gambits give a chance to take the measure of the engines` knowledge of attack/defense, too.

Yes, I quite agree. The pawn removal test was to determine how much of an advantage is needed to reach the draw/win threshold, it was not a proposal for use in tournaments like TCEC. But the conclusion is that any chosen gambits should offer only small compensation for the pawn, otherwise they are apt to be too drawish. I might try some tests to see what are suitable gambits.

[Laskos]

This fits reasonably with the report I saw several years ago on Rybka forum, where someone tested all possible Pawn-odds positions in Rybka self-play. IIRC scores of around 73% were reported there.

It might depend on the engine a bit,and particular its skill in the opening; with Fairy-Max I usually see a 68% advantage as a result of deleting a Pawn (but I always average over both colors, so that does not include any first-move advantage). It also seems to not matter very much which Pawn I delete there, probably because Fairy-Max does not have any skill, and therefore cannot exploit specific positional opening advantages very well, but allows these to evaporate.

BTW, I don't think that the issue with f2 deletion is King Safety, as the King usually doesn't keep camp on e1 very long. It is more that all other Pawn deletions offer some compensation by accelerating development.

It does depend on the skill. Komodo gets as Black roughly 77% from b2 pawn handicap at 5''+0.05'' self-play. Some care must be taken describing the results, as there are White wins too. Roughly 60% Black wins, 35% draws, 5% White wins in this case. Already a bit different from what Larry gets with fast Rybka. At longer time controls the Black performance increases further.

Your results agree more or less with my opinion as a grandmaster but as you say not so well with my test results using Fritz 15 MC. Perhaps it is just due to higher level of technique in Komodo, but another factor might be that both sides in your test used Contempt (unless you set them to zero), which would make for fewer draws and thus help the stronger side. Perhaps you might try removing c2, which is (I think) the best choice for White to remove other than the two edge pawns. My personal opinion is that removing any of the six non-edge pawns of White should be enough to give Black a theoretically won game, but not the two edge pawns. So I differ from Lyudmil only on the a2 pawn removal.

c2 and Contempt=0 gave at 5''+0.05'' 49% Black wins, 44% draws, 7% White wins in 400 games. Performance of 71%. Very close to what you got for b2. In hope to see whether it tends to win or draw, I will let Komodo play at 20''+0.2'' for a bit. From an old thread, the eval/performance should go like that with improved skill:



The "perfect" engine graph is hypothesized.
And there seem to be a point close to win/draw threshold where the score for a given handicap is stable to time control or skill.

[hgm]

If that is truly stable it should be exactly at the win-draw threshold, right?

[Laskos]

If that is truly stable it should be exactly at the win-draw threshold, right?

Yes, but those lines performance/eval are a bit fuzzy on concrete positions.

[Laskos]

This fits reasonably with the report I saw several years ago on Rybka forum, where someone tested all possible Pawn-odds positions in Rybka self-play. IIRC scores of around 73% were reported there.

It might depend on the engine a bit,and particular its skill in the opening; with Fairy-Max I usually see a 68% advantage as a result of deleting a Pawn (but I always average over both colors, so that does not include any first-move advantage). It also seems to not matter very much which Pawn I delete there, probably because Fairy-Max does not have any skill, and therefore cannot exploit specific positional opening advantages very well, but allows these to evaporate.

BTW, I don't think that the issue with f2 deletion is King Safety, as the King usually doesn't keep camp on e1 very long. It is more that all other Pawn deletions offer some compensation by accelerating development.

It does depend on the skill. Komodo gets as Black roughly 77% from b2 pawn handicap at 5''+0.05'' self-play. Some care must be taken describing the results, as there are White wins too. Roughly 60% Black wins, 35% draws, 5% White wins in this case. Already a bit different from what Larry gets with fast Rybka. At longer time controls the Black performance increases further.

Your results agree more or less with my opinion as a grandmaster but as you say not so well with my test results using Fritz 15 MC. Perhaps it is just due to higher level of technique in Komodo, but another factor might be that both sides in your test used Contempt (unless you set them to zero), which would make for fewer draws and thus help the stronger side. Perhaps you might try removing c2, which is (I think) the best choice for White to remove other than the two edge pawns. My personal opinion is that removing any of the six non-edge pawns of White should be enough to give Black a theoretically won game, but not the two edge pawns. So I differ from Lyudmil only on the a2 pawn removal.

c2 and Contempt=0 gave at 5''+0.05'' 49% Black wins, 44% draws, 7% White wins in 400 games. Performance of 71%. Very close to what you got for b2. In hope to see whether it tends to win or draw, I will let Komodo play at 20''+0.2'' for a bit.

Interesting, at 20''+0.2'' in 400 games the percentages are: 46% Black wins, 51% draws, 3% White wins. Performance of same 71%. If the difference is significant, this seems to mean the position tends to draw at longer TC.

[lkaufman]

This fits reasonably with the report I saw several years ago on Rybka forum, where someone tested all possible Pawn-odds positions in Rybka self-play. IIRC scores of around 73% were reported there.

It might depend on the engine a bit,and particular its skill in the opening; with Fairy-Max I usually see a 68% advantage as a result of deleting a Pawn (but I always average over both colors, so that does not include any first-move advantage). It also seems to not matter very much which Pawn I delete there, probably because Fairy-Max does not have any skill, and therefore cannot exploit specific positional opening advantages very well, but allows these to evaporate.

BTW, I don't think that the issue with f2 deletion is King Safety, as the King usually doesn't keep camp on e1 very long. It is more that all other Pawn deletions offer some compensation by accelerating development.

It does depend on the skill. Komodo gets as Black roughly 77% from b2 pawn handicap at 5''+0.05'' self-play. Some care must be taken describing the results, as there are White wins too. Roughly 60% Black wins, 35% draws, 5% White wins in this case. Already a bit different from what Larry gets with fast Rybka. At longer time controls the Black performance increases further.

Your results agree more or less with my opinion as a grandmaster but as you say not so well with my test results using Fritz 15 MC. Perhaps it is just due to higher level of technique in Komodo, but another factor might be that both sides in your test used Contempt (unless you set them to zero), which would make for fewer draws and thus help the stronger side. Perhaps you might try removing c2, which is (I think) the best choice for White to remove other than the two edge pawns. My personal opinion is that removing any of the six non-edge pawns of White should be enough to give Black a theoretically won game, but not the two edge pawns. So I differ from Lyudmil only on the a2 pawn removal.

c2 and Contempt=0 gave at 5''+0.05'' 49% Black wins, 44% draws, 7% White wins in 400 games. Performance of 71%. Very close to what you got for b2. In hope to see whether it tends to win or draw, I will let Komodo play at 20''+0.2'' for a bit.

Interesting, at 20''+0.2'' in 400 games the percentages are: 46% Black wins, 51% draws, 3% White wins. Performance of same 71%. If the difference is significant, this seems to mean the position tends to draw at longer TC.

The blitz and slow curves for Komodo in your graph intersect at about an eval of 1.1, which is way more than the eval of the opening position with c2 removed. So based on your graph, it should move towards a draw with more time. An eval of 1.1 in Komodo is really a huge advantage; it's hard to believe that the win/draw line could be this high.

[Laskos]

]Interesting, at 20''+0.2'' in 400 games the percentages are: 46% Black wins, 51% draws, 3% White wins. Performance of same 71%. If the difference is significant, this seems to mean the position tends to draw at longer TC.

The blitz and slow curves for Komodo in your graph intersect at about an eval of 1.1, which is way more than the eval of the opening position with c2 removed. So based on your graph, it should move towards a draw with more time. An eval of 1.1 in Komodo is really a huge advantage; it's hard to believe that the win/draw line could be this high.

Don't take too literally that graph, the TCEC part was done IIRC on 100 games only (too few for a reliable line). But the trend would be that of a slow convergence to a fuzzy step function.

[Lyudmil Tsvetkov]

Does not 50% black wins, almost 0% white wins, 50% draws mean the positions is much closer to a black win than draw?

A meidum situation would be the case when white and black win about the same percentage, then this certainly would mean draw.

So it is much closer to a win than draw.


Another important issue is that, according to me, there is a single good setup for black in the specific handicap, namely placing a pawn on e5, alternatively, placing a bishop on g7, if an early pawn on e5 or bishop on g7 do not feature in some of those games, this migth somehow explain why black does not win all.

Any setup with pawn on d5 is simply not good for black, this is the case with 1.b4, but b2 handicap should be very similar to it.

[corres]

I think that the high draw rate arises from the fact that engines chose not the BEST move to continue the game but ONE of the good moves. If you enhance the playing time significantly the engines find more moves to be good so longer time enhances the probability to find no the really best move.
In GM practice one pawn benefit gives much more chance to win the game as it happens in engine practice.