EGTB value

[Uri Blass]

Another update. Looks like the probing below iteration-depth / 2 is no improvement nor no worse:

Code: Select all

   2 Crafty-23.4R09       2616    4    4 17890   54%  2590   27% 
   3 Crafty-23.4          2616    4    4 17895   54%  2590   27% 
   4 Crafty-23.4R10       2615    6    6  7187   54%  2590   27% 

R10 only probes if ply <= iteration_depth / 2

Looks like no significant difference between any of 'em

I may run some 10m+10s games for a final confirmation, but my conclusion, at least for Crafty, is that EGTBs have no significant effect, good or bad... Will try to experiment with bitbases next...

I think that a possible problem is that part of your opponents do not use tablebases.
Crafty without tablebases may allow won KQP vs KQ against stockfish when stockfish may miss the win and even not convert to the right endgame.

Crafty with tablebases may not allow it and remain 2 pawns down with practically worse chances.

I suggest that you try simply to ignore draws and losses for Crafty and use the tablebases results only when they say that Crafty wins the game.

[bob]

Another update. Looks like the probing below iteration-depth / 2 is no improvement nor no worse:

Code: Select all

   2 Crafty-23.4R09       2616    4    4 17890   54%  2590   27% 
   3 Crafty-23.4          2616    4    4 17895   54%  2590   27% 
   4 Crafty-23.4R10       2615    6    6  7187   54%  2590   27% 

R10 only probes if ply <= iteration_depth / 2

Looks like no significant difference between any of 'em

I may run some 10m+10s games for a final confirmation, but my conclusion, at least for Crafty, is that EGTBs have no significant effect, good or bad... Will try to experiment with bitbases next...

I think that a possible problem is that part of your opponents do not use tablebases.
Crafty without tablebases may allow won KQP vs KQ against stockfish when stockfish may miss the win and even not convert to the right endgame.

Crafty with tablebases may not allow it and remain 2 pawns down with practically worse chances.

I suggest that you try simply to ignore draws and losses for Crafty and use the tablebases results only when they say that Crafty wins the game.

I suppose the question is "what do you want to measure?" My intent was simply Crafty without, vs Crafty with. No change. I suppose that if SF used them it would do a bit better, which means Crafty would do a bit worse. Or, as I suspect, it would make no difference for SF either...

[rbarreira]

I haven't looked at this in detail, but it looks very weird to me that we have databases of perfect moves and we can't make use of them to improve play.

I realize that if the tablebases are used during search that would slow down the search, but as someone else said earlier it should be good to at least use them in the root (i.e. if current game position is in the tablebase, play the move suggested there).

[zamar]

I realize that if the tablebases are used during search that would slow down the search, but as someone else said earlier it should be good to at least use them in the root (i.e. if current game position is in the tablebase, play the move suggested there).

The modern chess engines are such monsters with their 20-30 ply searches that if there is a win in 5 pieces position they will find it with >99% probability, so using EGTB might give 1-2 elo at maximum. Linking gaviota probing code in SF made it around 1% slower when compiling with GCC (I really don't know why but likely somehow related to the grown size of executable), this means -1 elo.

So it's very close to +-0 as Bob has already said.

[bob]

I haven't looked at this in detail, but it looks very weird to me that we have databases of perfect moves and we can't make use of them to improve play.

I realize that if the tablebases are used during search that would slow down the search, but as someone else said earlier it should be good to at least use them in the root (i.e. if current game position is in the tablebase, play the move suggested there).

There are several issues involved.

(1) probing in the search slows things down. 10 years ago, this was not so noticable. Today, with an effective branching factor of way less than 2.0 in endgames, it can be a huge loss.

(2) probing at the root means that you simply stumble into EGTB positions before you know what you have. Ideally you want to search your way into won positions, or at least into drawn positions. But probing at the root, what you will get is the distribution of win/lose/draw that is inherent in the position. Suppose that 2/3 are losses and 1/3 are wins. Probing at the root, you lose 2/3 of the games. If you probe out in the search, you have a much better chance of forcing things toward that 1/3 win ratio. If it doesn't slow you down too much so that you lose before you get to an egtb-at-the-root position.

(3) It is nice to have perfect information. But in a normal search, if you look at the statistics, you might search (at 1 minute per move) 1.2 billion nodes. If you start doing I/O at a significant number of those positions, you are going to reduce that tree to 1.2M or so. You make tactical oversights right and left for the positions that don't go deep enough to hit the EGTBs. And you lose.

[Uri Blass]

I haven't looked at this in detail, but it looks very weird to me that we have databases of perfect moves and we can't make use of them to improve play.

I realize that if the tablebases are used during search that would slow down the search, but as someone else said earlier it should be good to at least use them in the root (i.e. if current game position is in the tablebase, play the move suggested there).

There are several issues involved.

(1) probing in the search slows things down. 10 years ago, this was not so noticable. Today, with an effective branching factor of way less than 2.0 in endgames, it can be a huge loss.

I do not understand why probing in the search has to slow things down(assuming that you do not probe in every node in the search but only when the remaining depth is big enough so the time of searching the remaining depth is bigger than the time of probing the tablebases).

I think that the first test before playing games should be if you get the same depth faster in endgame analysis.

If you get the same depth faster but still do not earn rating points based on games then it is interesting(and maybe the reason can be that you are slightly slower in opening and middle game positions when you do not probe tablebases because checking that you do not need to probe tablebases after a capture cost time).

[bob]

I haven't looked at this in detail, but it looks very weird to me that we have databases of perfect moves and we can't make use of them to improve play.

I realize that if the tablebases are used during search that would slow down the search, but as someone else said earlier it should be good to at least use them in the root (i.e. if current game position is in the tablebase, play the move suggested there).

There are several issues involved.

(1) probing in the search slows things down. 10 years ago, this was not so noticable. Today, with an effective branching factor of way less than 2.0 in endgames, it can be a huge loss.

I do not understand why probing in the search has to slow things down(assuming that you do not probe in every node in the search but only when the remaining depth is big enough so the time of searching the remaining depth is bigger than the time of probing the tablebases).

I think that the first test before playing games should be if you get the same depth faster in endgame analysis.

If you get the same depth faster but still do not earn rating points based on games then it is interesting(and maybe the reason can be that you are slightly slower in opening and middle game positions when you do not probe tablebases because checking that you do not need to probe tablebases after a capture cost time).

Simple math. On my 8-core box, Crafty searches about 20M nodes per second up to 30M in the endgame. On a good disk, you can do a read every 5ms, or about 200 reads per second. Compare the speeds. During the time I can do a single read, I can search 150K nodes. That is huge. For every 200 I/O accesses, I could search another 30M nodes. The cost to check for probes is roughly zero since in the opening and middlegame, that branch gets predicted 100% correctly. But when you get down to the 12-16 pieces total, you begin to see EGTB probes. Each successful probe costs about 5ms if a disk access is required, actually quite a bit more since you read in fairly large compressed blocks, and then have to spend time uncompressing a block before the probe can be completed...

Either you rarely probe, which doesn't help much, or you probe a lot, which starts to cost multiple plies.

[michiguel]

I haven't looked at this in detail, but it looks very weird to me that we have databases of perfect moves and we can't make use of them to improve play.

I realize that if the tablebases are used during search that would slow down the search, but as someone else said earlier it should be good to at least use them in the root (i.e. if current game position is in the tablebase, play the move suggested there).

There are several issues involved.

(1) probing in the search slows things down. 10 years ago, this was not so noticable. Today, with an effective branching factor of way less than 2.0 in endgames, it can be a huge loss.

I do not understand why probing in the search has to slow things down(assuming that you do not probe in every node in the search but only when the remaining depth is big enough so the time of searching the remaining depth is bigger than the time of probing the tablebases).

I think that the first test before playing games should be if you get the same depth faster in endgame analysis.

If you get the same depth faster but still do not earn rating points based on games then it is interesting(and maybe the reason can be that you are slightly slower in opening and middle game positions when you do not probe tablebases because checking that you do not need to probe tablebases after a capture cost time).

Simple math. On my 8-core box, Crafty searches about 20M nodes per second up to 30M in the endgame. On a good disk, you can do a read every 5ms, or about 200 reads per second. Compare the speeds. During the time I can do a single read, I can search 150K nodes. That is huge. For every 200 I/O accesses, I could search another 30M nodes. The cost to check for probes is roughly zero since in the opening and middlegame, that branch gets predicted 100% correctly. But when you get down to the 12-16 pieces total, you begin to see EGTB probes. Each successful probe costs about 5ms if a disk access is required, actually quite a bit more since you read in fairly large compressed blocks, and then have to spend time uncompressing a block before the probe can be completed...

Either you rarely probe, which doesn't help much, or you probe a lot, which starts to cost multiple plies.

Your 5ms figure assumes no cache. But if you have a cache that makes you read from disk only 5% of the time, a fast decompressing scheme that is effectively faster than reading uncompressed data, you will end up with an average figure of 0.3 ms (and we are not even talking about SSD). That means, if you probe in a node in which it is required to search deep enough to cost only 10k nodes, the effect on speed will be negligible. That means you can search "relatively" close to the leaves with no time-to-depth cost (in fact it should lower because of the pruning performed at nodes closer to the root). What is the depth you can reach with 10 k nodes? that is "approximately" the distance to the leaves you can _safely_ afford.

Of course, these numbers may not apply to Nalimov, but they do apply to the Gaviota TBs.

Miguel

[jwes]

I haven't looked at this in detail, but it looks very weird to me that we have databases of perfect moves and we can't make use of them to improve play.

I realize that if the tablebases are used during search that would slow down the search, but as someone else said earlier it should be good to at least use them in the root (i.e. if current game position is in the tablebase, play the move suggested there).

There are several issues involved.

(1) probing in the search slows things down. 10 years ago, this was not so noticable. Today, with an effective branching factor of way less than 2.0 in endgames, it can be a huge loss.

If you were able to distribute the EGTBs over the cluster, you could have them all in memory without using too much memory on any node. Network transactions, while slow, should be much faster than disk accesses. It could be a project for a graduate student.

(3) It is nice to have perfect information. But in a normal search, if you look at the statistics, you might search (at 1 minute per move) 1.2 billion nodes. If you start doing I/O at a significant number of those positions, you are going to reduce that tree to 1.2M or so. You make tactical oversights right and left for the positions that don't go deep enough to hit the EGTBs. And you lose.

One idea would be to use the material as well as the depth to determine whether to probe. If you are well ahead or behind it is more likely you can properly evaluate without EGTBs. I am also wondering what percentage of games go to endgames where the outcome is not clear. If it is only one percent of the total, then even a 100 elo improvement in those positions would only result at most in a 1 elo improvement over all games.

[mcostalba]

I think you all miss the key point that Joona tried to explain in his post so probably you need something more explicit.

Here is the post of Joona:

"The modern chess engines are such monsters with their 20-30 ply searches that if there is a win in 5 pieces position they will find it with >99% probability, so using EGTB might give 1-2 elo at maximum. Linking gaviota probing code in SF made it around 1% slower when compiling with GCC (I really don't know why but likely somehow related to the grown size of executable), this means -1 elo.

So it's very close to +-0 as Bob has already said."

All you focus on the "speed decrease" side but none of you focus on the _real_ key point that is:

Code: Select all

The modern chess engines are such monsters with their 20-30 ply searches that if there is a win in 5 pieces position they will find it with >99% probability, so using EGTB might give 1-2 elo at maximum

Just to be more explicit, it seems is needed here, also in case of zero slowdown the possible ELO gain that could arise from EGTB is very low because if there is a possibility of winning a modern engine on fast hardware will find this winning path anyhow in the vast majority of cases.

Thats the reason why IMO EGBT are almost useless for playing strenght.