Edsel Apostol wrote:I've tried reducing/pruning bad captures in the latest Hannibal. I've reduced a bad capture move just like what we do with LMR and prune it just like what we do with Futility Pruning with the exception that I also add the value of the captured piece.
I'm surprised to find the results very close between the one with the change and the base version. Has anyone tried this idea and have you noticed the same test results?
20110819 is the base version. 20110819x is the version with the bad captures pruning/reduction idea.
One can notice that the 2 versions perform differently against the opponents. The 19x version performed better against stronger engines like Komodo, Critter1.2 and Houdini while performing worst against the weaker opponents. This suggests that maybe those strong engines are also doing this bad captures pruning/reduction ideas, though I might be wrong.
The difference in Elo is only 4 with the error bars at +-17. So which do you think is better, performing better against stronger engines or performing better against weaker engines?
Sometimes I consider to continue developing versions that perform a little bit worse than the best version when my intuition says that the idea has potential and just needs fine tuning.
Do you account for pinned pieces in your see function (specially the pinned ones against the king)? My intuition says that if you are using the see function to prune/reduce bad captures then you would want to have it the more accurate as possible.
I tried reducing/pruning bad captures based on the see and didn't get good results, although my see function doesn't account for pinned pieces. I should that try that in the next days.
Our SEE doesn't take into account pinned pieces to the king, so it is possible that we could improve the results by supporting this. Please let us know the results of your test.
It dates back quite a while, but Joel Rivat (Chess Guru, no longer active it seems) and I had a similar discussion about SEE. He was using it for a few things I was not, and he thought that making it more accurate would be an improvement.
I spent some time, adding absolute pins, and saw Crafty play a little worse. Mainly because SEE is used quite a lot, and slowing it down has an effect. I think there are enough errors in the SEE concept (namely that the best move is often not a capture) that its error rate will always be higher than we'd like. But if you use it right, and it is correct more often than not, it should result in a gain. If you can squeeze more accuracy without losing speed, it should always be a gain of some sort. But if there is a speed trade-off, then it becomes very unclear and a lot of testing is needed to get down to the error bar range necessary to measure the improvement.
Don wrote:
If losing capture:
reduce search using normal schedule, set scout to alpha-MARGIN
It's possible that we have already tried this. I have probably only tried about a million things in the last 3 years and don't remember all of them
Don
SF got something like +5 elo using this idea. The problem is that although engine is a bit stronger, it's much weaker in tactical puzzles (which often involve a bad capture or even two of them!).
But Joona claimed earlier that it didn't work! I would kill for 5 ELO! Are you saying that would give up 5 ELO for better tactics but a weaker program?
Earlier I was referring strictly bad capturing reducing/pruning and it didn't work. But reducing in low depths aggressively AND tweaking alpha, gave us something (take a look at SF 2.1), but the downside is (as I said) that engine became much weaker in tactics. As H.G wisely put it, if capture is futile _enough_ it can be reduced aggressively.
But still I view this as a separate thing than LMR, because we apply this technique only close to leaves, reduce much more aggressively and tweak alpha around 100cp.
bob wrote:
However, in the case I gave, the last move could have been Rh6 or Rxh6. You can't tell the difference unless you see the position prior to that move. Given that, what would say "reduce the non-capture, but not the capture, since the resulting position is identical in all other aspects.."
In classical poker you can get four aces after changing all the cards, or after changing only two cards (assuming already have three aces). The latter is just much more likely to happen.
In chess you can reach a critical position after silent non-capture or bad capture. The latter is just much more likely to happen.
bob wrote:
However, in the case I gave, the last move could have been Rh6 or Rxh6. You can't tell the difference unless you see the position prior to that move. Given that, what would say "reduce the non-capture, but not the capture, since the resulting position is identical in all other aspects.."
In classical poker you can get four aces after changing all the cards, or after changing only two cards (assuming already have three aces). The latter is just much more likely to happen.
In chess you can reach a critical position after silent non-capture or bad capture. The latter is just much more likely to happen.
What is this based on? There are actually fewer capture moves than non-captures... And chess programs hang pieces all over the board, as I see every time I dump a trace looking for a bug...
I'm actually re-testing that idea right now. I had tried it once and did not keep it, but I kept the version that tried it, so rather than tracking thru all the PGN data, I am re-running. Captures will get reduced by 1 now, rather than 2, when SEE < 0....
A am using the approx. same ordering scheme as yours. The see < 0 moves are at the end of the list. "Quiet" moves are above them. I am reducing only by 1 PLY for any move. That gives me a lot of room for testing and experiments. And yes, unfortunately, more accurate SEE is slowing down the entire program, since it's used a lot. Maybe that eats up the benefits of its correctness (if any).
Mincho Georgiev wrote:A am using the approx. same ordering scheme as yours. The see < 0 moves are at the end of the list. "Quiet" moves are above them. I am reducing only by 1 PLY for any move. That gives me a lot of room for testing and experiments. And yes, unfortunately, more accurate SEE is slowing down the entire program, since it's used a lot. Maybe that eats up the benefits of its correctness (if any).
That was my observation when Joel and I discussed this years ago. It is easy to make it more accurate. But also slower. This is an age-old balancing act. Only have 1400 games so far, with this change at +13 Elo. Something tells me that is not going to hold as it is too simple a change. Had to restart once, as my first fix was not exactly right and it was reducing where it should not near the tips, which was killing things to the tune of -40 or so...
Mincho Georgiev wrote:A am using the approx. same ordering scheme as yours. The see < 0 moves are at the end of the list. "Quiet" moves are above them. I am reducing only by 1 PLY for any move. That gives me a lot of room for testing and experiments. And yes, unfortunately, more accurate SEE is slowing down the entire program, since it's used a lot. Maybe that eats up the benefits of its correctness (if any).
That was my observation when Joel and I discussed this years ago. It is easy to make it more accurate. But also slower. This is an age-old balancing act. Only have 1400 games so far, with this change at +13 Elo. Something tells me that is not going to hold as it is too simple a change. Had to restart once, as my first fix was not exactly right and it was reducing where it should not near the tips, which was killing things to the tune of -40 or so...
Now down to "dead even" with last version, but after only 6000 games. That's usually the way it goes...
bob wrote:Now down to "dead even" with last version, but after only 6000 games. That's usually the way it goes...
Good to know that it isn't only my program that regularly shows something like 50+ ELO through the first 20% of the testing, only to end up with no improvement at all.
bob wrote:Now down to "dead even" with last version, but after only 6000 games. That's usually the way it goes...
Good to know that it isn't only my program that regularly shows something like 50+ ELO through the first 20% of the testing, only to end up with no improvement at all.
jm
I don't know whether it is an artifact introduced by BayesElo, but I generally see an Elo that is too high, and then it drops back down to the previous level (or occasionally lower). I don't know why I don't see just as many where the elo starts off too low and climbs.
I have even randomized my starting positions a couple of times, just to make sure that favorable positions are not tested first to inflate the Elo, or that bad ones are done last...
