Question regarding WAC number 2

Discussion of chess software programming and technical issues.

Moderator: Ras

bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Question regarding WAC number 2

Post by bob »

mcostalba wrote:
bob wrote: I can answer that. zero. From testing. I removed them from Crafty and found Elo went up. I removed them from SF 1.6, one by one, and found that the elo either went up slightly or did not change. Except for the check extension itself which is worthwhile although not +20 worthwhile...
Ok, I will remove all the extension at once, apart from SE and check exstension, and test and if there is no advantage we will remove all them in one go ;-)
That's an OK test. I think it better to go one at a time, because each chess engine is different, with different rules for things like reductions, pruning, etc. Sometimes you can have a bad idea that compensates for another bad idea, to minimize the effect. For example, if you reduce too much, an extra extension can compensate for that...

For that reason, I tested them one at a time in Crafty. And I have added them back in a couple of times since to test. Note that they are still "out". :)

Also remember that I am more conservative than most with the check extension, and do not extend a check where SEE < 0, which was another minor Elo gain for me...

I also tried a few variations, such as extending passed pawn pushes that were safe (according to SEE, SEE >= 0) and such. But could not get passed pawns, one-legal-reply, null-move mate threat nor recapture to work. Recapture still deserves some more testing, and is on my list...
mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: Question regarding WAC number 2

Post by mcostalba »

bob wrote:
mcostalba wrote:
bob wrote: I can answer that. zero. From testing. I removed them from Crafty and found Elo went up. I removed them from SF 1.6, one by one, and found that the elo either went up slightly or did not change. Except for the check extension itself which is worthwhile although not +20 worthwhile...
Ok, I will remove all the extension at once, apart from SE and check exstension, and test and if there is no advantage we will remove all them in one go ;-)
That's an OK test. I think it better to go one at a time, because each chess engine is different, with different rules for things like reductions, pruning, etc.
You said that you already tried to remove one by one in SF 1.6, so if removing all does not have any effect then I will trust you and take as valid your results ;-)
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Question regarding WAC number 2

Post by bob »

mcostalba wrote:
bob wrote:
mcostalba wrote:
bob wrote: I can answer that. zero. From testing. I removed them from Crafty and found Elo went up. I removed them from SF 1.6, one by one, and found that the elo either went up slightly or did not change. Except for the check extension itself which is worthwhile although not +20 worthwhile...
Ok, I will remove all the extension at once, apart from SE and check exstension, and test and if there is no advantage we will remove all them in one go ;-)
That's an OK test. I think it better to go one at a time, because each chess engine is different, with different rules for things like reductions, pruning, etc.
You said that you already tried to remove one by one in SF 1.6, so if removing all does not have any effect then I will trust you and take as valid your results ;-)
Should work. I was making a general comment that what works for one doesn't always work for another. Anyone following the "extension thread" ought to test their own engine carefully rather than just blindly playing "follow the leader" like lemmings. :)

Was somewhat disappointing for me since I had been doing all of these extensions for many years, dating back to Cray Blitz which even did the full Hsu SE. I still plan on testing that, but it is quite complex if you read the description. I now wonder if that helped Cray Blitz at all (or even deep thought for that matter). It might have been more valuable back when we were seeing 8-10 ply search depths in 3 minutes.
User avatar
silentshark
Posts: 327
Joined: Sat Mar 27, 2010 7:15 pm

Re: Question regarding WAC number 2

Post by silentshark »

bob wrote:
mcostalba wrote:
bob wrote:
mcostalba wrote:
bob wrote: I can answer that. zero. From testing. I removed them from Crafty and found Elo went up. I removed them from SF 1.6, one by one, and found that the elo either went up slightly or did not change. Except for the check extension itself which is worthwhile although not +20 worthwhile...
Ok, I will remove all the extension at once, apart from SE and check exstension, and test and if there is no advantage we will remove all them in one go ;-)
That's an OK test. I think it better to go one at a time, because each chess engine is different, with different rules for things like reductions, pruning, etc.
You said that you already tried to remove one by one in SF 1.6, so if removing all does not have any effect then I will trust you and take as valid your results ;-)
Should work. I was making a general comment that what works for one doesn't always work for another. Anyone following the "extension thread" ought to test their own engine carefully rather than just blindly playing "follow the leader" like lemmings. :)

Was somewhat disappointing for me since I had been doing all of these extensions for many years, dating back to Cray Blitz which even did the full Hsu SE. I still plan on testing that, but it is quite complex if you read the description. I now wonder if that helped Cray Blitz at all (or even deep thought for that matter). It might have been more valuable back when we were seeing 8-10 ply search depths in 3 minutes.
Bob, do you think at some stage even the good-old check extension won't be worth doing? I mean in 10 years, assuming we keep near to Moore's law, we will be able to search loads deeper. At some stage, I guess it won't be even worth extending checking moves..
Uri Blass
Posts: 10889
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: Question regarding WAC number 2

Post by Uri Blass »

silentshark wrote:
bob wrote:
mcostalba wrote:
bob wrote:
mcostalba wrote:
bob wrote: I can answer that. zero. From testing. I removed them from Crafty and found Elo went up. I removed them from SF 1.6, one by one, and found that the elo either went up slightly or did not change. Except for the check extension itself which is worthwhile although not +20 worthwhile...
Ok, I will remove all the extension at once, apart from SE and check exstension, and test and if there is no advantage we will remove all them in one go ;-)
That's an OK test. I think it better to go one at a time, because each chess engine is different, with different rules for things like reductions, pruning, etc.
You said that you already tried to remove one by one in SF 1.6, so if removing all does not have any effect then I will trust you and take as valid your results ;-)
Should work. I was making a general comment that what works for one doesn't always work for another. Anyone following the "extension thread" ought to test their own engine carefully rather than just blindly playing "follow the leader" like lemmings. :)

Was somewhat disappointing for me since I had been doing all of these extensions for many years, dating back to Cray Blitz which even did the full Hsu SE. I still plan on testing that, but it is quite complex if you read the description. I now wonder if that helped Cray Blitz at all (or even deep thought for that matter). It might have been more valuable back when we were seeing 8-10 ply search depths in 3 minutes.
Bob, do you think at some stage even the good-old check extension won't be worth doing? I mean in 10 years, assuming we keep near to Moore's law, we will be able to search loads deeper. At some stage, I guess it won't be even worth extending checking moves..
I cannot speak for bob but I see no reason to believe that extensions are going to be worthless and I believe that single reply extension or other extensions can be productive for Crafty assuming that you do them only in the right part of the cases and maybe bob did not find the right part of the cases.

I believe that it may be better to use some special evaluation to decide if to extend or not to extend.

basically it is better to extend only when the position is relatively unclear
and it is not clear who is winning.

For example
I see no reason to extend pawn push to the 7th if the side that push the pawn has a big advantage even without that pawn.

The cases when it is better to extend pawn go to the 7th are only cases when the position is unclear and it is not clear if the side that push the pawn is winning(in case of promoting the pawn) or losing in case of losing the pawn.
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Question regarding WAC number 2

Post by bob »

silentshark wrote:
bob wrote:
mcostalba wrote:
bob wrote:
mcostalba wrote:
bob wrote: I can answer that. zero. From testing. I removed them from Crafty and found Elo went up. I removed them from SF 1.6, one by one, and found that the elo either went up slightly or did not change. Except for the check extension itself which is worthwhile although not +20 worthwhile...
Ok, I will remove all the extension at once, apart from SE and check exstension, and test and if there is no advantage we will remove all them in one go ;-)
That's an OK test. I think it better to go one at a time, because each chess engine is different, with different rules for things like reductions, pruning, etc.
You said that you already tried to remove one by one in SF 1.6, so if removing all does not have any effect then I will trust you and take as valid your results ;-)
Should work. I was making a general comment that what works for one doesn't always work for another. Anyone following the "extension thread" ought to test their own engine carefully rather than just blindly playing "follow the leader" like lemmings. :)

Was somewhat disappointing for me since I had been doing all of these extensions for many years, dating back to Cray Blitz which even did the full Hsu SE. I still plan on testing that, but it is quite complex if you read the description. I now wonder if that helped Cray Blitz at all (or even deep thought for that matter). It might have been more valuable back when we were seeing 8-10 ply search depths in 3 minutes.
Bob, do you think at some stage even the good-old check extension won't be worth doing? I mean in 10 years, assuming we keep near to Moore's law, we will be able to search loads deeper. At some stage, I guess it won't be even worth extending checking moves..
This might well be true. The check evasion extension (today) is not a huge gain. I no longer have the exact numbers but will try to run current 23.4 with check extension disabled and will post the results...
User avatar
silentshark
Posts: 327
Joined: Sat Mar 27, 2010 7:15 pm

Re: Question regarding WAC number 2

Post by silentshark »

bob wrote:
silentshark wrote:
bob wrote:
mcostalba wrote:
bob wrote:
mcostalba wrote:
bob wrote: I can answer that. zero. From testing. I removed them from Crafty and found Elo went up. I removed them from SF 1.6, one by one, and found that the elo either went up slightly or did not change. Except for the check extension itself which is worthwhile although not +20 worthwhile...
Ok, I will remove all the extension at once, apart from SE and check exstension, and test and if there is no advantage we will remove all them in one go ;-)
That's an OK test. I think it better to go one at a time, because each chess engine is different, with different rules for things like reductions, pruning, etc.
You said that you already tried to remove one by one in SF 1.6, so if removing all does not have any effect then I will trust you and take as valid your results ;-)
Should work. I was making a general comment that what works for one doesn't always work for another. Anyone following the "extension thread" ought to test their own engine carefully rather than just blindly playing "follow the leader" like lemmings. :)

Was somewhat disappointing for me since I had been doing all of these extensions for many years, dating back to Cray Blitz which even did the full Hsu SE. I still plan on testing that, but it is quite complex if you read the description. I now wonder if that helped Cray Blitz at all (or even deep thought for that matter). It might have been more valuable back when we were seeing 8-10 ply search depths in 3 minutes.
Bob, do you think at some stage even the good-old check extension won't be worth doing? I mean in 10 years, assuming we keep near to Moore's law, we will be able to search loads deeper. At some stage, I guess it won't be even worth extending checking moves..
This might well be true. The check evasion extension (today) is not a huge gain. I no longer have the exact numbers but will try to run current 23.4 with check extension disabled and will post the results...
That will be interesting.. my guess is that it's worth a lot less now than when we were all doing 6 ply searches. Maybe it's somewhere as little as 20ELO?
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Question regarding WAC number 2

Post by bob »

Uri Blass wrote:
silentshark wrote:
bob wrote:
mcostalba wrote:
bob wrote:
mcostalba wrote:
bob wrote: I can answer that. zero. From testing. I removed them from Crafty and found Elo went up. I removed them from SF 1.6, one by one, and found that the elo either went up slightly or did not change. Except for the check extension itself which is worthwhile although not +20 worthwhile...
Ok, I will remove all the extension at once, apart from SE and check exstension, and test and if there is no advantage we will remove all them in one go ;-)
That's an OK test. I think it better to go one at a time, because each chess engine is different, with different rules for things like reductions, pruning, etc.
You said that you already tried to remove one by one in SF 1.6, so if removing all does not have any effect then I will trust you and take as valid your results ;-)
Should work. I was making a general comment that what works for one doesn't always work for another. Anyone following the "extension thread" ought to test their own engine carefully rather than just blindly playing "follow the leader" like lemmings. :)

Was somewhat disappointing for me since I had been doing all of these extensions for many years, dating back to Cray Blitz which even did the full Hsu SE. I still plan on testing that, but it is quite complex if you read the description. I now wonder if that helped Cray Blitz at all (or even deep thought for that matter). It might have been more valuable back when we were seeing 8-10 ply search depths in 3 minutes.
Bob, do you think at some stage even the good-old check extension won't be worth doing? I mean in 10 years, assuming we keep near to Moore's law, we will be able to search loads deeper. At some stage, I guess it won't be even worth extending checking moves..
I cannot speak for bob but I see no reason to believe that extensions are going to be worthless and I believe that single reply extension or other extensions can be productive for Crafty assuming that you do them only in the right part of the cases and maybe bob did not find the right part of the cases.
I tried limiting them in any way that came to mind. Yes, if you extend just the moves that need to be extended, they must gain something (how much would be a good debate topic however). And if you only reduce the moves that need to be reduced, that must also gain something.

The problem is "getting it right". I am limiting my check extensions and only apply this when SEE >= 0 for the move in question. And testing showed that this was better. There are probably other circumstances where even some non-losing checks should not be extended, for additional game.

The problem always boils down to extending too much, which hurts...


I believe that it may be better to use some special evaluation to decide if to extend or not to extend.

basically it is better to extend only when the position is relatively unclear
and it is not clear who is winning.
That won't work very well. The classic example is where I temporarily win a piece, and then check you several times to push the recapture beyond the horizon. Check extensions were originally done to prevent "spite checks" from eating two plies of search, exacerbating the horizon effect problem.

For example
I see no reason to extend pawn push to the 7th if the side that push the pawn has a big advantage even without that pawn.
For the most part, I would agree. But there will always be those oddball positions where pushing the pawn needs extending, because while not as forcing as a check (since you must escape check before doing anything else), pushing a pawn to the 7th still requires immediate attention. You could be a rook ahead and still getting mated, you certainly want to spot that...


The cases when it is better to extend pawn go to the 7th are only cases when the position is unclear and it is not clear if the side that push the pawn is winning(in case of promoting the pawn) or losing in case of losing the pawn.
The problem is "winning" is != "ahead in material" in every case.
mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: Question regarding WAC number 2

Post by mcostalba »

mcostalba wrote:
bob wrote: I can answer that. zero. From testing. I removed them from Crafty and found Elo went up. I removed them from SF 1.6, one by one, and found that the elo either went up slightly or did not change. Except for the check extension itself which is worthwhile although not +20 worthwhile...
Ok, I will remove all the extension at once, apart from SE and check exstension, and test and if there is no advantage we will remove all them in one go ;-)
Unfortunatly we are still not there :-(

Code: Select all

After 5404 games
No_extension vs Original version:  872 - 963 - 3569 ELO -5 (+- 3.8)
I don't have enough resolution to test one by one, so I think extension will still stay there.

People please, if you can, avoid stuff like "you should test with many engines" and the usual boring advices (they are not requested). Thanks in advance. :-)
jacobbl
Posts: 80
Joined: Wed Feb 17, 2010 3:57 pm

Re: Question regarding WAC number 2

Post by jacobbl »

If I understand you correctly, extentions is less usefull when you can seek very deep. In my engine, which is much weaker (just above 2000 ELO), do you still believe that there is no gain in doing extra extentions?

I've been doing some testing, but due to limited PC-resources I've just run about 2000 fast games (40 move in 1 min). I don't get any significant differances using a lot of different extentions, only check extentions or only good check extentions (SEE >= 0), but more testing is required.

Regards Jacob