Reducing/Pruning Bad Captures (SEE < 0)

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

Edsel Apostol
Posts: 803
Joined: Mon Jul 17, 2006 5:53 am
Full name: Edsel Apostol

Reducing/Pruning Bad Captures (SEE < 0)

Post by Edsel Apostol »

I've tried reducing/pruning bad captures in the latest Hannibal. I've reduced a bad capture move just like what we do with LMR and prune it just like what we do with Futility Pruning with the exception that I also add the value of the captured piece.

I'm surprised to find the results very close between the one with the change and the base version. Has anyone tried this idea and have you noticed the same test results?

Here's my test results by the way:

Code: Select all

Games Completed = 1200 of 1200 &#40;Avg game length = 58.391 sec&#41;
Settings = Gauntlet/32MB/20000ms+200ms/M 1000cp for 12 moves, D 150 moves/EPD&#58;D&#58;\chess\tests\little_blitzer_2.6\NoomenCombined.epd
Time = 23653 sec elapsed, 0 sec remaining
 1.  Hannibal 20110819        	540.5/1200	411-530-259  	&#40;L&#58; m=396 t=0 i=0 a=134&#41;	&#40;D&#58; r=143 i=37 f=30 s=4 a=45&#41;	&#40;tpm=448.5 d=13.9 nps=1230646&#41;
 2.  spark-1.0                	63.5/120	45-38-37  	&#40;L&#58; m=13 t=0 i=0 a=25&#41;	&#40;D&#58; r=19 i=7 f=5 s=0 a=6&#41;	&#40;tpm=383.0 d=12.1 nps=1783891&#41;
 3.  Protector 1.4.0 x64      	60.5/120	42-41-37  	&#40;L&#58; m=11 t=0 i=0 a=30&#41;	&#40;D&#58; r=22 i=9 f=3 s=0 a=3&#41;	&#40;tpm=454.0 d=11.9 nps=901816&#41;
 4.  Spike 1.4                	56.5/120	43-50-27  	&#40;L&#58; m=11 t=0 i=0 a=39&#41;	&#40;D&#58; r=20 i=5 f=0 s=1 a=1&#41;	&#40;tpm=453.3 d=12.4 nps=965890&#41;
 5.  Gull 1.1 x64             	57.0/120	41-47-32  	&#40;L&#58; m=21 t=0 i=0 a=26&#41;	&#40;D&#58; r=18 i=3 f=3 s=1 a=7&#41;	&#40;tpm=421.4 d=12.0 nps=2125374&#41;
 6.  Gull 1.2 x64             	67.5/120	51-36-33  	&#40;L&#58; m=5 t=0 i=0 a=31&#41;	&#40;D&#58; r=19 i=0 f=8 s=1 a=5&#41;	&#40;tpm=420.4 d=13.6 nps=1811320&#41;
 7.  Critter 0.90 64-bit      	68.5/120	59-42-19  	&#40;L&#58; m=4 t=0 i=0 a=38&#41;	&#40;D&#58; r=12 i=4 f=1 s=0 a=2&#41;	&#40;tpm=448.1 d=14.8 nps=1610554&#41;
 8.  Stockfish 2.1.1 JA 64bit 	52.5/120	40-55-25  	&#40;L&#58; m=2 t=0 i=0 a=53&#41;	&#40;D&#58; r=15 i=4 f=3 s=0 a=3&#41;	&#40;tpm=457.6 d=14.7 nps=1094947&#41;
 9.  Komodo64 2.03 JA         	77.5/120	67-32-21  	&#40;L&#58; m=10 t=0 i=0 a=22&#41;	&#40;D&#58; r=7 i=1 f=3 s=1 a=9&#41;	&#40;tpm=431.7 d=12.6 nps=1131603&#41;
10.  Critter 1.2 64-bit       	70.5/120	63-42-15  	&#40;L&#58; m=2 t=0 i=0 a=40&#41;	&#40;D&#58; r=7 i=2 f=2 s=0 a=4&#41;	&#40;tpm=430.7 d=15.0 nps=1483919&#41;
11.  Houdini 1.5a x64         	85.5/120	79-28-13  	&#40;L&#58; m=0 t=0 i=0 a=28&#41;	&#40;D&#58; r=4 i=2 f=2 s=0 a=5&#41;	&#40;tpm=385.4 d=13.0 nps=1676299&#41;

Games Completed = 1200 of 1200 &#40;Avg game length = 58.395 sec&#41;
Settings = Gauntlet/32MB/20000ms+200ms/M 1000cp for 12 moves, D 150 moves/EPD&#58;D&#58;\chess\tests\little_blitzer_2.6\NoomenCombined.epd
Time = 25189 sec elapsed, 0 sec remaining
 1.  Hannibal 20110819x       	533.5/1200	393-526-281  	&#40;L&#58; m=399 t=0 i=0 a=127&#41;	&#40;D&#58; r=167 i=45 f=38 s=2 a=29&#41;	&#40;tpm=449.9 d=13.9 nps=1162299&#41;
 2.  spark-1.0                	66.0/120	50-38-32  	&#40;L&#58; m=14 t=0 i=0 a=24&#41;	&#40;D&#58; r=16 i=8 f=7 s=0 a=1&#41;	&#40;tpm=389.9 d=11.9 nps=1652586&#41;
 3.  Protector 1.4.0 x64      	62.5/120	40-35-45  	&#40;L&#58; m=13 t=0 i=0 a=22&#41;	&#40;D&#58; r=26 i=9 f=4 s=0 a=6&#41;	&#40;tpm=446.6 d=11.9 nps=826591&#41;
 4.  Spike 1.4                	58.0/120	40-44-36  	&#40;L&#58; m=11 t=0 i=0 a=33&#41;	&#40;D&#58; r=34 i=1 f=1 s=0 a=0&#41;	&#40;tpm=453.5 d=12.3 nps=929081&#41;
 5.  Gull 1.1 x64             	64.0/120	49-41-30  	&#40;L&#58; m=20 t=0 i=0 a=21&#41;	&#40;D&#58; r=22 i=3 f=2 s=0 a=3&#41;	&#40;tpm=428.8 d=11.9 nps=2033459&#41;
 6.  Gull 1.2 x64             	73.5/120	58-31-31  	&#40;L&#58; m=10 t=0 i=0 a=21&#41;	&#40;D&#58; r=16 i=4 f=6 s=0 a=5&#41;	&#40;tpm=430.3 d=13.5 nps=1687940&#41;
 7.  Critter 0.90 64-bit      	65.5/120	50-39-31  	&#40;L&#58; m=2 t=0 i=0 a=37&#41;	&#40;D&#58; r=21 i=6 f=4 s=0 a=0&#41;	&#40;tpm=434.6 d=15.1 nps=1509492&#41;
 8.  Stockfish 2.1.1 JA 64bit 	55.0/120	44-54-22  	&#40;L&#58; m=1 t=0 i=0 a=53&#41;	&#40;D&#58; r=12 i=5 f=4 s=1 a=0&#41;	&#40;tpm=452.5 d=14.2 nps=1043395&#41;
 9.  Komodo64 2.03 JA         	77.0/120	65-31-24  	&#40;L&#58; m=7 t=0 i=0 a=24&#41;	&#40;D&#58; r=11 i=5 f=5 s=0 a=3&#41;	&#40;tpm=446.1 d=12.9 nps=1040259&#41;
10.  Critter 1.2 64-bit       	68.0/120	58-42-20  	&#40;L&#58; m=2 t=0 i=0 a=40&#41;	&#40;D&#58; r=6 i=4 f=3 s=1 a=6&#41;	&#40;tpm=427.1 d=15.0 nps=1410248&#41;
11.  Houdini 1.5a x64         	77.0/120	72-38-10  	&#40;L&#58; m=1 t=0 i=0 a=37&#41;	&#40;D&#58; r=3 i=0 f=2 s=0 a=5&#41;	&#40;tpm=385.2 d=12.8 nps=1569842&#41;
20110819 is the base version. 20110819x is the version with the bad captures pruning/reduction idea.

One can notice that the 2 versions perform differently against the opponents. The 19x version performed better against stronger engines like Komodo, Critter1.2 and Houdini while performing worst against the weaker opponents. This suggests that maybe those strong engines are also doing this bad captures pruning/reduction ideas, though I might be wrong.

The difference in Elo is only 4 with the error bars at +-17. So which do you think is better, performing better against stronger engines or performing better against weaker engines?

Sometimes I consider to continue developing versions that perform a little bit worse than the best version when my intuition says that the idea has potential and just needs fine tuning.
Top
Mincho Georgiev
Posts: 454
Joined: Sat Apr 04, 2009 6:44 pm
Location: Bulgaria

Re: Reducing/Pruning Bad Captures (SEE < 0)

Post by Mincho Georgiev »

I'm not sure if I understand you correctly, but I assume you are saying that you are reducing the bad captures (not prune it). If that's the case, it is exactly what I've tried already 3 times with exactly the same results as yours, i.e. no success. I guess there is a trade-off between visited nodes and the thin tactical weakness emerged as a result of the above. Yesterday however, I found a severe bug in my see implementation which can be the cause not to have a positive result from this. I believe Bob said more than once that he is reducing the bad captures successfully, so I have to try it again after that bug was fixed.
Top
Edsel Apostol
Posts: 803
Joined: Mon Jul 17, 2006 5:53 am
Full name: Edsel Apostol

Re: Reducing/Pruning Bad Captures (SEE < 0)

Post by Edsel Apostol »

Mincho Georgiev wrote:I'm not sure if I understand you correctly, but I assume you are saying that you are reducing the bad captures (not prune it). If that's the case, it is exactly what I've tried already 3 times with exactly the same results as yours, i.e. no success. I guess there is a trade-off between visited nodes and the thin tactical weakness emerged as a result of the above. Yesterday however, I found a severe bug in my see implementation which can be the cause not to have a positive result from this. I believe Bob said more than once that he is reducing the bad captures successfully, so I have to try it again after that bug was fixed.
I am reducing them (LMR style) and pruning them (Futility pruning style + value of captured piece) at the same time. I actually have tried reducing bad captures before on my old engine TwistedLogic 20090922 version and it seems to have no noticeable difference with the version without that idea. Maybe the gain of searching 0.5 to 1.5 plies deeper on average is being offset by the resulting tactical weakness just like what you have mentioned.

Please let us know if the fix in your SEE implementation gives a positive result. It is a requirement that SEE should work correctly for this idea (reducing/pruning bad captures) to work.

You can also try pruning bad captures (Futility pruning style). I will test not doing them as I did with the 19x version, and will post the results here.
Top
Mincho Georgiev
Posts: 454
Joined: Sat Apr 04, 2009 6:44 pm
Location: Bulgaria

Re: Reducing/Pruning Bad Captures (SEE < 0)

Post by Mincho Georgiev »

I will, Edsel. It will just take some time, right now I'm running tests that shows that the bug is actually costing me 25 ELO. When this is done with good confidence, I will get back to 1. reducing the bad see evaluated captures and if it's good - 2. The FP idea that you're testing. Funny, I was just thinking on this subject last night and you came up with it today. Nice coincidence :)
Top
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Reducing/Pruning Bad Captures (SEE < 0)

Post by bob »

Edsel Apostol wrote:I've tried reducing/pruning bad captures in the latest Hannibal. I've reduced a bad capture move just like what we do with LMR and prune it just like what we do with Futility Pruning with the exception that I also add the value of the captured piece.

I'm surprised to find the results very close between the one with the change and the base version. Has anyone tried this idea and have you noticed the same test results?

Here's my test results by the way:

Code: Select all

Games Completed = 1200 of 1200 &#40;Avg game length = 58.391 sec&#41;
Settings = Gauntlet/32MB/20000ms+200ms/M 1000cp for 12 moves, D 150 moves/EPD&#58;D&#58;\chess\tests\little_blitzer_2.6\NoomenCombined.epd
Time = 23653 sec elapsed, 0 sec remaining
 1.  Hannibal 20110819        	540.5/1200	411-530-259  	&#40;L&#58; m=396 t=0 i=0 a=134&#41;	&#40;D&#58; r=143 i=37 f=30 s=4 a=45&#41;	&#40;tpm=448.5 d=13.9 nps=1230646&#41;
 2.  spark-1.0                	63.5/120	45-38-37  	&#40;L&#58; m=13 t=0 i=0 a=25&#41;	&#40;D&#58; r=19 i=7 f=5 s=0 a=6&#41;	&#40;tpm=383.0 d=12.1 nps=1783891&#41;
 3.  Protector 1.4.0 x64      	60.5/120	42-41-37  	&#40;L&#58; m=11 t=0 i=0 a=30&#41;	&#40;D&#58; r=22 i=9 f=3 s=0 a=3&#41;	&#40;tpm=454.0 d=11.9 nps=901816&#41;
 4.  Spike 1.4                	56.5/120	43-50-27  	&#40;L&#58; m=11 t=0 i=0 a=39&#41;	&#40;D&#58; r=20 i=5 f=0 s=1 a=1&#41;	&#40;tpm=453.3 d=12.4 nps=965890&#41;
 5.  Gull 1.1 x64             	57.0/120	41-47-32  	&#40;L&#58; m=21 t=0 i=0 a=26&#41;	&#40;D&#58; r=18 i=3 f=3 s=1 a=7&#41;	&#40;tpm=421.4 d=12.0 nps=2125374&#41;
 6.  Gull 1.2 x64             	67.5/120	51-36-33  	&#40;L&#58; m=5 t=0 i=0 a=31&#41;	&#40;D&#58; r=19 i=0 f=8 s=1 a=5&#41;	&#40;tpm=420.4 d=13.6 nps=1811320&#41;
 7.  Critter 0.90 64-bit      	68.5/120	59-42-19  	&#40;L&#58; m=4 t=0 i=0 a=38&#41;	&#40;D&#58; r=12 i=4 f=1 s=0 a=2&#41;	&#40;tpm=448.1 d=14.8 nps=1610554&#41;
 8.  Stockfish 2.1.1 JA 64bit 	52.5/120	40-55-25  	&#40;L&#58; m=2 t=0 i=0 a=53&#41;	&#40;D&#58; r=15 i=4 f=3 s=0 a=3&#41;	&#40;tpm=457.6 d=14.7 nps=1094947&#41;
 9.  Komodo64 2.03 JA         	77.5/120	67-32-21  	&#40;L&#58; m=10 t=0 i=0 a=22&#41;	&#40;D&#58; r=7 i=1 f=3 s=1 a=9&#41;	&#40;tpm=431.7 d=12.6 nps=1131603&#41;
10.  Critter 1.2 64-bit       	70.5/120	63-42-15  	&#40;L&#58; m=2 t=0 i=0 a=40&#41;	&#40;D&#58; r=7 i=2 f=2 s=0 a=4&#41;	&#40;tpm=430.7 d=15.0 nps=1483919&#41;
11.  Houdini 1.5a x64         	85.5/120	79-28-13  	&#40;L&#58; m=0 t=0 i=0 a=28&#41;	&#40;D&#58; r=4 i=2 f=2 s=0 a=5&#41;	&#40;tpm=385.4 d=13.0 nps=1676299&#41;

Games Completed = 1200 of 1200 &#40;Avg game length = 58.395 sec&#41;
Settings = Gauntlet/32MB/20000ms+200ms/M 1000cp for 12 moves, D 150 moves/EPD&#58;D&#58;\chess\tests\little_blitzer_2.6\NoomenCombined.epd
Time = 25189 sec elapsed, 0 sec remaining
 1.  Hannibal 20110819x       	533.5/1200	393-526-281  	&#40;L&#58; m=399 t=0 i=0 a=127&#41;	&#40;D&#58; r=167 i=45 f=38 s=2 a=29&#41;	&#40;tpm=449.9 d=13.9 nps=1162299&#41;
 2.  spark-1.0                	66.0/120	50-38-32  	&#40;L&#58; m=14 t=0 i=0 a=24&#41;	&#40;D&#58; r=16 i=8 f=7 s=0 a=1&#41;	&#40;tpm=389.9 d=11.9 nps=1652586&#41;
 3.  Protector 1.4.0 x64      	62.5/120	40-35-45  	&#40;L&#58; m=13 t=0 i=0 a=22&#41;	&#40;D&#58; r=26 i=9 f=4 s=0 a=6&#41;	&#40;tpm=446.6 d=11.9 nps=826591&#41;
 4.  Spike 1.4                	58.0/120	40-44-36  	&#40;L&#58; m=11 t=0 i=0 a=33&#41;	&#40;D&#58; r=34 i=1 f=1 s=0 a=0&#41;	&#40;tpm=453.5 d=12.3 nps=929081&#41;
 5.  Gull 1.1 x64             	64.0/120	49-41-30  	&#40;L&#58; m=20 t=0 i=0 a=21&#41;	&#40;D&#58; r=22 i=3 f=2 s=0 a=3&#41;	&#40;tpm=428.8 d=11.9 nps=2033459&#41;
 6.  Gull 1.2 x64             	73.5/120	58-31-31  	&#40;L&#58; m=10 t=0 i=0 a=21&#41;	&#40;D&#58; r=16 i=4 f=6 s=0 a=5&#41;	&#40;tpm=430.3 d=13.5 nps=1687940&#41;
 7.  Critter 0.90 64-bit      	65.5/120	50-39-31  	&#40;L&#58; m=2 t=0 i=0 a=37&#41;	&#40;D&#58; r=21 i=6 f=4 s=0 a=0&#41;	&#40;tpm=434.6 d=15.1 nps=1509492&#41;
 8.  Stockfish 2.1.1 JA 64bit 	55.0/120	44-54-22  	&#40;L&#58; m=1 t=0 i=0 a=53&#41;	&#40;D&#58; r=12 i=5 f=4 s=1 a=0&#41;	&#40;tpm=452.5 d=14.2 nps=1043395&#41;
 9.  Komodo64 2.03 JA         	77.0/120	65-31-24  	&#40;L&#58; m=7 t=0 i=0 a=24&#41;	&#40;D&#58; r=11 i=5 f=5 s=0 a=3&#41;	&#40;tpm=446.1 d=12.9 nps=1040259&#41;
10.  Critter 1.2 64-bit       	68.0/120	58-42-20  	&#40;L&#58; m=2 t=0 i=0 a=40&#41;	&#40;D&#58; r=6 i=4 f=3 s=1 a=6&#41;	&#40;tpm=427.1 d=15.0 nps=1410248&#41;
11.  Houdini 1.5a x64         	77.0/120	72-38-10  	&#40;L&#58; m=1 t=0 i=0 a=37&#41;	&#40;D&#58; r=3 i=0 f=2 s=0 a=5&#41;	&#40;tpm=385.2 d=12.8 nps=1569842&#41;
20110819 is the base version. 20110819x is the version with the bad captures pruning/reduction idea.

One can notice that the 2 versions perform differently against the opponents. The 19x version performed better against stronger engines like Komodo, Critter1.2 and Houdini while performing worst against the weaker opponents. This suggests that maybe those strong engines are also doing this bad captures pruning/reduction ideas, though I might be wrong.

The difference in Elo is only 4 with the error bars at +-17. So which do you think is better, performing better against stronger engines or performing better against weaker engines?

Sometimes I consider to continue developing versions that perform a little bit worse than the best version when my intuition says that the idea has potential and just needs fine tuning.
This has been in Crafty for a couple of versions now. I prune/reduce "bad captures" AND "bad checks". And it was worth a significant Elo improvement. I believe that by the time we were done, the changes overall were worth 15-20 Elo or so, I'll try to look up the exact results. Just did, it was actually about +12 Elo, not 15-20. The rest came from a few other minor changes...

This was first done in 23.3...
Top
zamar
Posts: 613
Joined: Sun Jan 18, 2009 7:03 am

Re: Reducing/Pruning Bad Captures (SEE < 0)

Post by zamar »

One could really expect that pruning/reducing bad captures is a great idea, but surprisingly we were never able to make this work for Stockfish.
Joona Kiiski
Top
Ferdy
Posts: 4833
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: Reducing/Pruning Bad Captures (SEE < 0)

Post by Ferdy »

IMO you need more test games, and also verify result at slower TC these changes are time sensitive.

Since introducing the bad capture reduction, I also introduced the capture killer scheme, same with non-capture killers but it is a capture, and it's see() is bad. Hope those sound capture sacrifices will be saved differently as capture killers here. Revise the move ordering for captures a bit so the capture LMR will be more defined.

Magandang gabi dyan pare.
Top
bhlangonijr
Posts: 482
Joined: Thu Oct 16, 2008 4:23 am
Location: Milky Way

Re: Reducing/Pruning Bad Captures (SEE < 0)

Post by bhlangonijr »

Edsel Apostol wrote:I've tried reducing/pruning bad captures in the latest Hannibal. I've reduced a bad capture move just like what we do with LMR and prune it just like what we do with Futility Pruning with the exception that I also add the value of the captured piece.

I'm surprised to find the results very close between the one with the change and the base version. Has anyone tried this idea and have you noticed the same test results?

Here's my test results by the way:

Code: Select all

Games Completed = 1200 of 1200 &#40;Avg game length = 58.391 sec&#41;
Settings = Gauntlet/32MB/20000ms+200ms/M 1000cp for 12 moves, D 150 moves/EPD&#58;D&#58;\chess\tests\little_blitzer_2.6\NoomenCombined.epd
Time = 23653 sec elapsed, 0 sec remaining
 1.  Hannibal 20110819        	540.5/1200	411-530-259  	&#40;L&#58; m=396 t=0 i=0 a=134&#41;	&#40;D&#58; r=143 i=37 f=30 s=4 a=45&#41;	&#40;tpm=448.5 d=13.9 nps=1230646&#41;
 2.  spark-1.0                	63.5/120	45-38-37  	&#40;L&#58; m=13 t=0 i=0 a=25&#41;	&#40;D&#58; r=19 i=7 f=5 s=0 a=6&#41;	&#40;tpm=383.0 d=12.1 nps=1783891&#41;
 3.  Protector 1.4.0 x64      	60.5/120	42-41-37  	&#40;L&#58; m=11 t=0 i=0 a=30&#41;	&#40;D&#58; r=22 i=9 f=3 s=0 a=3&#41;	&#40;tpm=454.0 d=11.9 nps=901816&#41;
 4.  Spike 1.4                	56.5/120	43-50-27  	&#40;L&#58; m=11 t=0 i=0 a=39&#41;	&#40;D&#58; r=20 i=5 f=0 s=1 a=1&#41;	&#40;tpm=453.3 d=12.4 nps=965890&#41;
 5.  Gull 1.1 x64             	57.0/120	41-47-32  	&#40;L&#58; m=21 t=0 i=0 a=26&#41;	&#40;D&#58; r=18 i=3 f=3 s=1 a=7&#41;	&#40;tpm=421.4 d=12.0 nps=2125374&#41;
 6.  Gull 1.2 x64             	67.5/120	51-36-33  	&#40;L&#58; m=5 t=0 i=0 a=31&#41;	&#40;D&#58; r=19 i=0 f=8 s=1 a=5&#41;	&#40;tpm=420.4 d=13.6 nps=1811320&#41;
 7.  Critter 0.90 64-bit      	68.5/120	59-42-19  	&#40;L&#58; m=4 t=0 i=0 a=38&#41;	&#40;D&#58; r=12 i=4 f=1 s=0 a=2&#41;	&#40;tpm=448.1 d=14.8 nps=1610554&#41;
 8.  Stockfish 2.1.1 JA 64bit 	52.5/120	40-55-25  	&#40;L&#58; m=2 t=0 i=0 a=53&#41;	&#40;D&#58; r=15 i=4 f=3 s=0 a=3&#41;	&#40;tpm=457.6 d=14.7 nps=1094947&#41;
 9.  Komodo64 2.03 JA         	77.5/120	67-32-21  	&#40;L&#58; m=10 t=0 i=0 a=22&#41;	&#40;D&#58; r=7 i=1 f=3 s=1 a=9&#41;	&#40;tpm=431.7 d=12.6 nps=1131603&#41;
10.  Critter 1.2 64-bit       	70.5/120	63-42-15  	&#40;L&#58; m=2 t=0 i=0 a=40&#41;	&#40;D&#58; r=7 i=2 f=2 s=0 a=4&#41;	&#40;tpm=430.7 d=15.0 nps=1483919&#41;
11.  Houdini 1.5a x64         	85.5/120	79-28-13  	&#40;L&#58; m=0 t=0 i=0 a=28&#41;	&#40;D&#58; r=4 i=2 f=2 s=0 a=5&#41;	&#40;tpm=385.4 d=13.0 nps=1676299&#41;

Games Completed = 1200 of 1200 &#40;Avg game length = 58.395 sec&#41;
Settings = Gauntlet/32MB/20000ms+200ms/M 1000cp for 12 moves, D 150 moves/EPD&#58;D&#58;\chess\tests\little_blitzer_2.6\NoomenCombined.epd
Time = 25189 sec elapsed, 0 sec remaining
 1.  Hannibal 20110819x       	533.5/1200	393-526-281  	&#40;L&#58; m=399 t=0 i=0 a=127&#41;	&#40;D&#58; r=167 i=45 f=38 s=2 a=29&#41;	&#40;tpm=449.9 d=13.9 nps=1162299&#41;
 2.  spark-1.0                	66.0/120	50-38-32  	&#40;L&#58; m=14 t=0 i=0 a=24&#41;	&#40;D&#58; r=16 i=8 f=7 s=0 a=1&#41;	&#40;tpm=389.9 d=11.9 nps=1652586&#41;
 3.  Protector 1.4.0 x64      	62.5/120	40-35-45  	&#40;L&#58; m=13 t=0 i=0 a=22&#41;	&#40;D&#58; r=26 i=9 f=4 s=0 a=6&#41;	&#40;tpm=446.6 d=11.9 nps=826591&#41;
 4.  Spike 1.4                	58.0/120	40-44-36  	&#40;L&#58; m=11 t=0 i=0 a=33&#41;	&#40;D&#58; r=34 i=1 f=1 s=0 a=0&#41;	&#40;tpm=453.5 d=12.3 nps=929081&#41;
 5.  Gull 1.1 x64             	64.0/120	49-41-30  	&#40;L&#58; m=20 t=0 i=0 a=21&#41;	&#40;D&#58; r=22 i=3 f=2 s=0 a=3&#41;	&#40;tpm=428.8 d=11.9 nps=2033459&#41;
 6.  Gull 1.2 x64             	73.5/120	58-31-31  	&#40;L&#58; m=10 t=0 i=0 a=21&#41;	&#40;D&#58; r=16 i=4 f=6 s=0 a=5&#41;	&#40;tpm=430.3 d=13.5 nps=1687940&#41;
 7.  Critter 0.90 64-bit      	65.5/120	50-39-31  	&#40;L&#58; m=2 t=0 i=0 a=37&#41;	&#40;D&#58; r=21 i=6 f=4 s=0 a=0&#41;	&#40;tpm=434.6 d=15.1 nps=1509492&#41;
 8.  Stockfish 2.1.1 JA 64bit 	55.0/120	44-54-22  	&#40;L&#58; m=1 t=0 i=0 a=53&#41;	&#40;D&#58; r=12 i=5 f=4 s=1 a=0&#41;	&#40;tpm=452.5 d=14.2 nps=1043395&#41;
 9.  Komodo64 2.03 JA         	77.0/120	65-31-24  	&#40;L&#58; m=7 t=0 i=0 a=24&#41;	&#40;D&#58; r=11 i=5 f=5 s=0 a=3&#41;	&#40;tpm=446.1 d=12.9 nps=1040259&#41;
10.  Critter 1.2 64-bit       	68.0/120	58-42-20  	&#40;L&#58; m=2 t=0 i=0 a=40&#41;	&#40;D&#58; r=6 i=4 f=3 s=1 a=6&#41;	&#40;tpm=427.1 d=15.0 nps=1410248&#41;
11.  Houdini 1.5a x64         	77.0/120	72-38-10  	&#40;L&#58; m=1 t=0 i=0 a=37&#41;	&#40;D&#58; r=3 i=0 f=2 s=0 a=5&#41;	&#40;tpm=385.2 d=12.8 nps=1569842&#41;
20110819 is the base version. 20110819x is the version with the bad captures pruning/reduction idea.

One can notice that the 2 versions perform differently against the opponents. The 19x version performed better against stronger engines like Komodo, Critter1.2 and Houdini while performing worst against the weaker opponents. This suggests that maybe those strong engines are also doing this bad captures pruning/reduction ideas, though I might be wrong.

The difference in Elo is only 4 with the error bars at +-17. So which do you think is better, performing better against stronger engines or performing better against weaker engines?

Sometimes I consider to continue developing versions that perform a little bit worse than the best version when my intuition says that the idea has potential and just needs fine tuning.
Do you account for pinned pieces in your see function (specially the pinned ones against the king)? My intuition says that if you are using the see function to prune/reduce bad captures then you would want to have it the more accurate as possible.

I tried reducing/pruning bad captures based on the see and didn't get good results, although my see function doesn't account for pinned pieces. I should that try that in the next days.
Top
User avatar
Evert
Posts: 2929
Joined: Sat Jan 22, 2011 12:42 am
Location: NL

Re: Reducing/Pruning Bad Captures (SEE < 0)

Post by Evert »

Ferdy wrote:Since introducing the bad capture reduction, I also introduced the capture killer scheme, same with non-capture killers but it is a capture, and it's see() is bad. Hope those sound capture sacrifices will be saved differently as capture killers here. Revise the move ordering for captures a bit so the capture LMR will be more defined.
I experimented with that in Jazz as well (where it's called a "good capture") and it seemed worse overall with initial testing, but I didn't try tweaking it. It's quite possible that a "bad SEE capture" is normally bad except in the one instance where it isn't in which case it's pointless to not reduce it.
Of course there's various ways in which this can be tweaked, for instance, where in the list do you order the "good capture"? Behind the killer moves? Ahead of other bad captures but behind quiet moves? Ahead of good captures?
Top
Edsel Apostol
Posts: 803
Joined: Mon Jul 17, 2006 5:53 am
Full name: Edsel Apostol

Re: Reducing/Pruning Bad Captures (SEE < 0)

Post by Edsel Apostol »

Okay, I have tried the reductions only without the pruning and the result seems worst, though it performed better against the stronger opponents Komodo, Critter and Houdini.

Code: Select all

Games Completed = 1200 of 1200 &#40;Avg game length = 58.333 sec&#41;
Settings = Gauntlet/32MB/20000ms+200ms/M 1000cp for 12 moves, D 150 moves/EPD&#58;D&#58;\chess\tests\little_blitzer_2.6\NoomenCombined.epd
Time = 23678 sec elapsed, 0 sec remaining
 1.  Hannibal 20110819z       	520.5/1200	386-545-269  	&#40;L&#58; m=410 t=0 i=0 a=135&#41;	&#40;D&#58; r=161 i=45 f=25 s=1 a=37&#41;	&#40;tpm=451.5 d=13.9 nps=1160334&#41;
 2.  spark-1.0                	66.0/120	48-36-36  	&#40;L&#58; m=10 t=0 i=0 a=26&#41;	&#40;D&#58; r=17 i=8 f=5 s=0 a=6&#41;	&#40;tpm=378.2 d=11.9 nps=1678926&#41;
 3.  Protector 1.4.0 x64      	62.0/120	47-43-30  	&#40;L&#58; m=15 t=0 i=0 a=28&#41;	&#40;D&#58; r=13 i=11 f=4 s=0 a=2&#41;	&#40;tpm=472.3 d=11.5 nps=810782&#41;
 4.  Spike 1.4                	65.0/120	48-38-34  	&#40;L&#58; m=9 t=0 i=0 a=29&#41;	&#40;D&#58; r=30 i=3 f=0 s=1 a=0&#41;	&#40;tpm=451.4 d=12.2 nps=913818&#41;
 5.  Gull 1.1 x64             	60.5/120	45-44-31  	&#40;L&#58; m=17 t=0 i=0 a=27&#41;	&#40;D&#58; r=15 i=10 f=3 s=0 a=3&#41;	&#40;tpm=429.2 d=11.8 nps=2021567&#41;
 6.  Gull 1.2 x64             	74.5/120	59-30-31  	&#40;L&#58; m=7 t=0 i=0 a=23&#41;	&#40;D&#58; r=22 i=2 f=4 s=0 a=3&#41;	&#40;tpm=428.4 d=13.6 nps=1703228&#41;
 7.  Critter 0.90 64-bit      	73.5/120	62-35-23  	&#40;L&#58; m=3 t=0 i=0 a=32&#41;	&#40;D&#58; r=15 i=2 f=0 s=0 a=6&#41;	&#40;tpm=447.6 d=15.1 nps=1535964&#41;
 8.  Stockfish 2.1.1 JA 64bit 	62.5/120	50-45-25  	&#40;L&#58; m=4 t=0 i=0 a=41&#41;	&#40;D&#58; r=17 i=5 f=2 s=0 a=1&#41;	&#40;tpm=457.4 d=14.1 nps=1027073&#41;
 9.  Komodo64 2.03 JA         	76.5/120	61-28-31  	&#40;L&#58; m=9 t=0 i=0 a=19&#41;	&#40;D&#58; r=16 i=2 f=4 s=0 a=9&#41;	&#40;tpm=427.4 d=13.1 nps=1062703&#41;
10.  Critter 1.2 64-bit       	66.5/120	60-47-13  	&#40;L&#58; m=1 t=0 i=0 a=46&#41;	&#40;D&#58; r=9 i=1 f=1 s=0 a=2&#41;	&#40;tpm=444.1 d=14.6 nps=1381157&#41;
11.  Houdini 1.5a x64         	72.5/120	65-40-15  	&#40;L&#58; m=0 t=0 i=0 a=40&#41;	&#40;D&#58; r=7 i=1 f=2 s=0 a=5&#41;	&#40;tpm=385.3 d=13.0 nps=1562789&#41;
So the version with prunings and reductions is around -4 elo and the version with reductions only is around -11 elo compared to the base version.

Reduction and Pruning = Base version - 4 elo
Reduction only = Base version - 11 elo

I'm going to try next the Pruning only.
Top

Return to “Computer Chess Club: Programming and Technical Discussions”