MIP - Material Imbalance Pruning

[Rebel]

As far as I know Material Imbalance Pruning is a new technique that offers quite some new features to improve the search, or even the evaluation. Possibilities -

1. Reductions
2. Pruning
3. Lazy eval
4. Score corrections.

more...

[hgm]

Is this really different from evaluating the material score in centi-Pawns, and taking any decisions based on that? Rybka already used a 'material table' that was used to correct the simplistic sum-of-piece-values evaluation, and with proper tuning of the piece values (with the usual interpolation from opening to end-game) the corrections for particular combinations (except for the very pawn-poor end-game positions, which tend to be extra drawish) should be very minor.

I could also imagine that King Safety would be a much more important factor in deciding whether you can afford hefty reduction than, say, the number of Pawns you are ahead.

[Rebel]

The Rybka MIT was more complex and had different goals, MIP is a different and more simple method based on massive statistical data (13.7 billion positions) and figure out by trial on error its possibilities, I gave 4 ideas that could be tried.

[pohl4711]

As far as I know Material Imbalance Pruning is a new technique that offers quite some new features to improve the search, or even the evaluation. Possibilities -

1. Reductions
2. Pruning
3. Lazy eval
4. Score corrections.

more...

If you want to filter big pgn bases for games with material imbalance super quickly (perhaps for net-learning or so), you can use pgn-extract (I copy some parts of my InterestingWinsSearch Tool (you can use the imbalance file from the bin-folder of the tool) in here)
https://www.sp-cc.de/files/interesting_ ... ch_tool.7z

.bat-file must contain:
pgn-extract --quiet -zimbalance source.pgn --output imbalance.pgn

or (if you want to delete all comments in the output (imbalance.pgn) file)
pgn-extract --quiet -C -N -V -zimbalance source.pgn --output imbalance.pgn

or (only won games):
pgn-extract --quiet -Tr1-0 -Tr0-1 -zimbalance source.pgn --output imbalance.pgn

or (only drawn games):
pgn-extract --quiet -Tr1/2-1/2 -zimbalance source.pgn --output imbalance.pgn

or ((specific engine) combine this with -Tr, if you want, see above):
pgn-extract --quiet -Tp%engine% -zimbalance source.pgn --output imbalance.pgn
or -Tw%engine% or -Tb%engine%, when specific color is needed


imbalance-file (same folder as pgn-extract!) must contain (14 is the number of consecutive plies, one pattern has to be true, you can lower this to 10 or so with an text-editor, if you want to get more search-hits):
14 q1r*l*p* q1<=r1>=l1>=p1>=
14 q*r*l*p* q=r1<=l1>=p2>=
14 q1r*l*p* q1<=r=l3>p=
14 q1r*l*p* q1<=r2>=l=p1<=
14 q*r*l*p* q=r1<=l2>=p1<=
14 q*r*l*p* q=r2>=l2<=p4<=
14 q1r*l*p* q1<=r1>=l=p4>=
14 q1r*l*p* q1<=r=l2>=p3>=
14 q1r*l*p* q1<=r1>=l1>=p2>=
14 q*r*l*p* q=r1<=l1>=p3>=
14 q1r*l*p* q1<=r=l3>p1>=
14 q1r*l*p* q1<=r2>=l=p=
14 q*r*l*p* q=r1<=l2>=p=
14 q*r*l*p* q=r2>=l2<=p3<=
14 q1r*l*p* q1<=r1>=l=p5>=
14 q1r*l*p* q1<=r=l2>=p4>=
14 q1r*l*p* q1<=r1>=l1>=p=
14 q*r*l*p* q=r1<=l1>=p1>=
14 q1r*l*p* q1<=r=l3>p1<=
14 q1r*l*p* q1<=r2>=l=p2<=
14 q*r*l*p* q=r1<=l2>=p2<=
14 q*r*l*p* q=r2>=l2<=p5<=
14 q1r*l*p* q1<=r1>=l=p3>=
14 q1r*l*p* q1<=r=l2>=p2>=

[chrisw]

As far as I know Material Imbalance Pruning is a new technique that offers quite some new features to improve the search, or even the evaluation. Possibilities -

1. Reductions
2. Pruning
3. Lazy eval
4. Score corrections.

more...

If you want to filter big pgn bases for games with material imbalance super quickly (perhaps for net-learning or so), you can use pgn-extract (I copy some parts of my InterestingWinsSearch Tool (you can use the imbalance file from the bin-folder of the tool) in here)
https://www.sp-cc.de/files/interesting_ ... ch_tool.7z

.bat-file must contain:
pgn-extract --quiet -zimbalance source.pgn --output imbalance.pgn

or (if you want to delete all comments in the output (imbalance.pgn) file)
pgn-extract --quiet -C -N -V -zimbalance source.pgn --output imbalance.pgn

or (only won games):
pgn-extract --quiet -Tr1-0 -Tr0-1 -zimbalance source.pgn --output imbalance.pgn

or (only drawn games):
pgn-extract --quiet -Tr1/2-1/2 -zimbalance source.pgn --output imbalance.pgn

or ((specific engine) combine this with -Tr, if you want, see above):
pgn-extract --quiet -Tp%engine% -zimbalance source.pgn --output imbalance.pgn
or -Tw%engine% or -Tb%engine%, when specific color is needed


imbalance-file (same folder as pgn-extract!) must contain (14 is the number of consecutive plies, one pattern has to be true, you can lower this to 10 or so with an text-editor, if you want to get more search-hits):
14 q1r*l*p* q1<=r1>=l1>=p1>=
14 q*r*l*p* q=r1<=l1>=p2>=
14 q1r*l*p* q1<=r=l3>p=
14 q1r*l*p* q1<=r2>=l=p1<=
14 q*r*l*p* q=r1<=l2>=p1<=
14 q*r*l*p* q=r2>=l2<=p4<=
14 q1r*l*p* q1<=r1>=l=p4>=
14 q1r*l*p* q1<=r=l2>=p3>=
14 q1r*l*p* q1<=r1>=l1>=p2>=
14 q*r*l*p* q=r1<=l1>=p3>=
14 q1r*l*p* q1<=r=l3>p1>=
14 q1r*l*p* q1<=r2>=l=p=
14 q*r*l*p* q=r1<=l2>=p=
14 q*r*l*p* q=r2>=l2<=p3<=
14 q1r*l*p* q1<=r1>=l=p5>=
14 q1r*l*p* q1<=r=l2>=p4>=
14 q1r*l*p* q1<=r1>=l1>=p=
14 q*r*l*p* q=r1<=l1>=p1>=
14 q1r*l*p* q1<=r=l3>p1<=
14 q1r*l*p* q1<=r2>=l=p2<=
14 q*r*l*p* q=r1<=l2>=p2<=
14 q*r*l*p* q=r2>=l2<=p5<=
14 q1r*l*p* q1<=r1>=l=p3>=
14 q1r*l*p* q1<=r=l2>=p2>=

Directly parsing packed binary (SF format pgn learning files) in C is going to be orders of magnitude faster.