MVV/LVA move ordering and qSearch

Discussion of chess software programming and technical issues.

Moderator: Ras

Uri Blass
Posts: 10896
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: MVV/LVA move ordering and qSearch - real data

Post by Uri Blass »

bob wrote:
Uri Blass wrote:
bob wrote:hmmm...

went to a lot of trouble to run these tests. No one has a comment after seeing real data as opposed to conjectured possibilities??? Clearly PxN is better than QxQ with regard to producing a cutoff, as I had suspected. The QxQ advantage is somewhere else, _not_ in its superiority to PxN based on expected material gain.

All that is left is the original idea from the discussion when I converted to this ordering after others had reported it was better, namely that it reduces the size of the sub-tree below the capture since the Q gets removed.
I can only say that the results that you give are counter intuitive and
it may be interesting to get some epd file of random 100 positions when both PXN and QxQ are possible in the qsearch to see what happens and why do you get that there are more fail high for PxN.
Why is it "counter-intuitive" that PxN, which wins a piece for a pawn if not a piece outright, is better than QxQ which is an even trade (although obviously it might win more.) It seems completely logical to me that a move that wins material has a greater probability of failing high than a move that appears to be an even trade. What am I missing that you are thinking about?


I expected that you should get often cases when QxQ is better
because PxN is simply refuted by QxQ of the opponent like the following diagram.

[d]8/4k3/3q4/7n/6P1/3Q4/8/6K1 w - - 0 1
That makes no sense to me. It is my move, and I can play PxN. What was my opponent's last move? Only possible choice is a queen move that attacks my queen. Otherwise I could have taken his queen already. What do you suppose the probability of a hanging queen is as opposed to a defended queen?

I did some further measurements, but only a quick one-game test to see what the probability is that a QxQ wins the queen outright, as opposed to just being a trade. The only time that QxQ becomes a frequent winner is in endgames after a pawn promotion. Otherwise the queen rips get played in the normal search before the q-search is reached. It takes very unusual conditions in the q-search to expose an attack on an undefended queen, so that the opponent would not want to play PxN but would prefer to play QxQ to save his queen instead.

Remember that the q-search doesn't try non-captures for most people, which would seem to make this scenario pretty rare. The first game I gave data for above was 150+ moves long. At about 2 seconds per move average. Yet there were only about 16,000 cases where both moves could be played in the same position. And by a 2:1 majority PxN would fail high more times than QxQ.
In the diagram that I gave
if black's last move was Qc6d6 and now it is white to move in the Qsearch
then QxQ can fail high when not PxN

I cannot believe that the case that black last move is Qc6d6 is not common so
I guess that in this case usually both moves are going to fail low but
searching QxQ first may help you to fail low faster
so maybe it is a better idea to have a different qsearch for moves that you expect to fail high and for moves that you expect to fail low.

Maybe you should search PxN first if you expect the first move to fail high(you are trying to refute moves of the opponent that is not the first move) and QxQ first if you expect the move to fail low.

Uri
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: MVV/LVA move ordering and qSearch - real data

Post by bob »

Uri Blass wrote:
bob wrote:
Uri Blass wrote:
bob wrote:hmmm...

went to a lot of trouble to run these tests. No one has a comment after seeing real data as opposed to conjectured possibilities??? Clearly PxN is better than QxQ with regard to producing a cutoff, as I had suspected. The QxQ advantage is somewhere else, _not_ in its superiority to PxN based on expected material gain.

All that is left is the original idea from the discussion when I converted to this ordering after others had reported it was better, namely that it reduces the size of the sub-tree below the capture since the Q gets removed.
I can only say that the results that you give are counter intuitive and
it may be interesting to get some epd file of random 100 positions when both PXN and QxQ are possible in the qsearch to see what happens and why do you get that there are more fail high for PxN.
Why is it "counter-intuitive" that PxN, which wins a piece for a pawn if not a piece outright, is better than QxQ which is an even trade (although obviously it might win more.) It seems completely logical to me that a move that wins material has a greater probability of failing high than a move that appears to be an even trade. What am I missing that you are thinking about?


I expected that you should get often cases when QxQ is better
because PxN is simply refuted by QxQ of the opponent like the following diagram.

[d]8/4k3/3q4/7n/6P1/3Q4/8/6K1 w - - 0 1
That makes no sense to me. It is my move, and I can play PxN. What was my opponent's last move? Only possible choice is a queen move that attacks my queen. Otherwise I could have taken his queen already. What do you suppose the probability of a hanging queen is as opposed to a defended queen?

I did some further measurements, but only a quick one-game test to see what the probability is that a QxQ wins the queen outright, as opposed to just being a trade. The only time that QxQ becomes a frequent winner is in endgames after a pawn promotion. Otherwise the queen rips get played in the normal search before the q-search is reached. It takes very unusual conditions in the q-search to expose an attack on an undefended queen, so that the opponent would not want to play PxN but would prefer to play QxQ to save his queen instead.

Remember that the q-search doesn't try non-captures for most people, which would seem to make this scenario pretty rare. The first game I gave data for above was 150+ moves long. At about 2 seconds per move average. Yet there were only about 16,000 cases where both moves could be played in the same position. And by a 2:1 majority PxN would fail high more times than QxQ.
In the diagram that I gave
if black's last move was Qc6d6 and now it is white to move in the Qsearch
then QxQ can fail high when not PxN

I cannot believe that the case that black last move is Qc6d6 is not common so
I guess that in this case usually both moves are going to fail low but
searching QxQ first may help you to fail low faster
so maybe it is a better idea to have a different qsearch for moves that you expect to fail high and for moves that you expect to fail low.
Now we are getting somewhere. Namely "nowhere". Look at your position. You said the last move was probably Qc6d6. But we are doing a full-width search, which means that black could have played one of these just as easily (Qa6 [hangs queen'), (Qh6 [hangs queen]), (Q-c3 or c4 or c5 [hangs queen]), etc. So out of those moves that give me the ability to capture the queen, almost all hang your queen and SEE would jump on that instantly, just like MVV/LVA. So that is no advantage at all. In the case you gave, you picked one out of a bunch of possible squares where MVV/LVA is better.

No move ordering is perfect. I would expect to find positions where both MVV/LVA _and_ SEE get it dead wrong. But I would expect fo find far more cases where SEE is correct than when MVV/LVA is correct. I've already explained the "why" in MVV/LVA. It comes from a finite-state-machine implementation of move ordering necessary for the Belle chess machine to function. It wasn't developed because it was better than SEE. It was developed because SEE could not be done in the Belle hardware (just like hash moves were not done in Belle, nor killers).

This discussion has been about why MVV/LVA, a provably _worse_ ordering strategy than SEE, sill produces a _smaller_ search tree than SEE. It is _not_ about better ordering. SEE is simply more correct, and there is no way to prove otherwise. So, we come back full-circle again. Why is this true? The general consensus when this discussion led me to test MVV/LVA ordering in Crafty, was that capturing heavy pieces first reduces the size of the tree, because we are already in a position where we are going to fail high, we just need to get to an endpoint so we can compute a score and back it up. I still believe that to be the only reasonable explanation of why MVV/LVA is better.

I will also add, verified by testing, that if you don't do the SEE() enhancement (where you use SEE to verify that a MVV/LVA capture is not losing material) then MVV/LVA is a loser by a factor of two or so. The q-search blows up searching losing captures that are hopeless, but expensive.




Maybe you should search PxN first if you expect the first move to fail high(you are trying to refute moves of the opponent that is not the first move) and QxQ first if you expect the move to fail low.

Uri
I don't have any "expectation" unless you want to use some sort of "quick evaluation" (or the stand-pat score in a q-search). And even then what I expect to happen often does not, there is no perfect move ordering in the chess tree. I use this very idea to try to avoid parallel search splits where I expect a fail high, but I am often wrong.
User avatar
Zach Wegner
Posts: 1922
Joined: Thu Mar 09, 2006 12:51 am
Location: Earth

Re: MVV/LVA move ordering and qSearch - real data

Post by Zach Wegner »

I think your data is a bit misleading. It's very easy to fail high in qsearch, since the opponent can just stand pat. I think in general QxQ is a better move, and that PxN can almost always be done after the opponent recaptures. The only cases where it can't is where the knight recaptures, or something similar (the knight checks and then the opponent recaptures). Just to get an idea of what is happening, I had ZCT print out positions where PxN and QxQ are both possible. Most of the positions are nonsense, but in most of them QxQ looks better.

[d]Q2qk2r/p1pp1ppp/1pN1pn2/8/8/bP2P3/P1PP1PPP/RN2Kb1R b KQk - 0 8
[d]Q2qkb1r/p1pp1ppp/bp2pn2/8/3n4/1P2P3/P1PP1PPP/RNB1K2R w KQk - 0 8
[d]r3kb1r/ppp1pppp/3p1n2/4n3/5Pq1/1PN1P3/P1PP2PP/R1BQK1NR w KQkq - 1 8
[d]r1b1k2r/pppp1ppp/4p3/6q1/3n2Q1/NP2P3/P1PP1PPP/R3KB1R w KQkq - 1 8
[d]r1b1k2r/pppp1ppp/4p3/8/3n2q1/NP2P3/P1PP2PP/R2QKB1R w kq - 0 10
[d]rnb1k2r/pppp1ppp/4p3/8/2B1n3/NP1PPQ2/P1P2PqP/R3K1NR w KQkq - 0 8
[d]rnb1k2r/pppp1ppp/4p3/8/2B1n1Q1/NP1PP3/P1P2PqP/R3K1NR w KQkq - 0 8
[d]r1b1kbnr/pppp1ppp/4p3/8/1n6/PPN2N2/2PPPPPP/q2QKB1R w Kkq - 0 7
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: MVV/LVA move ordering and qSearch - real data

Post by bob »

Zach Wegner wrote:I think your data is a bit misleading. It's very easy to fail high in qsearch, since the opponent can just stand pat. I think in general QxQ is a better move, and that PxN can almost always be done after the opponent recaptures. The only cases where it can't is where the knight recaptures, or something similar (the knight checks and then the opponent recaptures). Just to get an idea of what is happening, I had ZCT print out positions where PxN and QxQ are both possible. Most of the positions are nonsense, but in most of them QxQ looks better.
You totally miss the point here. It is not about positions where QxQ and PxN are both possible, it was much more specific. QxQ is an even trade, while PxN wins material. Yet QxQ was claimed to be "a better move in general".

I explicitly measured positions where the above was true. And there, PxN is better by a _wide_ margin (factor of two). I didn't care about positions where QxQ is actually better than PxN in terms of material, SEE would get that correct more often than MVV/LVA would.

So this is _not_ about which is better overall. It is about why QxQ seems to be more effective than PxN in the overall search space. The conjecture was that QxQ was simply better than Px (for example, HGM said that when QxQ is possible, PxN is generally a very bad move."N. I set out to determine if that was true. The answer was "no". This is not about QxQ being a qualitatively better move in terms of score, it seems to be more about QxQ reducing the size of the tree. Simple SEE ordering would find all the cases where QxQ is good in general. But it would try PxN before QxQ if QxQ is just an even exchange while PxN wins material. And it would be correct in the ordering. Yet MVV/LVA gets the order wrong much more often, yet the size of the trees are apparently reduced enough to offset the poor move ordering.


[d]Q2qk2r/p1pp1ppp/1pN1pn2/8/8/bP2P3/P1PP1PPP/RN2Kb1R b KQk - 0 8
[d]Q2qkb1r/p1pp1ppp/bp2pn2/8/3n4/1P2P3/P1PP1PPP/RNB1K2R w KQk - 0 8
[d]r3kb1r/ppp1pppp/3p1n2/4n3/5Pq1/1PN1P3/P1PP2PP/R1BQK1NR w KQkq - 1 8
[d]r1b1k2r/pppp1ppp/4p3/6q1/3n2Q1/NP2P3/P1PP1PPP/R3KB1R w KQkq - 1 8
[d]r1b1k2r/pppp1ppp/4p3/8/3n2q1/NP2P3/P1PP2PP/R2QKB1R w kq - 0 10
[d]rnb1k2r/pppp1ppp/4p3/8/2B1n3/NP1PPQ2/P1P2PqP/R3K1NR w KQkq - 0 8
[d]rnb1k2r/pppp1ppp/4p3/8/2B1n1Q1/NP1PP3/P1P2PqP/R3K1NR w KQkq - 0 8
[d]r1b1kbnr/pppp1ppp/4p3/8/1n6/PPN2N2/2PPPPPP/q2QKB1R w Kkq - 0 7
Uri Blass
Posts: 10896
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: MVV/LVA move ordering and qSearch - real data

Post by Uri Blass »

<snipped>
bob wrote: I don't have any "expectation" unless you want to use some sort of "quick evaluation" (or the stand-pat score in a q-search). And even then what I expect to happen often does not, there is no perfect move ordering in the chess tree. I use this very idea to try to avoid parallel search splits where I expect a fail high, but I am often wrong.

I think about expectation based on order of moves(maybe it has a name beta type nodes or alpha type nodes but I am not sure about definitions in computer chess)

The idea is to have a list of the place of the moves in the move list and use this information to predict fail high and fail low with the idea that not first moves usually fail low.

For example in the opening position.

Suppose that you search the line 1.a4 e5 2.a5 d5 and go to qsearch with white to move


1.a4 is move number 7 in the move list for white that you searched
1...e5 is move number 1 in the move list for black that you searched
2.a5 is move number 8 in the move list for white that you searched
2...d5 is move number 1 in the move list for black that you searched

You have the following list of numbers
7,1,8,1

You can expect the qsearch to fail low

The general rule is simple
If the last number in your list is bigger than 1 you can expect the move to fail high
If the last number in your list is 1 and the previous number is bigger than 1 you can expect the move to fail low.

If the last 2 numbers are 1 then you should calculate your expectation based on the sequence without the last 2 numbers so sequence like
7,1,8,1,1,1 can be translated to 7,1,8,1 and qsearch fail low expectation
when sequence like 7,1,8,2,1,1 can be translated to 7,1,8.2 and qsearch fail high expectation.

The cases that you cannot decide because all the numbers are 1 are rare and it is not very important what is your expectation in them.

Uri
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: MVV/LVA move ordering and qSearch - real data

Post by bob »

Uri Blass wrote:<snipped>
bob wrote: I don't have any "expectation" unless you want to use some sort of "quick evaluation" (or the stand-pat score in a q-search). And even then what I expect to happen often does not, there is no perfect move ordering in the chess tree. I use this very idea to try to avoid parallel search splits where I expect a fail high, but I am often wrong.

I think about expectation based on order of moves(maybe it has a name beta type nodes or alpha type nodes but I am not sure about definitions in computer chess)

The idea is to have a list of the place of the moves in the move list and use this information to predict fail high and fail low with the idea that not first moves usually fail low.

For example in the opening position.

Suppose that you search the line 1.a4 e5 2.a5 d5 and go to qsearch with white to move


1.a4 is move number 7 in the move list for white that you searched
1...e5 is move number 1 in the move list for black that you searched
2.a5 is move number 8 in the move list for white that you searched
2...d5 is move number 1 in the move list for black that you searched

You have the following list of numbers
7,1,8,1

You can expect the qsearch to fail low

The general rule is simple
If the last number in your list is bigger than 1 you can expect the move to fail high
If the last number in your list is 1 and the previous number is bigger than 1 you can expect the move to fail low.

If the last 2 numbers are 1 then you should calculate your expectation based on the sequence without the last 2 numbers so sequence like
7,1,8,1,1,1 can be translated to 7,1,8,1 and qsearch fail low expectation
when sequence like 7,1,8,2,1,1 can be translated to 7,1,8.2 and qsearch fail high expectation.

The cases that you cannot decide because all the numbers are 1 are rare and it is not very important what is your expectation in them.

Uri
OK, that is the basic idea behind "Young Brothers Wait" in parallel search. Once the first move is completed and it doesn't fail high, there is a very high probability that this is an ALL node. But in the q-search, where we are talking about this MVV/LVA stuff, this doesn't work, because there are very few moves to search. And we are not trying to find the "best" move at a ply to search, we are trying to find the best capture instead. Except the definition of "best" has changed. Given QxQ (even) or PxN (win a piece), PxN is better. But MVV/LVA gives better results with ordering. So it isn't about "expectation" or anything else, it is about removing heavy pieces first. Which is counter-intuitive I agree, but it works because it has been tested by several, including myself.

If anyone can propose a different reason why it is better to search QxQ before PxN, when SEE says QxQ wins _less_ than PxN, then I'm all for hearing it. Intuition says this is wrong. Actual experimental results say it is correct.
Uri Blass
Posts: 10896
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: MVV/LVA move ordering and qSearch - real data

Post by Uri Blass »

bob wrote:
Uri Blass wrote:<snipped>
bob wrote: I don't have any "expectation" unless you want to use some sort of "quick evaluation" (or the stand-pat score in a q-search). And even then what I expect to happen often does not, there is no perfect move ordering in the chess tree. I use this very idea to try to avoid parallel search splits where I expect a fail high, but I am often wrong.

I think about expectation based on order of moves(maybe it has a name beta type nodes or alpha type nodes but I am not sure about definitions in computer chess)

The idea is to have a list of the place of the moves in the move list and use this information to predict fail high and fail low with the idea that not first moves usually fail low.

For example in the opening position.

Suppose that you search the line 1.a4 e5 2.a5 d5 and go to qsearch with white to move


1.a4 is move number 7 in the move list for white that you searched
1...e5 is move number 1 in the move list for black that you searched
2.a5 is move number 8 in the move list for white that you searched
2...d5 is move number 1 in the move list for black that you searched

You have the following list of numbers
7,1,8,1

You can expect the qsearch to fail low

The general rule is simple
If the last number in your list is bigger than 1 you can expect the move to fail high
If the last number in your list is 1 and the previous number is bigger than 1 you can expect the move to fail low.

If the last 2 numbers are 1 then you should calculate your expectation based on the sequence without the last 2 numbers so sequence like
7,1,8,1,1,1 can be translated to 7,1,8,1 and qsearch fail low expectation
when sequence like 7,1,8,2,1,1 can be translated to 7,1,8.2 and qsearch fail high expectation.

The cases that you cannot decide because all the numbers are 1 are rare and it is not very important what is your expectation in them.

Uri
OK, that is the basic idea behind "Young Brothers Wait" in parallel search. Once the first move is completed and it doesn't fail high, there is a very high probability that this is an ALL node. But in the q-search, where we are talking about this MVV/LVA stuff, this doesn't work, because there are very few moves to search. And we are not trying to find the "best" move at a ply to search, we are trying to find the best capture instead. Except the definition of "best" has changed. Given QxQ (even) or PxN (win a piece), PxN is better. But MVV/LVA gives better results with ordering. So it isn't about "expectation" or anything else, it is about removing heavy pieces first. Which is counter-intuitive I agree, but it works because it has been tested by several, including myself.

If anyone can propose a different reason why it is better to search QxQ before PxN, when SEE says QxQ wins _less_ than PxN, then I'm all for hearing it. Intuition says this is wrong. Actual experimental results say it is correct.
1)You can still have different statistics for all nodes and for other nodes in the qsearch based on what happened earlier.

If you already search RxQ and it failed low in the qsearch then you can fully expect fail low for QxQ so it is probably all node.

If QxQ is the first move in the qsearch then you need to go to earlier plies to decide if it is all nodes or not all nodes

My point is that statistics of better move may be different in fail high nodes and in other nodes.


2)I think that there is a possibility that QxQ(see=0) wins more material than PxN in the cases when both moves fail high so it cause QxQ to generate a smaller tree.

I cannot believe that PxN is better in most of the cases and it is still better to search first QxQ(see=0)

In theory it can be correct because of smaller tree after inferior QxQ but I think that in this case the right solution is to be more aggresive in reductions after PxN and not to search the inferior move first.

Uri
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: MVV/LVA move ordering and qSearch - real data

Post by bob »

Uri Blass wrote:
bob wrote:
Uri Blass wrote:<snipped>
bob wrote: I don't have any "expectation" unless you want to use some sort of "quick evaluation" (or the stand-pat score in a q-search). And even then what I expect to happen often does not, there is no perfect move ordering in the chess tree. I use this very idea to try to avoid parallel search splits where I expect a fail high, but I am often wrong.

I think about expectation based on order of moves(maybe it has a name beta type nodes or alpha type nodes but I am not sure about definitions in computer chess)

The idea is to have a list of the place of the moves in the move list and use this information to predict fail high and fail low with the idea that not first moves usually fail low.

For example in the opening position.

Suppose that you search the line 1.a4 e5 2.a5 d5 and go to qsearch with white to move


1.a4 is move number 7 in the move list for white that you searched
1...e5 is move number 1 in the move list for black that you searched
2.a5 is move number 8 in the move list for white that you searched
2...d5 is move number 1 in the move list for black that you searched

You have the following list of numbers
7,1,8,1

You can expect the qsearch to fail low

The general rule is simple
If the last number in your list is bigger than 1 you can expect the move to fail high
If the last number in your list is 1 and the previous number is bigger than 1 you can expect the move to fail low.

If the last 2 numbers are 1 then you should calculate your expectation based on the sequence without the last 2 numbers so sequence like
7,1,8,1,1,1 can be translated to 7,1,8,1 and qsearch fail low expectation
when sequence like 7,1,8,2,1,1 can be translated to 7,1,8.2 and qsearch fail high expectation.

The cases that you cannot decide because all the numbers are 1 are rare and it is not very important what is your expectation in them.

Uri
OK, that is the basic idea behind "Young Brothers Wait" in parallel search. Once the first move is completed and it doesn't fail high, there is a very high probability that this is an ALL node. But in the q-search, where we are talking about this MVV/LVA stuff, this doesn't work, because there are very few moves to search. And we are not trying to find the "best" move at a ply to search, we are trying to find the best capture instead. Except the definition of "best" has changed. Given QxQ (even) or PxN (win a piece), PxN is better. But MVV/LVA gives better results with ordering. So it isn't about "expectation" or anything else, it is about removing heavy pieces first. Which is counter-intuitive I agree, but it works because it has been tested by several, including myself.

If anyone can propose a different reason why it is better to search QxQ before PxN, when SEE says QxQ wins _less_ than PxN, then I'm all for hearing it. Intuition says this is wrong. Actual experimental results say it is correct.
1)You can still have different statistics for all nodes and for other nodes in the qsearch based on what happened earlier.

If you already search RxQ and it failed low in the qsearch then you can fully expect fail low for QxQ so it is probably all node.

If QxQ is the first move in the qsearch then you need to go to earlier plies to decide if it is all nodes or not all nodes

My point is that statistics of better move may be different in fail high nodes and in other nodes.


2)I think that there is a possibility that QxQ(see=0) wins more material than PxN in the cases when both moves fail high so it cause QxQ to generate a smaller tree.

I cannot believe that PxN is better in most of the cases and it is still better to search first QxQ(see=0)

In theory it can be correct because of smaller tree after inferior QxQ but I think that in this case the right solution is to be more aggresive in reductions after PxN and not to search the inferior move first.

Uri
There are no reductions after PxN. We are talking q-search. Just captures, + checks (in my case), or getting out of checks only at the 2nd ply of q-search.
Uri Blass
Posts: 10896
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: MVV/LVA move ordering and qSearch - real data

Post by Uri Blass »

bob wrote:
Uri Blass wrote:
bob wrote:
Uri Blass wrote:<snipped>
bob wrote: I don't have any "expectation" unless you want to use some sort of "quick evaluation" (or the stand-pat score in a q-search). And even then what I expect to happen often does not, there is no perfect move ordering in the chess tree. I use this very idea to try to avoid parallel search splits where I expect a fail high, but I am often wrong.

I think about expectation based on order of moves(maybe it has a name beta type nodes or alpha type nodes but I am not sure about definitions in computer chess)

The idea is to have a list of the place of the moves in the move list and use this information to predict fail high and fail low with the idea that not first moves usually fail low.

For example in the opening position.

Suppose that you search the line 1.a4 e5 2.a5 d5 and go to qsearch with white to move


1.a4 is move number 7 in the move list for white that you searched
1...e5 is move number 1 in the move list for black that you searched
2.a5 is move number 8 in the move list for white that you searched
2...d5 is move number 1 in the move list for black that you searched

You have the following list of numbers
7,1,8,1

You can expect the qsearch to fail low

The general rule is simple
If the last number in your list is bigger than 1 you can expect the move to fail high
If the last number in your list is 1 and the previous number is bigger than 1 you can expect the move to fail low.

If the last 2 numbers are 1 then you should calculate your expectation based on the sequence without the last 2 numbers so sequence like
7,1,8,1,1,1 can be translated to 7,1,8,1 and qsearch fail low expectation
when sequence like 7,1,8,2,1,1 can be translated to 7,1,8.2 and qsearch fail high expectation.

The cases that you cannot decide because all the numbers are 1 are rare and it is not very important what is your expectation in them.

Uri
OK, that is the basic idea behind "Young Brothers Wait" in parallel search. Once the first move is completed and it doesn't fail high, there is a very high probability that this is an ALL node. But in the q-search, where we are talking about this MVV/LVA stuff, this doesn't work, because there are very few moves to search. And we are not trying to find the "best" move at a ply to search, we are trying to find the best capture instead. Except the definition of "best" has changed. Given QxQ (even) or PxN (win a piece), PxN is better. But MVV/LVA gives better results with ordering. So it isn't about "expectation" or anything else, it is about removing heavy pieces first. Which is counter-intuitive I agree, but it works because it has been tested by several, including myself.

If anyone can propose a different reason why it is better to search QxQ before PxN, when SEE says QxQ wins _less_ than PxN, then I'm all for hearing it. Intuition says this is wrong. Actual experimental results say it is correct.
1)You can still have different statistics for all nodes and for other nodes in the qsearch based on what happened earlier.

If you already search RxQ and it failed low in the qsearch then you can fully expect fail low for QxQ so it is probably all node.

If QxQ is the first move in the qsearch then you need to go to earlier plies to decide if it is all nodes or not all nodes

My point is that statistics of better move may be different in fail high nodes and in other nodes.


2)I think that there is a possibility that QxQ(see=0) wins more material than PxN in the cases when both moves fail high so it cause QxQ to generate a smaller tree.

I cannot believe that PxN is better in most of the cases and it is still better to search first QxQ(see=0)

In theory it can be correct because of smaller tree after inferior QxQ but I think that in this case the right solution is to be more aggresive in reductions after PxN and not to search the inferior move first.

Uri
There are no reductions after PxN. We are talking q-search. Just captures, + checks (in my case), or getting out of checks only at the 2nd ply of q-search.
You can still have smaller qsearch.

For example not allowing equal captures in the qsearch in part of the cases when the score is bad for the side to move.

The point is that if the qsearch started with PxN then the position is bad for the side to move and in case that QxQ is an equal capture the reply QxQ is probably going to cause fail low so you may decide to skip it
in order to save time.

Uri
mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: MVV/LVA move ordering and qSearch - real data

Post by mcostalba »

Uri Blass wrote: I cannot believe that PxN is better in most of the cases and it is still better to search first QxQ(see=0)

In theory it can be correct because of smaller tree after inferior QxQ but I think that in this case the right solution is to be more aggresive in reductions after PxN and not to search the inferior move first.
Uri you miss the point that in qsearch you have very often hanging pieces. If you do some statistic you will be surprised by how much positions with hanging pieces you have in qsearch.

QXQ is _always_ the best move when the capturing queen is undefended. You cannot try PxN before, otherwise you lose your queen.