New Scorpio bitbase files

[Tony]

The reason why I use the distance from root is to avoid silly moves like this.

[d] 8/6k1/pK6/8/8/3R4/8/8 w - - 0 1

Now in this position white should just capture the black pawn and move on. But before I added the PLY factor, it evaluates almost all moves as equal. That is Kxa6,Ka5,Kc5,Kc6 etc are all the same, but obviously Kxa3 is the best because it happens earlier. The 50 move rule can not help us here because the other moves can also capture the pawn quite easily. For example Ka4->black king moves->Kxa3 (or could still move to Kb6 and continue forever until it starts looking at the 50 move rule). This turned out to be a big factor most of the times, but in some cases other factors could govern also.

One thing I forgot to mention as regards material, bonus will be given when a side is ahead in material and _also_ when the number of pieces is lower. For example a KQBKB is less prefered than a KQK even though they both have the same material differene. This is I think the other term which has a big factor.

Edit: I want to add that the example I gave above is very simple. But the white king could be further from the black pawn. In that case the white king should go for the shortest path possible to capture the black pawn.
What I experienced is since there are many ways to capture the black moves even if you make non-optimal moves, it stops making distinction between the different paths. I found the PLY factor suitable for that purpose.


Daniel

All non pawn captures will give a higher "50 move counter before first reset"

But "lower pieceCount higher score" and "distance of pawn to promotion square" will help.

Tony

[Daniel Shawul]

Replace it with the highest value the 50 move counter got before it was reset the first time in the current line.

Thinking about your suggestion, I think that both methods may be basically the same. We start with the same fifty move count at the root , and we cutoff when we find a non-reversible move. At that point the value of the fifty move count minus its value at the root is equal to the PLY (which is what I use)?

All non pawn captures will give a higher "50 move counter before first reset"

But "lower pieceCount higher score" and "distance of pawn to promotion square" will help.

Agreed.

[bob]

You are creating ad-hoc rules that can fail. For example, in a knn vs kp ending, if you go gobble the pawn, you throw away the win.

That will _never_happen becasue the bitbases are checked after the capture. Only if the capture is winning will it be considered in the first place.

I am talking about _before_ you are into the bitbase files. Capturing does not necessarily bring a win closer, whether you have the tables or not. And in fact, it can sometimes turn the win into a draw or worse.

Even worse, you may well need that capture much later to reset the 50-move counter when a mate is going to stretch beyond 50 full moves...

What could you have done in my situation?? Do you expect me to delay
the capture in the hope that it will help in avoiding the 50 move rule later?
You can only find such kind of exact information (where exactly the capture should happen) in EGTB with the mate in N condition. Assuming we have only bitbases, then all I can do at best is to weigh the different factors in to condsderation. If you have other ideas , lets hear it. Other wise stop making such silly claims which are easily refuted.

You didn't refute a single thing I wrote. If you believe that making a capture is "making progress" then go for it. I just pointed out that this is not necessarily so, and if that is your measure of "making progress" then it is a flawed measure.

Nothing more, nothing less...

I've already told you how I handle this case. I use a normal search, with a normal evaluation, which understands these concepts, and realizes when to trade or not trade pawns and when to trade or not trade pieces. It is just a paraphrasing of the GM advice "castle if you want to, or if you must, but _not_ just because you can." In this case, "capture if you want, or if you must, but not just because you can and believe that it somehow helps you make progress"...

My advice was simply that your measure of progress is wrong. You might have other good ideas/measures in your code, I don't know. But a capture is _not_ a good indicator of progress. Unless you are wanting to progress toward a draw because of the 50-move rule. Ditto for pawn advances which as also not necessarily a measure of making progress, because once moved, that opportunity is lost forever as a way of resetting the 50 move counter later in the game.

However, in any case, you can take your hostility and shove it. I have more to do than deal with that kind of nonsense, so caio...

[Tony]

Replace it with the highest value the 50 move counter got before it was reset the first time in the current line.

Thinking about your suggestion, I think that both methods may be basically the same. We start with the same fifty move count at the root , and we cutoff when we find a non-reversible move. At that point the value of the fifty move count minus its value at the root is equal to the PLY (which is what I use)?

My suggestion automaicly takes care of pawn pushes even if they are the oponents. Think about KNNKP.
The point is not only to make progress. If I let your pawn reach the 7th rank, my options are pretty limited, so it's easier for me to get the correct move.

Rereading your remark... I don't cutoff after a non-reversible move. I still like to find checkmates
If you would cutoff there is no difference I guess.

Cheers,

Tony

[Tony]

What could you have done in my situation?? Do you expect me to delay
the capture in the hope that it will help in avoiding the 50 move rule later?
You can only find such kind of exact information (where exactly the capture should happen) in EGTB with the mate in N condition. Assuming we have only bitbases, then all I can do at best is to weigh the different factors in to condsderation. If you have other ideas , lets hear it. Other wise stop making such silly claims which are easily refuted.

I'm not sure. There are variations where you should not take the first chance for transposition. OTOH transposing into a winning position with less pieces should make it simpler.

You should make some tests to settle this. Include positions where the "plain win" is more than 50 moves away without resetting the 50 move counter, so the "searching bitbases" could actually score better.

I haven't found any positions that XiniX messes up, and it didn't even have the full bitbases ( I used a lossy compression). That includes that cm in 107 ( iirc) in a KNNKP ending. But thats doesn't mean there aren't.

Tony

[Daniel Shawul]

What, and now I am being hostile one?.
Let's back up a little bit.

a) You first said "my method will not work". And then I tell you how it works and then when you are convinced it works, you _ignored_ to acknowledge until I forced you to.

b) Then when I pointed out many times why your method won't work for
a root position with 7man/8man ,then again you somehow _ignored_ to respond.

So I say from these you just want to get out of this looking good to the observer. If somebody just make some claim and forgets to acknowledge when he is found out wrong, then I would have no interest in discussion.

We all have better things to do! Infact I spent way too much time already arguing with you than doing staff i should be doing.

I am not hostile but trying to convince someone who never thinks he may be wrong could be frustrating. Therefore don't mind me if I prefer discussing with some one else.

However, in any case, you can take your hostility and shove it. I have more to do than deal with that kind of nonsense, so caio...

Now that is uncalled for... I just want to say again that I have no hostility towards you. I was hoping I could get help on parallel programming from you where you have much more experience... Still want to do that btw.

So if you want to leave the arena looking good and painting me with the "hostility" paint. Ok fine. Ciao

[Daniel Shawul]

I'm not sure. There are variations where you should not take the first chance for transposition. OTOH transposing into a winning position with less pieces should make it simpler.

Yes, if after the capture if the position (with less pieces) is still a winning position then usually it is better to do that. I tuned my evaluations in such a way that it just does not start throwing away material just to get lower pieces. My target was not to solve all of the problems, just trying to solve
most by somehow weighing the different factors.

You should make some tests to settle this. Include positions where the "plain win" is more than 50 moves away without resetting the 50 move counter, so the "searching bitbases" could actually score better.

I haven't found any positions that XiniX messes up, and it didn't even have the full bitbases ( I used a lossy compression). That includes that cm in 107 ( iirc) in a KNNKP ending. But thats doesn't mean there aren't.

That would be interesting to do. I haven't tried to solve the cm107 in KNNKP so far.It is a draw by the 50
move rule even if it solves it, but still I think if it can solve that then you have a very good way of making progress. Mine can sometimes fail to mate a difficult KQPKQ (or KRPKR). With some weight adjustments and
giving the whole search depth (instead of the 2/3 that I am using) usually solves it. My experience is if it can be solved by the regular search I have, then I can make it to be solved by what I use that was my experience.

If you have some positions to share that would be great. I will try to look for some myself.

Daniel

[Daniel Shawul]

Rereading your remark... I don't cutoff after a non-reversible move. I still like to find checkmates Smile
If you would cutoff there is no difference I guess.

Hmm... I can only find mates before any captures/promotions/pawn pushes are made. Once there are no captures left then it can spot the mate scores. In my experience , it never failed to mate a won position (i.e if the 50 move problem can be avoided)

My suggestion automaicly takes care of pawn pushes even if they are the oponents. Think about KNNKP. The point is not only to make progress. If I let your pawn reach the 7th rank, my options are pretty limited, so it's easier for me to get the correct move.

I haven't specifically tried to tune to KNNKP so may be there are things I miss there. I have the king_distance to pawns term but I don't think I have the pawn_rank term, which may be necessary for the KNNKP.
If you have some positions where Xinix benefits from that I would like to see how mine performs . It probably won't perform good since that term is probably missing.

Daniel

[Tony]

Rereading your remark... I don't cutoff after a non-reversible move. I still like to find checkmates Smile
If you would cutoff there is no difference I guess.

Hmm... I can only find mates before any captures/promotions/pawn pushes are made. Once there are no captures left then it can spot the mate scores. In my experience , it never failed to mate a won position (i.e if the 50 move problem can be avoided)

My suggestion automaicly takes care of pawn pushes even if they are the oponents. Think about KNNKP. The point is not only to make progress. If I let your pawn reach the 7th rank, my options are pretty limited, so it's easier for me to get the correct move.

I haven't specifically tried to tune to KNNKP so may be there are things I miss there. I have the king_distance to pawns term but I don't think I have the pawn_rank term, which may be necessary for the KNNKP.
If you have some positions where Xinix benefits from that I would like to see how mine performs . It probably won't perform good since that term is probably missing.

Daniel

The pawn term is implicit because of the transpose bonus.

What I didn't see you mention ( and is worthwhile) is the folowing. When in a won bitbase position, prune all the moves that don't lead to a lost position for the opponent.

Tony

[bob]

What could you have done in my situation?? Do you expect me to delay
the capture in the hope that it will help in avoiding the 50 move rule later?
You can only find such kind of exact information (where exactly the capture should happen) in EGTB with the mate in N condition. Assuming we have only bitbases, then all I can do at best is to weigh the different factors in to condsderation. If you have other ideas , lets hear it. Other wise stop making such silly claims which are easily refuted.

I'm not sure. There are variations where you should not take the first chance for transposition. OTOH transposing into a winning position with less pieces should make it simpler.

You should make some tests to settle this. Include positions where the "plain win" is more than 50 moves away without resetting the 50 move counter, so the "searching bitbases" could actually score better.

I haven't found any positions that XiniX messes up, and it didn't even have the full bitbases ( I used a lossy compression). That includes that cm in 107 ( iirc) in a KNNKP ending. But thats doesn't mean there aren't.

Tony

My concern is that "won" doesn't necessarily translate to 1-0 in the real game. There are lots of positions where capturing or advancing the pawn prematurely, even though the resulting position is won by bitbase info, leads to a draw because of the 50 move rule. KRP vs KR is a good example where you can run the pawn down to the 7th too quickly and see the win evaporate. There are similar cases such as in KNN vs KP where it is easy to let the pawn advance too far which pushes the win beyond 50 moves. This gets magnified with 6 piece endings which is really where the bitbases will become more useful because of the 1tb+ size of th egtbs. Those endings get harder and harder to win with the 50 move rule looming large.

I played around with this a good while back, and decided that a combination of the "swindle-mode" approach, along with an evaluation that had some specific code for particularly problematic endings might work. And for some endings, like KRP vs KR it worked pretty well and was able to win won positions against a version of Crafty with EGTBs which means "optimal" play against the bitbase version. But when I expanded to 6 piece files, the "rules" became way too complex and I stopped working on the problem.