Glaurung 2.2 - no EGTB usage?

[Werner]

Hello Werner!
I'm not sure I understand your point -- there are no mate scores in the analysis above. Thanks to the incredible speed and RAM of modern computers, Glaurung does eventually search deep enough to find mates even in the most difficult KBN vs K positions, but it takes a lot more than 18 plies (around 50 plies, I would guess).

...sorry my fault - I mixed up tbs and bitbases
and I remember engines which are using bitbases have a similar output +74.95 in 5men endgames... so I thought Glaurung would use more bitbases and not only KPK

Thanks for the answer and good luck

Werner

[Tord Romstad]

For me, a draw score is not worth much without knowing why the position is drawn. "I looked this position up in a multi-gigabyte file, and it says it's a draw" isn't a very useful explanation. I haven't looked at Timman's study, but because it's composed by a human, I suppose it is possible for a human player without the help of a computer to find the solution. If my program fails to solve such a problem, and I understood the solution, I would try to identify what knowledge my program is missing and add it to my search or evaluation function. This is harder work than simply adding tablebases, but also much more valuable: I may end up dicsovering principles and evaluation rules which can be applied to a wider range of positions with a bigger number of pieces, and I may improve my own chess knowledge at the same time.

For the engine which is about to lose the endgame,knowing about draw score is as good as a win.
Endgame studies are focused on deriving rules that are useful for _humans_ to improve their endgame
play. If you insist on knowing how the WDL is arrived , you should use DTM tablebases.

That doesn't really tell much about how the WDL is arrived. A gazillion of positions, variations and DTM counts have no interest whatsoever to me, unless the results in it can be formulated in terms of some higher-level rules.

You can't program most of what is found by some KD technique from tabelbases to benefit the engine,
especially _not so in the evaluation function_, as you seem to suggest.

I've found that you often can, and you reap the benefits even in endgames with a bigger number of pieces. Here's my favorite example from Glaurung's admittedly rather crude endgame evaluation code:

[d]8/8/8/3kq3/8/5R2/6P1/6K1 w - -

All engines using 5-piece tablebases or bitbases will instantly recognize this position as a draw, as will Glaurung. Add a white pawn on b2, however, and most programs using tablebases will think that black is winning. Playing black, they will avoid capturing the b2 pawn as long as possible, because they see that the resulting position is a tablebase draw. Glaurung, using a few lines of extremely simple endgame knowledge, recognizes that the position is still a draw.

The eval already has a good enough
predictor for the final result, so to improve it you need to do more than
weights or whatever.

Yes, but I am not talking about weights, but about new heuristics and patterns.

Most of the moves involve maneuvers (tactical) that just can't be programmed there.
Ok may be you can for some endgames with 5 or less pieces, but it is impossible for most 5 piece and
above endgames without using some sort of search. Most of what is stored in bitbases are _exceptions_, the
rest are really compressed well. Those difficult positions are for humans to discover and enjoy themselves
with. My point being, the tbs are large because they have to. That ofcourse depends on what kind of information
you want from it.

My point is that the exceptions caused by purely tactical maneuvers are not interesting. We can't learn anything from such positions, neither in terms of chess, AI or computer science in general. Because they are basically won, lost or drawn just by random chance, they can't give us any valuable knowledge.

I may end up discovering principles and evaluation rules which can be applied to a wider range of positions with a bigger number of pieces,
and I may improve my own chess knowledge at the same time.

Yes you may, but not something that would be useful for your engine.
For instance how do you program the KBBKN mate in 78 to your engine
so that it would benefit from it.

I don't have any tablebases installed on my machine, but when I have let Glaurung play against non-EGTB opponents, it has usually managed to win KBBKN endgames, IIRC. My evaluation function for this endgame is very simple, but seems to work well: The attacking side tries to keep his king close to the defending side's king, to drive the defending king towards edges and corners, and to keep the defending side's king and knight far away from each other.

Even for the much simpler KBNK we have to use some table along with _search_ to drive the king to corners. Even with that my engine used to fail mating kbnk! Most of these involve manuevers which are caught with search , and if you have to do search you loose the whole point of having bitbases.

I don't see how you can mate in KBNK without a search with bitbases. All the bitbase can tell you is that the positions where either the bishop or the knight is not captured within a move or two are won for the stronger side. How do you use this information to find a mate?

I think just blindly using tablebases as giant lookup tables during the search is quite ugly and stuipd, and likely to be counter-productive in the long run. The right way to use them is probably as a testbed for high-level evaluation routines, or even for generating new high-level endgame heuristics programmatically. Using genetic programming to develop endgame evaluators using tablebase positions as a test suite would be a very interesting project, for instance.

GAs use weighing factors to give you an approximate eval.

But I wasn't talking about genetic algorithms, but about genetic programming. Once again, it is not about adjusting weights, but about discovering completely new heuristics, patterns and algorithms.

By the way, in my previous posts in this thread, I have mainly been talking about tablebases, not about bitbases. I find bitbases a lot more interesting than tablebases.

Tord

[Tord Romstad]

Hello Werner!
I'm not sure I understand your point -- there are no mate scores in the analysis above. Thanks to the incredible speed and RAM of modern computers, Glaurung does eventually search deep enough to find mates even in the most difficult KBN vs K positions, but it takes a lot more than 18 plies (around 50 plies, I would guess).

...sorry my fault - I mixed up tbs and bitbases
and I remember engines which are using bitbases have a similar output +74.95 in 5men endgames... so I thought Glaurung would use more bitbases and not only KPK

In a certain sense, you are almost correct. The score of +74.95 means that Glaurung knows that the position is won, but haven't found a mating line yet. The only difference from bitbases is how the information is stored: Instead of looking up the position in a bitbase and deciding that it is won, Glaurung decides that it is won based on evaluation rules.

Tord

[michiguel]

Hi,

Tord wrote:
Personally I find tablebases in general to be of limited interest, at least
with the current level of technology. I don't want to waste gigabytes of
disk space for something that is worth at most a handful of Elo points.

I'm not shure what you mean by "current level of technology",

Simply that nobody has, in my opinion, found an interesting and useful way to make use of the tablebases. Perhaps some interesting way to use them will emerge in the future, but we're not there yet.

perhaps you think of having the 5-men on slow disk. But it is for most people possible to copy the 4-men to memory, so you have nearly no slow down.

That depends on your machine. As my primary development platform has only 256 MB of RAM, storing the 4-men tablebases in memory isn't a very realistic option.

I also don't see the point of tablebases for 4-men endgames: Bitbases should be sufficient. I doubt that there are any won 4-man endgames which a moderatly strong modern engine wouldn't be able to win without tablebases.

For me use of table bases is not to get some elos, but to have a more certain evaluation.

For me, a draw score is not worth much without knowing why the position is drawn. "I looked this position up in a multi-gigabyte file, and it says it's a draw" isn't a very useful explanation. I haven't looked at Timman's study, but because it's composed by a human, I suppose it is possible for a human player without the help of a computer to find the solution. If my program fails to solve such a problem, and I understood the solution, I would try to identify what knowledge my program is missing and add it to my search or evaluation function. This is harder work than simply adding tablebases, but also much more valuable: I may end up dicsovering principles and evaluation rules which can be applied to a wider range of positions with a bigger number of pieces, and I may improve my own chess knowledge at the same time.

I think just blindly using tablebases as giant lookup tables during the search is quite ugly and stuipd, and likely to be counter-productive in the long run. The right way to use them is probably as a testbed for high-level evaluation routines, or even for generating new high-level endgame heuristics programmatically. Using genetic programming to develop endgame evaluators using tablebase positions as a test suite would be a very interesting project, for instance.

Tord

I may agree with you except that there is nothing ugly or stupid about the truth.

As you say in another post, you use a KPK bitbase at at startup (I do too). Why is that not ugly?

There may be smart ways to use tablebases that we do not know. I agree. but we are not going to find out if we do not try them. That is why I coded it my own egtbs. I have the feeling that the common knowledge that egtbs do not provide an increase in strength is wrong. I have the feeling that they are used in suboptimal conditions, but I have no data to back this up. In addition, if I want to analyze a position, as a chessplayer, it is important to get the best quality of analysis possible. egtbs provide that.

Miguel

[Tord Romstad]

I may agree with you except that there is nothing ugly or stupid about the truth.

As you say in another post, you use a KPK bitbase at at startup (I do too). Why is that not ugly?

It is, to a certain extent. The main reasons I do it are:

• I had to do it once, in order to learn how to do it.
• Once done, it works, and I've been too lazy to fix it.
• It's a tiny bitbase. The ugliness to me is proportional to the size of the table: I don't mind so much using 24 kB of memory to evaluate a single basic endgame perfectly, but using hundreds of megabytes for just a handful of Elo points is too much. Unless I can get at least half an Elo point per megabyte, I'm just not interested.

There may be smart ways to use tablebases that we do not know. I agree. but we are not going to find out if we do not try them.

Yes, I agree. It's a very valuable resource. I just don't think that blindly probing them and using the results without trying to understand won't get us anywhere.

That is why I coded it my own egtbs. I have the feeling that the common knowledge that egtbs do not provide an increase in strength is wrong. I have the feeling that they are used in suboptimal conditions, but I have no data to back this up. In addition, if I want to analyze a position, as a chessplayer, it is important to get the best quality of analysis possible. egtbs provide that.

I'm not a chess player, but the use of chess programs as analysis tools among non-professional chess players has always seemed strange to me. Isn't analysing without a computer both more fun and more beneficial for your chess skills?

Tord

[Zlaire]

I'm not a chess player, but the use of chess programs as analysis tools among non-professional chess players has always seemed strange to me. Isn't analysing without a computer both more fun and more beneficial for your chess skills?

For non-professional players you can look at the computer analysis as the "answer key".

Do your own analysis and come to a conclusion, then look at the computer's lines and see how far off you were...

It's like doing math problems, you might arrive at an answer but that doesn't help much if you have no way of verifying it.

[Tord Romstad]

For non-professional players you can look at the computer analysis as the "answer key".

Do your own analysis and come to a conclusion, then look at the computer's lines and see how far off you were...

It's like doing math problems, you might arrive at an answer but that doesn't help much if you have no way of verifying it.

Yes, that makes some sense, although my many years of experience in teaching various undergraduate mathematics courses have left me with somewhat mixed feelings about the value of having all the answers easily available. Being able to know when an answer is right without looking it up is an important skill in mathematics, and probably also in chess. I think the practice of giving the answer only to odd-numbered exercises, which is common in many elementary mathematics book, is a good idea.

Perhaps we should program our chess engines to refuse to analyse positions where the least significant bit of the hash key is 0.


Tord

[Daniel Shawul]

hat doesn't really tell much about how the WDL is arrived. A gazillion of positions, variations and DTM counts have no interest whatsoever to me, unless the results in it can be formulated in terms of some higher-level rules.

I've found that you often can, and you reap the benefits even in endgames with a bigger number of pieces. Here's my favorite example from Glaurung's admittedly rather crude endgame evaluation code:

[d]8/8/8/3kq3/8/5R2/6P1/6K1 w - -

All engines using 5-piece tablebases or bitbases will instantly recognize this position as a draw, as will Glaurung. Add a white pawn on b2, however, and most programs using tablebases will think that black is winning. Playing black, they will avoid capturing the b2 pawn as long as possible, because they see that the resulting position is a tablebase draw. Glaurung, using a few lines of extremely simple endgame knowledge, recognizes that the position is still a draw.

This is not as such a big knowledge discovery. It is almost always
true that if the weaker side can draw a stronger side, then adding more material to it will only enhance his chances and probably lead to a win. The real knowledge is contained in the 5 men set. One can use FREEZER to come up with such kind of positions(7men 8men) by using the already available tbs.
If you for example make Rf8 and analyze with one more pawn on b2, Glaurung first evaluates it as draw, and then search tells it it is a win. So in this case your eval fails obviously with no repercussions.

I don't have any tablebases installed on my machine, but when I have let Glaurung play against non-EGTB opponents, it has usually managed to win KBBKN endgames, IIRC. My evaluation function for this endgame is very simple, but seems to work well: The attacking side tries to keep his king close to the defending side's king, to drive the defending king towards edges and corners, and to keep the defending side's king and knight far away from each other

My point is, can it do the Mate in 78 without making a mistake against an engine using tbs? I think that it will have real difficulty to effect the Mate in 78 position. But your eval tells it to prefer the position with say +4.00 score,while the one with the tb knows it is draw from the start(and hopefully avoid exchanges which lead to the positon). Are you sure about this?

I don't see how you can mate in KBNK without a search with bitbases. All the bitbase can tell you is that the positions where either the bishop or the knight is not captured within a move or two are won for the stronger side. How do you use this information to find a mate?

Well it doesn't. It uses the same KBNK table as every one else does to try drive the king to the corner. For most other positions also using the bitbases rely on search to effect mates and draws. To have no problems with such kind of problems we obvioulsy need to store more. That is plain information theory If you want rules (least possible size), prepare for more headaches with exceptional cases.

But I wasn't talking about genetic algorithms, but about genetic programming. Once again, it is not about adjusting weights, but about discovering completely new heuristics, patterns and algorithms.

I am not sure where the big difference is? You have a bunch of parameters to be selected by some evolutionary algorithm. You select rules which decreases the entropy the most, and hope that the rules are good predictors. The KpK endgame has been the base for knowledge discovery tests for many years. I read about a case where some guys came up with all the rules (very clumsy) to solve it completely except 14 positions. So no wonder why most(including yourself) don't try to program it , rather do store/generate the kpk bitbase. I tried to classify it myself using quinlan's method but I was not as succefull.

[Tord Romstad]

This is not as such a big knowledge discovery. It is almost always
true that if the weaker side can draw a stronger side, then adding more material to it will only enhance his chances and probably lead to a win. The real knowledge is contained in the 5 men set. One can use FREEZER to come up with such kind of positions(7men 8men) by using the already available tbs.
If you for example make Rf8 and analyze with one more pawn on b2, Glaurung first evaluates it as draw, and then search tells it it is a win. So in this case your eval fails obviously with no repercussions.

I'm afraid I don't understand what you're trying to say here, and I don't know what FREEZER is...

I don't have any tablebases installed on my machine, but when I have let Glaurung play against non-EGTB opponents, it has usually managed to win KBBKN endgames, IIRC. My evaluation function for this endgame is very simple, but seems to work well: The attacking side tries to keep his king close to the defending side's king, to drive the defending king towards edges and corners, and to keep the defending side's king and knight far away from each other

My point is, can it do the Mate in 78 without making a mistake against an engine using tbs? I think that it will have real difficulty to effect the Mate in 78 position.

Probably not, but who cares? Chess engines are just computer programs, and they don't get happy if it wins a difficult endgame, or disappointed if they fail to win a theoretically won position. Computer chess isn't about winning games, but about improving human knowledge. The aim is to learn new things about chess or computer science. We're not likely to learn anything by letting chess programs play all basic endgames by looking up the positions in a giant lookup table. Trying to extract high-level knowledge from the tablebases may turn out not to be fruitful, but at least it remains possible that we will learn something from it.

The KBB vs KN endgame has limited practical importance, but if I was very interested in it, why would I try to play it by using a tablebase? That's a completely pointless and trivial exercise. A batter challenge would be to write a highly optimized search and evaluation for this particular endgame.

I don't see how you can mate in KBNK without a search with bitbases. All the bitbase can tell you is that the positions where either the bishop or the knight is not captured within a move or two are won for the stronger side. How do you use this information to find a mate?

Well it doesn't. It uses the same KBNK table as every one else does to try drive the king to the corner. For most other positions also using the bitbases rely on search to effect mates and draws.

In that case, I don't know what point you are trying to make. Whether or not bitbases are used in this particular endgame just doesn't matter.

The KpK endgame has been the base for knowledge discovery tests for many years. I read about a case where some guys came up with all the rules (very clumsy) to solve it completely except 14 positions.

I haven't read about that, but to me it sounds extremely cool. I would like to see more research in this direction. Even if nothing fundamentally new from a chess point of view were discovered, a computer program which could discover by itself the rules described in basic endgame books for humans would be extremely fascinating.

Tord

[Daniel Shawul]

I'm afraid I don't understand what you're trying to say here, and I don't know what FREEZER is...

Ok, I will explain again.
In the position you provided white (KRP) can draw black (KQ). Then it goes without saying that KRPP can atleast draw KQ.. and so on. I can actually get more benefits of this kind of situation, by using the whole 5 men tb as a base for 6 men tbs. Similar example KBP vs KQ ===> gives KBPP vs KQ, KBNP vs KQ etc. So you see I can do lots of those for 6 men if I want to by adding a small piece of code just at the beginning of probe. Trying to code that in the eval is tedious to say the least.

FREEZER is a commercial product of shredder chess found at http://www.shredderchess.com/chess-program/freezer.html . I don't actually have the software but I have read about what it can do...

Probably not, but who cares? Chess engines are just computer programs, and they don't get happy if it wins a difficult endgame, or disappointed if they fail to win a theoretically won position. Computer chess isn't about winning games, but about improving human knowledge. The aim is to learn new things about chess or computer science. We're not likely to learn anything by letting chess programs play all basic endgames by looking up the positions in a giant lookup table. Trying to extract high-level knowledge from the tablebases may turn out not to be fruitful, but at least it remains possible that we will learn something from it.

We agree on one thing , that the endgame studies are usually meant for humans. But I am more interested on improving the engine's endgame play, which I mentioned in my previous posts...

In that case, I don't know what point you are trying to make. Whether or not bitbases are used in this particular endgame just doesn't matter.

If you want less size , you lose information. Which in this case is not being be able to effect mate/raw in some very rare positions. I am willing to accept that and so does many other people who use bitbases. Now if you want only rules ,then you have even more trouble. I haven't seen one engine which does that, and thats why I brought up the KPK example to demonstrate the difficulty of it. The big size of bitbases is mainly due to exceptions. Out of the 290 five men bitbases, the most difficult ones like KQPKQ , KNPKR etc (some 10-15 bitbases) account for more than 2/3rd of the total size.

Even if nothing fundamentally new from a chess point of view were discovered, a computer program which could discover by itself the rules described in basic endgame books for humans would be extremely fascinating.

I agree there

Daniel