Glaurung 2.2 - no EGTB usage?

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

govert
Posts: 270
Joined: Thu Jan 15, 2009 12:52 pm

Re: Glaurung 2.2 - no EGTB usage?

Post by govert »

Tord Romstad wrote: As several others have pointed out, Crafty is not and never was released under the GPL, but it is not surprising that you thought it were: There is a very widespread myth that Crafty is a GPLed program, and you'll see it described as such many places. I wonder where this myth comes from? Could it be because it used to be the "standard chess program" in Linux, and/or because people confuse Crafty with GNUChess?

Tord
My guess is that most people (including me) are not totally clear about what the GPL licence means, and thus takes a statement as " its source code is available for everyone to study and modify for personal use." as being GPLed.

btw. what IS the main difference between Crafty's licence and GPL? Maybe someone has a link that summarizes the key points with GPL?
Tord Romstad
Posts: 1808
Joined: Wed Mar 08, 2006 9:19 pm
Location: Oslo, Norway

Re: Glaurung 2.2 - no EGTB usage?

Post by Tord Romstad »

govert wrote:My guess is that most people (including me) are not totally clear about what the GPL licence means, and thus takes a statement as " its source code is available for everyone to study and modify for personal use." as being GPLed.

btw. what IS the main difference between Crafty's licence and GPL? Maybe someone has a link that summarizes the key points with GPL?
The GNU General Public License, briefly explained, gives you complete freedom to do everything you want with the program and the source code, as long as you grant the same freedom to all users of your modified program. You may distribute a modified version of the program, for free or for money, as long as your users or customers get access to your modified source code and the right to use it themselves under the conditions of the GPL.

This page gives a nice introduction to the concept of free software, the philosophy of the GNU project, and the GPL.

Crafty's license is much more restrictive, as you can see from the following excerpt from the main.c file:

Code: Select all

 *  All rights reserved.  No part of this program may be reproduced in any     *
 *  form or by any means, for other than your personal use, without the        *
 *  express written permission of the authors.  This program may not be used   *
 *  in whole, nor in part, to enter any computer chess competition without     *
 *  written permission from the authors.  Such permission will include the     *
 *  requirement that the program be entered under the name "Crafty" so that    *
 *  the program's ancestry will be known.
By contrast to the GPL, you are not allowed to sell or redistribute modified versions of Crafty without express permission of the authors, and you are not allowed to participate in tournaments using your modified version. Of course, you also couldn't participate in most tournaments with a modified version of Glaurung, but that's because of the tournament rules, and not because the license prohibits it.

Tord
govert
Posts: 270
Joined: Thu Jan 15, 2009 12:52 pm

Re: Glaurung 2.2 - no EGTB usage?

Post by govert »

Thankyou Tord.
Good explanation. Good link.
BBauer
Posts: 658
Joined: Wed Mar 08, 2006 8:58 pm

Re: Glaurung 2.2 - no EGTB usage?

Post by BBauer »

Hi,
Tord wrote:
Personally I find tablebases in general to be of limited interest, at least
with the current level of technology. I don't want to waste gigabytes of
disk space for something that is worth at most a handful of Elo points.
I'm not shure what you mean by "current level of technology", perhaps you think of having the 5-men on slow disk. But it is for most people possible to copy the 4-men to memory, so you have nearly no slow down. I remember Bob Hyatt say he copied the complete 5-men set to his 12-gig machine before starting a game.
For me use of table bases is not to get some elos, but to have a more certain evaluation.

Here an example with glaurung2.2, position fom Timmans study

Code: Select all

FEN: 8/8/8/1p1k4/3P4/4b3/KPP5/8 w - - 0 1 
Glaurung_x64:
  16	00:00	   1.560.500	2.386.085	-3,00	Ka2b3 Be3xd4 Kb3b4 Kd5c6 c2c4
  17	00:00	   1.914.429	2.495.996	-3,00	Ka2b3 Be3xd4 Kb3b4 Kd5c6 c2c4
  18	00:01	   3.006.534	2.663.006	-3,33	Ka2b3 Be3xd4 Kb3b4 Kd5c6 Kb4b3
  19	00:01	   3.975.159	2.795.470	-3,35	Ka2b3 Be3xd4 Kb3b4 Kd5c6 Kb4b3
  20	00:02	   5.427.998	2.940.410	-3,47	Ka2b3 Be3xd4 Kb3b4 Kd5c6 Kb4b3
  21	00:02	   7.721.871	3.054.537	-3,41	Ka2b3 Be3xd4 Kb3b4 Kd5c6 Kb4b3
  22	00:03	  10.653.040	3.174.326	-3,41	Ka2b3 Be3xd4 Kb3b4 Kd5c6 Kb4b3
  23	00:04	  14.106.099	3.266.056	-3,41	Ka2b3 Be3xd4 Kb3b4 Kd5c6 Kb4b3
  24	00:05	  19.451.879	3.370.041	-3,42	Ka2b3 Be3xd4 Kb3b4 Kd5c6 Kb4b3
  25	00:09	  31.083.329	3.507.484	-3,09	Ka2b3 Be3xd4 Kb3b4 Kd5c6 c2c4
  26	00:12	  42.575.952	3.583.834	-3,09	Ka2b3 Be3xd4 Kb3b4 Kd5c6 c2c4
  27	00:15	  56.927.140	3.666.568	-3,09	Ka2b3 Be3xd4 Kb3b4 Kd5c6 c2c4
  28	00:20	  75.076.421	3.697.435	-3,09	Ka2b3 Be3xd4 Kb3b4 Kd5c6 c2c4
  29	00:29	 109.324.449	3.742.065	-3,03	Ka2b3 Be3xd4 Kb3b4 Kd5c6 c2c4
  30	00:37	 140.873.684	3.795.395	-3,04	Ka2b3 Be3xd4 Kb3b4 Kd5c6 c2c4
  31	00:45	 174.410.934	3.840.128	-3,29	Ka2b3 Be3xd4 Kb3b4 Kd5c6 c2c4
  32	01:00	 233.026.500	3.886.236	-3,07	Ka2b3 Be3xd4 Kb3b4 Kd5c6 c2c4
  33	01:15	 291.376.079	3.894.100	-4,47	Ka2b3 Be3xd4 Kb3b4 Kd5c6 c2c4
  34	01:28	 345.486.008	3.904.281	-4,47	Ka2b3 Be3xd4 Kb3b4 Kd5c6 c2c4
  35	01:49	 425.437.295	3.903.345	-4,47	Ka2b3 Be3xd4 Kb3b4 Kd5c6 c2c4
  36	02:19	 540.468.545	3.875.660	-4,47	Ka2b3 Be3xd4 Kb3b4 Kd5c6 c2c4
  37	02:57	 684.245.851	3.855.840	-4,47	Ka2b3 Be3xd4 Kb3b4 Kd5c6 c2c4
  38	03:45	 864.063.033	3.843.337	-4,52	Ka2b3 Be3xd4 Kb3b4 Kd5c6 c2c4
  39	04:42	1.079.608.904	3.825.700	-4,70	Ka2b3 Be3xd4 Kb3b4 Kd5c6 c2c4
  40	05:47	1.322.740.161	3.807.333	-4,88	Ka2b3 Be3xd4 Kb3b4 Kd5c6 c2c4
  41	07:18	1.648.700.785	3.761.666	-6,00	Ka2b3 Be3xd4 Kb3b4 Kd5c6 c2c4
  42	09:33	2.133.001.703	3.718.025	-6,15	Ka2b3 Be3xd4 Kb3b4 Kd5c6 c2c4
  43	15:41	3.368.532.922	3.577.537	-6,74	Ka2b3 Be3xd4 Kb3b4 Kd5c6 c2c4
  44	29:10	6.055.444.334	3.460.542	-6,75	Ka2b3 Be3xd4 Kb3b4 Kd5c6 c2c4 Bd4c5+ 
        Kb4c3 b5b4+ Kc3d3 Kc6d6 Kd3e4 b4b3 Ke4d3 Bc5b4 Kd3d4 Kd6c6 c4c5 Kc6b5
        c5c6 Bb4a5 Kd4d3 Kb5b4 Kd3d4 Ba5c7 Kd4d3 Bc7b6 Kd3d2 Kb4c4 Kd2e2 Kc4d4
        Ke2d1 Bb6a5 Kd1e2 Kd4e4 Ke2d1 Ke4d5 Kd1e2 Kd5xc6 Ke2d1 Ba5c7 Kd1d2 Bc7e5 
        Kd2c1 Be5f4+ Kc1b1 Kc6d5 Kb1a1 Kd5c6
The evaluation goes down to -6.75 though glaurung2.2 finds the way to a draw. Of course tablebases as well as bitbases need a good implementation to your code.
kind regards
Bernhard
User avatar
Werner
Posts: 2994
Joined: Wed Mar 08, 2006 10:09 pm
Location: Germany
Full name: Werner Schüle

Re: Glaurung 2.2 - no EGTB usage?

Post by Werner »

Hi Tord,
if you use no tbs, so why does Glaurung evaluate KNB - K with a mate score?

FEN: 4k3/8/8/8/8/8/4N3/4KB2 w - - 0 1

Glaurung 2.2 w32 1CPU:
2 00:00 23 0 +74.36 Ke1d2 Ke8f8
3 00:00 96 0 +74.46 Ke1d2 Ke8f8 Kd2c3
4 00:00 224 0 +74.41 Ke1d2 Ke8f8 Kd2c3 Kf8g8
5 00:00 422 0 +74.50 Ke1d2 Ke8f8 Kd2c3 Kf8g8 Kc3c4
6 00:00 704 0 +74.46 Ke1d2 Ke8f8 Kd2c3 Kf8g8 Kc3d4 Kg8h8
7 00:00 1.709 0 +74.55 Ke1d2 Ke8f8 Kd2c3 Kf8g8 Kc3d4 Kg8h8 Kd4e5
8 00:00 3.348 0 +74.55 Ke1d2 Ke8f8 Kd2c3 Kf8g8 Kc3d4 Kg8g7 Kd4e5 Kg7h8
9 00:00 6.983 0 +74.63 Ke1d2 Ke8f8 Kd2c3 Kf8f7 Kc3d4 Kf7e6 Se2c1 Ke6f6 Kd4d5
10 00:00 14.675 0 +74.60 Ke1d2 Ke8f8 Kd2c3 Kf8f7 Kc3d4 Kf7e6 Kd4e4 Ke6f7 Ke4e5 Kf7g8
11 00:00 28.826 0 +74.63 Ke1d2 Ke8f8 Kd2c3 Kf8f7 Kc3d4 Kf7f6 Kd4d5 Kf6f5 Se2c1 Kf5f6 Kd5d4
12 00:00 64.975 2.030.468 +74.65 Ke1d2 Ke8f8 Kd2c3 Kf8f7 Kc3d4 Kf7f6 Kd4d5 Kf6f5 Se2d4+ Kf5g6 Kd5e5 Kg6g7 Lf1c4
13 00:00 119.171 1.891.603 +74.60 Ke1d2 Ke8f8 Kd2c3 Kf8f7 Kc3d4 Kf7f6 Kd4e4 Kf6e6 Se2f4+ Ke6f6 Ke4d4 Kf6f7 Kd4e5 Kf7g8
14 00:00 235.496 2.140.872 +74.65 Ke1d2 Ke8f8 Kd2e3 Kf8g7 Ke3e4 Kg7g6 Se2f4+ Kg6g7 Ke4e5 Kg7f7 Sf4e2 Kf7g8 Ke5f6 Kg8h8 Se2c1
15 00:00 418.136 2.059.783 +74.70 Ke1d2 Ke8f8 Kd2e3 Kf8g7 Ke3e4 Kg7g6 Se2f4+ Kg6g7 Ke4f5 Kg7f7 Lf1c4+ Kf7g7 Sf4g6 Kg7h6 Kf5f6 Kh6h7 Lc4f1
16 00:00 1.022.207 2.107.643 +74.70 Ke1d2 Ke8f7 Kd2e3 Kf7e6 Ke3e4 Ke6d6 Se2f4 Kd6c5 Sf4d5 Kc5d6 Lf1e2 Kd6e6 Le2h5 Ke6d6 Ke4d4 Kd6e6
17 00:01 1.621.826 2.162.434 +74.75 Ke1d2 Ke8f7 Kd2e3 Kf7e6 Se2f4+ Ke6e5 Lf1d3 Ke5f6 Ke3e4 Kf6g7 Ld3f1 Kg7f6 Lf1c4 Kf6g5 Ke4e5 Kg5g4 Lc4e2+ Kg4g5 Le2f1
18 00:01 3.136.926 2.205.995 +74.75 Ke1d2 Ke8f7 Se2f4 Kf7f6 Kd2e2 Kf6g7 Ke2e3 Kg7h6 Ke3e4 Kh6g7 Ke4e5 Kg7f8 Ke5f6 Kf8g8 Lf1c4+ Kg8h8 Sf4g6+ Kh8h7 Lc4a2 Kh7h6 La2c4

In older installation I found a file KPK.bin - so is it possible to generate more endgames and use them ??

best
Werner
Tord Romstad
Posts: 1808
Joined: Wed Mar 08, 2006 9:19 pm
Location: Oslo, Norway

Re: Glaurung 2.2 - no EGTB usage?

Post by Tord Romstad »

BBauer wrote:Hi,
Tord wrote:
Personally I find tablebases in general to be of limited interest, at least
with the current level of technology. I don't want to waste gigabytes of
disk space for something that is worth at most a handful of Elo points.
I'm not shure what you mean by "current level of technology",
Simply that nobody has, in my opinion, found an interesting and useful way to make use of the tablebases. Perhaps some interesting way to use them will emerge in the future, but we're not there yet.
perhaps you think of having the 5-men on slow disk. But it is for most people possible to copy the 4-men to memory, so you have nearly no slow down.
That depends on your machine. As my primary development platform has only 256 MB of RAM, storing the 4-men tablebases in memory isn't a very realistic option.

I also don't see the point of tablebases for 4-men endgames: Bitbases should be sufficient. I doubt that there are any won 4-man endgames which a moderatly strong modern engine wouldn't be able to win without tablebases.
For me use of table bases is not to get some elos, but to have a more certain evaluation.
For me, a draw score is not worth much without knowing why the position is drawn. "I looked this position up in a multi-gigabyte file, and it says it's a draw" isn't a very useful explanation. I haven't looked at Timman's study, but because it's composed by a human, I suppose it is possible for a human player without the help of a computer to find the solution. If my program fails to solve such a problem, and I understood the solution, I would try to identify what knowledge my program is missing and add it to my search or evaluation function. This is harder work than simply adding tablebases, but also much more valuable: I may end up dicsovering principles and evaluation rules which can be applied to a wider range of positions with a bigger number of pieces, and I may improve my own chess knowledge at the same time.

I think just blindly using tablebases as giant lookup tables during the search is quite ugly and stuipd, and likely to be counter-productive in the long run. The right way to use them is probably as a testbed for high-level evaluation routines, or even for generating new high-level endgame heuristics programmatically. Using genetic programming to develop endgame evaluators using tablebase positions as a test suite would be a very interesting project, for instance.

Tord
Tord Romstad
Posts: 1808
Joined: Wed Mar 08, 2006 9:19 pm
Location: Oslo, Norway

Re: Glaurung 2.2 - no EGTB usage?

Post by Tord Romstad »

Hello Werner!
Werner wrote:Hi Tord,
if you use no tbs, so why does Glaurung evaluate KNB - K with a mate score?

FEN: 4k3/8/8/8/8/8/4N3/4KB2 w - - 0 1

Glaurung 2.2 w32 1CPU:
2 00:00 23 0 +74.36 Ke1d2 Ke8f8
3 00:00 96 0 +74.46 Ke1d2 Ke8f8 Kd2c3
4 00:00 224 0 +74.41 Ke1d2 Ke8f8 Kd2c3 Kf8g8
5 00:00 422 0 +74.50 Ke1d2 Ke8f8 Kd2c3 Kf8g8 Kc3c4
6 00:00 704 0 +74.46 Ke1d2 Ke8f8 Kd2c3 Kf8g8 Kc3d4 Kg8h8
7 00:00 1.709 0 +74.55 Ke1d2 Ke8f8 Kd2c3 Kf8g8 Kc3d4 Kg8h8 Kd4e5
8 00:00 3.348 0 +74.55 Ke1d2 Ke8f8 Kd2c3 Kf8g8 Kc3d4 Kg8g7 Kd4e5 Kg7h8
9 00:00 6.983 0 +74.63 Ke1d2 Ke8f8 Kd2c3 Kf8f7 Kc3d4 Kf7e6 Se2c1 Ke6f6 Kd4d5
10 00:00 14.675 0 +74.60 Ke1d2 Ke8f8 Kd2c3 Kf8f7 Kc3d4 Kf7e6 Kd4e4 Ke6f7 Ke4e5 Kf7g8
11 00:00 28.826 0 +74.63 Ke1d2 Ke8f8 Kd2c3 Kf8f7 Kc3d4 Kf7f6 Kd4d5 Kf6f5 Se2c1 Kf5f6 Kd5d4
12 00:00 64.975 2.030.468 +74.65 Ke1d2 Ke8f8 Kd2c3 Kf8f7 Kc3d4 Kf7f6 Kd4d5 Kf6f5 Se2d4+ Kf5g6 Kd5e5 Kg6g7 Lf1c4
13 00:00 119.171 1.891.603 +74.60 Ke1d2 Ke8f8 Kd2c3 Kf8f7 Kc3d4 Kf7f6 Kd4e4 Kf6e6 Se2f4+ Ke6f6 Ke4d4 Kf6f7 Kd4e5 Kf7g8
14 00:00 235.496 2.140.872 +74.65 Ke1d2 Ke8f8 Kd2e3 Kf8g7 Ke3e4 Kg7g6 Se2f4+ Kg6g7 Ke4e5 Kg7f7 Sf4e2 Kf7g8 Ke5f6 Kg8h8 Se2c1
15 00:00 418.136 2.059.783 +74.70 Ke1d2 Ke8f8 Kd2e3 Kf8g7 Ke3e4 Kg7g6 Se2f4+ Kg6g7 Ke4f5 Kg7f7 Lf1c4+ Kf7g7 Sf4g6 Kg7h6 Kf5f6 Kh6h7 Lc4f1
16 00:00 1.022.207 2.107.643 +74.70 Ke1d2 Ke8f7 Kd2e3 Kf7e6 Ke3e4 Ke6d6 Se2f4 Kd6c5 Sf4d5 Kc5d6 Lf1e2 Kd6e6 Le2h5 Ke6d6 Ke4d4 Kd6e6
17 00:01 1.621.826 2.162.434 +74.75 Ke1d2 Ke8f7 Kd2e3 Kf7e6 Se2f4+ Ke6e5 Lf1d3 Ke5f6 Ke3e4 Kf6g7 Ld3f1 Kg7f6 Lf1c4 Kf6g5 Ke4e5 Kg5g4 Lc4e2+ Kg4g5 Le2f1
18 00:01 3.136.926 2.205.995 +74.75 Ke1d2 Ke8f7 Se2f4 Kf7f6 Kd2e2 Kf6g7 Ke2e3 Kg7h6 Ke3e4 Kh6g7 Ke4e5 Kg7f8 Ke5f6 Kf8g8 Lf1c4+ Kg8h8 Sf4g6+ Kh8h7 Lc4a2 Kh7h6 La2c4
I'm not sure I understand your point -- there are no mate scores in the analysis above. Thanks to the incredible speed and RAM of modern computers, Glaurung does eventually search deep enough to find mates even in the most difficult KBN vs K positions, but it takes a lot more than 18 plies (around 50 plies, I would guess).

As long as my program is able to mate with KBN vs K when playing, even with just a couple of seconds for the whole game, I don't care that it needs a long time to announce mate in positions like the one above.
In older installation I found a file KPK.bin
That's not a tablebase, but a bitbase for KP vs K. It still exists in the newer versions, but no longer as a separate file. It's computed from scratch every time the engine starts up.
- so is it possible to generate more endgames and use them ??
Not at the moment, but perhaps some time in the future. It will probably be in the form of bitbases and not tablebases, though.

Just before I pressed the "Submit" button, Glaurung actually found the mate in your KBN vs K position: It took 49 plies and a little more than 30 minutes:

Code: Select all

20    +74.75   00:00    5335k Kd2 Kf8 Kc3 Kf7 Kd4 Kg6 Ke5 Kf7 Bg2 Kg7 Nf4 Kf7
                              Bd5+ Kg7 Kf5 Kh8 Kf6 Kh7 Ng6 Kh6 Nf8
21    +74.75   00:02   12435k Kd2 Kf8 Kc3 Kf7 Kd4 Ke6 Bg2 Kd6 Nf4 Ke7 Ke5 Kd7
                              Kf6 Kd6 Ne6 Kd7 Nc5+ Kd6 Nb7+ Kc7 Bd5 Kb6 Ke6
22    +74.80   00:03   20165k Kd2 Kf8 Kc3 Kf7 Kd4 Ke6 Bg2 Kd6 Nf4 Ke7 Ke5 Kd7
                              Kd5 Kc7 Bf1 Kd8 Ne2 Ke7 Bh3 Kd8 Kd6 Ke8
23    +74.80   00:04   25346k Kd2 Kf8 Kc3 Kf7 Kd4 Ke6 Bg2 Kd6 Nf4 Ke7 Ke5 Kd7
                              Kd5 Kc7 Bf1 Kd8 Ne2 Ke7 Bh3 Kd8 Kd6 Ke8 Nc1
24    +74.80   00:05   34887k Kd2 Kf8 Kc3 Kf7 Kd4 Ke6 Bg2 Kd6 Nf4 Ke7 Ke5 Kd7
                              Kd5 Kc7 Bf1 Kd8 Bc4 Ke7 Ke5 Kf8 Kf6 Ke8 Be6 Kf8
                              Ng6+ Ke8
25    +74.80   00:08   49742k Kd2 Kf8 Kc3 Kf7 Kd4 Ke6 Bg2 Kd6 Nf4 Ke7 Ke5 Kd7
                              Kd5 Kc7 Bf1 Kb6 Bd3 Ka5 Kc5 Ka4 Bc4 Ka3 Nd3 Ka4
                              Kc6
26    +74.85   00:11   67773k Kd2 Kf8 Kc3 Kf7 Kd4 Ke6 Bg2 Kd6 Nf4 Ke7 Ke5 Kd7
                              Bd5 Ke7 Ng6+ Kd7 Kd4 Kd6 Kc4 Kc7 Bg2 Kb6 Bh3 Ka5
                              Bd7 Kb6 Kd4
27    +74.85   00:17  106163k Kd2 Kf8 Kc3 Kf7 Kd4 Ke6 Bg2 Kd6 Nf4 Ke7 Ke5 Kd7
                              Bd5 Ke7 Ng6+ Kd7 Kd4 Kd6 Kc4 Kc7 Kc5 Kd7 Bc4 Kd8
                              Kc6 Ke8 Bd5 Kd8
28    +74.85   00:24  145615k Kd2 Kf8 Kc3 Kf7 Kd4 Ke6 Bg2 Kd6 Nf4 Ke7 Ke5 Kd7
                              Bd5 Ke7 Ng6+ Kd7 Kd4 Kd6 Kc4 Kd7 Kc5 Kc7 Bg2 Kd7
                              Bh3+ Kc7 Be6 Kb7 Ne7 Kc7
29    +74.90   00:36  219048k Kd2 Kf8 Kc3 Kf7 Kd4 Ke6 Bg2 Kd6 Bd5 Ke7 Ke5 Kd7
                              Nf4 Ke7 Ng6+ Kd7 Kd4 Kd6 Kc4 Kd7 Kc5 Kc7 Bc6 Kc8
                              Kd6 Kb8 Be8 Ka7 Kc6 Ka6
30    +74.90   00:57  350179k Kd2 Kf8 Kc3 Kf7 Kd4 Ke6 Bg2 Kd6 Bd5 Ke7 Ke5 Kd7
                              Nf4 Ke7 Ng6+ Kd7 Kd4 Kd6 Kc4 Kd7 Kc5 Kc7 Bc6 Kc8
                              Kd6 Kb8 Be8 Ka7 Kc6 Ka6 Kc5
31    +74.95   01:26  521517k Kd2 Kf8 Ng3 Kf7 Bc4+ Ke7 Ke3 Kf6 Kf4 Kg6 Ke5 Kg5
                              Ne4+ Kg6 Bd3 Kg7 Kf5 Kf8 Kf6 Kg8 Be2 Kh7 Nf2 Kh6
                              Ng4+ Kh5 Kf5 Kh4 Nf2 Kg3 Ne4+ Kg2 Kf4 Kh2 Bc4
32    +74.95   01:40  601944k Kd2 Kf8 Ng3 Kf7 Bc4+ Ke7 Ke3 Kf6 Kf4 Kg6 Ke5 Kg5
                              Ne4+ Kg6 Nd6 Kg5 Nf7+ Kg6 Ke4 Kh5 Kf5 Kh4 Kf4 Kh5
                              Bd3 Kh4 Be2 Kh3 Nh6 Kh2 Kf3 Kh3 Nf5 Kh2 Bf1
33    +75.00   02:13  803767k Kd2 Kf8 Kc3 Kf7 Kd4 Kg7 Ke5 Kg6 Nc1 Kg5 Bd3 Kg4
                              Bf5+ Kf3 Nb3 Kf2 Ke4 Kg3 Nd4 Kf2 Bg4 Kg3 Bd1 Kf2
                              Bf3 Kf1 Kf4 Kf2 Nc2 Kf1 Kg3 Kg1 Be2 Kh1
34    +75.00   02:24  865725k Kd2 Kf8 Kc3 Kf7 Kd4 Kg7 Ke5 Kg6 Nc1 Kg5 Bd3 Kg4
                              Bf5+ Kf3 Nb3 Kf2 Ke4 Kg3 Nd4 Kf2 Bg4 Kg3 Bd1 Kf2
                              Bf3 Kf1 Kf4 Kf2 Nc2 Kf1 Kg3 Kg1 Be2 Kh1 Bd3
35    +75.00   02:41  960261k Kd2 Kf8 Kc3 Kf7 Kd4 Kg7 Ke5 Kg6 Nc1 Kg5 Bd3 Kg4
                              Bf5+ Kf3 Nb3 Kf2 Ke4 Kg3 Nd4 Kf2 Bg4 Kg3 Bd1 Kf2
                              Bf3 Kf1 Kf4 Kf2 Nc2 Kf1 Kg3 Kg1 Be2 Kh1 Nd4 Kg1
                              Nf3+ Kh1
36    +75.00   02:55    1040M Kd2 Kf8 Kc3 Kf7 Kd4 Kg7 Ke5 Kg6 Nc1 Kg5 Bd3 Kg4
                              Bf5+ Kf3 Nb3 Kf2 Ke4 Kg3 Nd4 Kf2 Bg4 Kg3 Bd1 Kf2
                              Bf3 Kf1 Kf4 Kf2 Nc2 Kf1 Kg3 Kg1 Be2 Kh1 Nd4 Kg1
                              Nf3+ Kh1 Ng5
37    +75.00   03:07    1103M Kd2 Kf8 Kc3 Kf7 Kd4 Kg7 Ke5 Kg6 Nc1 Kg5 Bd3 Kg4
                              Bf5+ Kf3 Nb3 Kf2 Ke4 Kg3 Nd4 Kf2 Bg4 Kg3 Bd1 Kf2
                              Bf3 Kf1 Kf4 Kf2 Nc2 Kf1 Kg3 Kg1 Be2 Kh1 Nd4 Kg1
                              Nf3+ Kh1 Ng5 Kg1 Nh3+ Kh1
38    +75.00   03:20    1174M Kd2 Kf8 Kc3 Kf7 Kd4 Kg7 Ke5 Kg6 Nc1 Kg5 Bd3 Kg4
                              Bf5+ Kf3 Nb3 Kg3 Nd4 Kh4 Be4 Kg5 Nc6 Kg4 Bd5 Kg3
                              Nd4 Kf2 Bc4 Ke1 Nf3+ Kf2 Kf4 Kg2 Be6 Kh1 Ng5 Kh2
                              Bh3 Kg1 Kg3 Kh1
39    +75.00   03:30    1231M Kd2 Kf8 Kc3 Kf7 Kd4 Kg7 Ke5 Kg6 Nc1 Kg5 Bd3 Kg4
                              Bf5+ Kf3 Nb3 Kg3 Nd4 Kh4 Be4 Kg5 Nc6 Kg4 Bd5 Kg3
                              Nd4 Kf2 Bc4 Ke1 Nf3+ Kf2 Kf4 Kg2 Be6 Kh1 Ne1 Kg1
                              Ke3 Kh2 Kf2 Kh1 Bh3
40    +75.00   03:47    1327M Kd2 Kf8 Kc3 Kf7 Kd4 Kg7 Ke5 Kg6 Nc1 Kg5 Bd3 Kg4
                              Bf5+ Kf3 Nb3 Kg3 Nd4 Kh4 Be4 Kg5 Nc6 Kg4 Bd5 Kg3
                              Nd4 Kf2 Bc4 Ke1 Nf3+ Kf2 Kf4 Kg2 Be6 Kh1 Ne1 Kg1
                              Ke3 Kh2 Kf2 Kh1 Bf5 Kh2 Nf3+ Kh1
41    +75.00   04:03    1411M Kd2 Kf8 Kc3 Kf7 Kd4 Kg7 Ke5 Kg6 Nc1 Kg5 Bd3 Kg4
                              Bf5+ Kf3 Nb3 Kg3 Nd4 Kh4 Be4 Kg5 Nc6 Kg4 Bd5 Kg3
                              Nd4 Kf2 Bc4 Ke1 Nf3+ Kf2 Kf4 Kg2 Be6 Kh1 Ne1 Kg1
                              Ke3 Kh2 Kf2 Kh1 Bf5 Kh2 Nf3+ Kh1 Bd3
42    +75.00   04:27    1543M Kd2 Kf8 Kc3 Kf7 Kd4 Kg7 Ke5 Kg6 Nc1 Kg5 Bd3 Kg4
                              Bf5+ Kf3 Nb3 Kg3 Nd4 Kh4 Be4 Kg5 Nc6 Kg4 Bd5 Kg3
                              Nd4 Kf2 Bc4 Ke1 Nf3+ Kf2 Kf4 Kg2 Be6 Kh1 Ne1 Kg1
                              Ke3 Kh2 Kf2 Kh1 Bf5 Kh2 Nf3+ Kh1 Nd2 Kh2 Nf1+ Kh1
43    +75.00   13:50    5017M Kd2 Kf8 Kc3 Kf7 Kd4 Kg7 Ke5 Kg6 Nc1 Kg7 Kf5 Kh6
                              Kf6 Kh7 Nd3 Kh6 Bh3 Kh7 Kg5 Kg7 Nb2 Kf7 Bf1 Ke7
                              Nd1 Kd6 Kf4 Kc6 Ke5 Kc5 Bh3 Kb6 Bf5 Kb7 Kd5 Ka6
                              Kc6 Ka7 Ne3 Kb8 Kb6 Ka8 Be6
44    +75.00   14:48    5345M Kd2 Kf8 Kc3 Kf7 Kd4 Kg7 Ke5 Kg6 Nc1 Kg7 Kf5 Kh6
                              Kf6 Kh7 Nd3 Kh6 Bh3 Kh7 Kg5 Kg7 Nb2 Kf7 Bf1 Ke7
                              Nd1 Kd6 Bd3 Ke5 Bc4 Kd4 Bf1 Ke4 Nc3+ Ke3 Kg4 Kf2
                              Ba6 Kg2 Ne4 Kh2 Bf1 Kg1 Bh3 Kh1 Kg3
45    +75.00   15:29    5582M Kd2 Kf8 Kc3 Kf7 Kd4 Kg7 Ke5 Kg6 Nc1 Kg7 Kf5 Kh6
                              Kf6 Kh7 Nd3 Kh6 Bh3 Kh7 Kg5 Kg7 Nb2 Kf7 Bf1 Ke7
                              Nd1 Kd6 Bd3 Ke5 Bc4 Kd4 Bf1 Ke4 Nc3+ Ke3 Kg4 Kf2
                              Be2 Kg2 Kh4 Kh2 Bc4 Kg2 Nb5 Kg1 Kg3 Kh1
46    +75.00   19:03    7003M Kd2 Kf8 Kc3 Kf7 Kd4 Kf6 Nf4 Kf5 Nd5 Kg6 Ke3 Kh5
                              Kf4 Kg6 Bd3+ Kg7 Kf5 Kf7 Bc4 Kg7 Kg5 Kh7 Kf6 Kg8
                              Be2 Kh7 Bf1 Kg8 Nc3 Kh7 Kg5 Kg7 Kf4 Kh6 Bd3 Kg7
                              Ke3 Kf6 Kf3 Kg5 Nd1 Kh5 Ne3 Kh4 Ng2+ Kh3 Bf5+ Kh2
                              Kf2 Kh1
47    +75.00   20:22    7441M Kd2 Kf8 Kc3 Kf7 Kd4 Kf6 Nf4 Kf5 Nd5 Kg6 Ke3 Kh5
                              Kf4 Kg6 Bd3+ Kg7 Kf5 Kf7 Bc4 Kg7 Kg5 Kh7 Kf6 Kg8
                              Be2 Kh7 Bf1 Kg8 Bh3 Kh7 Kf7 Kh6 Bf5 Kh5 Kf6 Kh4
                              Be6 Kg3 Kg5 Kf3 Bg4+ Kg2 Kh4 Kh1 Be2 Kg1 Kg3 Kh1
                              Nf4
48    +75.00   33:17   12538M Kd2 Ke7 Kd3 Ke6 Ng3 Kf7 Kc3 Kg6 Bd3+ Kf6 Nh1 Kg5
                              Nf2 Kh6 Kd4 Kg7 Ke3 Kf6 Ng4+ Kf7 Ke2 Ke6 Kf2 Kf7
                              Bc4+ Ke7 Ke2 Kd6 Ke3 Ke7 Bb3 Kd7 Ne5+ Kd6 Nf7+
                              Kc7 Kf4 Kc6 Ne5+ Kc7 Nd3 Kb6 Ke5 Ka5 Kd6 Kb5 Kc7
                              Ka6 Be6 Ka7 Bc8 Ka8 Ne5 Ka7 Nc6+ Ka8
49       #29   35:07   13238M Kd2 Ke7 Kd3 Ke6 Kd4 Kf6 Ke4 Ke6 Nf4+ Ke7 Bd3 Kf6
                              Bc4 Kg7 Ke5 Kh7 Kf6 Kh8 Ng6+ Kh7 Bd5 Kh6 Bg8 Kh5
                              Ne5 Kh4 Kf5 Kg3 Ng4 Kg2 Bc4 Kf3 Bb5 Kg3 Bc6 Kh4
                              Kf4 Kh5 Be8+ Kh4 Ne3 Kh3 Bb5 Kh4 Nf5+ Kh3 Bf1+
                              Kh2 Kf3 Kg1 Ne3 Kh1 Kf2 Kh2 Ng4+ Kh1 Bg2#
50       #29   35:09   13253M Kd2 Ke7 Kd3 Ke6 Kd4 Kf6 Ke4 Ke6 Nf4+ Ke7 Bd3 Kf6
                              Bc4 Kg7 Ke5 Kh7 Kf6 Kh8 Ng6+ Kh7 Bd5 Kh6 Bg8 Kh5
                              Ne5 Kh4 Kf5 Kg3 Ng4 Kg2 Bc4 Kf3 Bb5 Kg3 Bc6 Kh4
                              Kf4 Kh5 Be8+ Kh4 Ne3 Kh3 Bb5 Kh4 Nf5+ Kh3 Bf1+
                              Kh2 Kf3 Kg1 Ne3 Kh1 Kf2 Kh2 Ng4+ Kh1 Bg2#
51       #29   35:14   13278M Kd2 Ke7 Kd3 Ke6 Kd4 Kf6 Ke4 Ke6 Nf4+ Ke7 Bd3 Kf6
                              Bc4 Kg7 Ke5 Kh7 Kf6 Kh8 Ng6+ Kh7 Bd5 Kh6 Bg8 Kh5
                              Ne5 Kh4 Kf5 Kg3 Ng4 Kg2 Bc4 Kf3 Bb5 Kg3 Bc6 Kh4
                              Kf4 Kh5 Be8+ Kh4 Ne3 Kh3 Bb5 Kh4 Nf5+ Kh3 Bf1+
                              Kh2 Kf3 Kg1 Ne3 Kh1 Kf2 Kh2 Ng4+ Kh1 Bg2#
52       #28   35:25   13337M Kd2 Ke7 Kd3 Ke6 Ke4 Kd6 Bh3 Ke7 Nf4 Kf7 Ke5 Kg7
                              Be6 Kh7 Kf6 Kh6 Bf7 Kh7 Ng6 Kh6 Bg8 Kh5 Ne5 Kh4
                              Kf5 Kg3 Ng4 Kg2 Bc4 Kf3 Bb5 Kg3 Bc6 Kh4 Kf4 Kh5
                              Be8+ Kh4 Ne3 Kh3 Bb5 Kh4 Nf5+ Kh3 Bf1+ Kh2 Kf3
                              Kg1 Ne3 Kh1 Kf2 Kh2 Ng4+ Kh1 Bg2#
53       #28   35:32   13376M Kd2 Ke7 Kd3 Ke6 Ke4 Kd6 Bh3 Ke7 Nf4 Kf7 Ke5 Kg7
                              Be6 Kh7 Kf6 Kh6 Bf7 Kh7 Ng6 Kh6 Bg8 Kh5 Ne5 Kh4
                              Kf5 Kg3 Ng4 Kg2 Bc4 Kf3 Bb5 Kg3 Bc6 Kh4 Kf4 Kh5
                              Be8+ Kh4 Ne3 Kh3 Bb5 Kh4 Nf5+ Kh3 Bf1+ Kh2 Kf3
                              Kg1 Ne3 Kh1 Kf2 Kh2 Ng4+ Kh1 Bg2#
Tord
User avatar
hgm
Posts: 28391
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Glaurung 2.2 - no EGTB usage?

Post by hgm »

Generating KBNK takes about 2 seconds on a slow laptop, so why bother pre-computing it?
Tord Romstad
Posts: 1808
Joined: Wed Mar 08, 2006 9:19 pm
Location: Oslo, Norway

Re: Glaurung 2.2 - no EGTB usage?

Post by Tord Romstad »

hgm wrote:Generating KBNK takes about 2 seconds on a slow laptop, so why bother pre-computing it?
I am not quite sure what point you are trying to make, as I don't pre-compute KBNK, but anyway:
  • Even two seconds can be a catastrophic amount of time to waste at a critical moment in a blitz game.
  • My phone is an order of magnitude slower than even a slow laptop, and has nowhere near as much RAM.
  • My program is able to mate with KBN vs K easily with just a few plies of search and a piece square table eval, so why bother with generating a tablebase for the entire endgame when it occurs?
Tord
Daniel Shawul
Posts: 4186
Joined: Tue Mar 14, 2006 11:34 am
Location: Ethiopia

Re: Glaurung 2.2 - no EGTB usage?

Post by Daniel Shawul »

For me, a draw score is not worth much without knowing why the position is drawn. "I looked this position up in a multi-gigabyte file, and it says it's a draw" isn't a very useful explanation. I haven't looked at Timman's study, but because it's composed by a human, I suppose it is possible for a human player without the help of a computer to find the solution. If my program fails to solve such a problem, and I understood the solution, I would try to identify what knowledge my program is missing and add it to my search or evaluation function. This is harder work than simply adding tablebases, but also much more valuable: I may end up dicsovering principles and evaluation rules which can be applied to a wider range of positions with a bigger number of pieces, and I may improve my own chess knowledge at the same time.
For the engine which is about to lose the endgame,knowing about draw score is as good as a win.
Endgame studies are focused on deriving rules that are useful for _humans_ to improve their endgame
play. If you insist on knowing how the WDL is arrived , you should use DTM tablebases.

You can't program most of what is found by some KD technique from tabelbases to benefit the engine,
especially _not so in the evaluation function_, as you seem to suggest. The eval already has a good enough
predictor for the final result, so to improve it you need to do more than weights or whatever.
Most of the moves involve maneuvers (tactical) that just can't be programmed there.
Ok may be you can for some endgames with 5 or less pieces, but it is impossible for most 5 piece and
above endgames without using some sort of search. Most of what is stored in bitbases are _exceptions_, the
rest are really compressed well. Those difficult positions are for humans to discover and enjoy themselves
with. My point being, the tbs are large because they have to. That ofcourse depends on what kind of information
you want from it.
I may end up discovering principles and evaluation rules which can be applied to a wider range of positions with a bigger number of pieces,
and I may improve my own chess knowledge at the same time.
Yes you may, but not something that would be useful for your engine.
For instance how do you program the KBBKN mate in 78 to your engine
so that it would benefit from it. Even for the much simpler KBNK we have to use some table along with _search_ to drive the king to corners. Even with that my engine used to fail mating kbnk! Most of these involve manuevers which are caught with search , and if you have to do search you loose the whole point of having bitbases.
I think just blindly using tablebases as giant lookup tables during the search is quite ugly and stuipd, and likely to be counter-productive in the long run. The right way to use them is probably as a testbed for high-level evaluation routines, or even for generating new high-level endgame heuristics programmatically. Using genetic programming to develop endgame evaluators using tablebase positions as a test suite would be a very interesting project, for instance.
GAs use weighing factors to give you an approximate eval. So again they will have difficulties to catch tactical issues. Most of what is found by a GA is stored compressed really well. I tried Knowledge discovery to derive rules for endgame to no avail. I then added 1 or 2 ply search on top of it, to get better improvements on 4 men, but it barely improved the 5 men size. The knighdreamer guys have also got the same results. IMHO, trying to predict the score without doing search is just worthless.

About the significance of bitbases, I believe they are important. A 200MB 5 men bitbase on RAM surely brings some improvement immeasurable or not.