Queen and pawn endgame

[Rodolfo Leoni]

hash contains no knowledge, it just saves time by avoiding repeated calculations, so nothing very promising to expect from that.

you don't need Monte Carlo to randomise games, just running in busy environment or using multiple cores.

It's not matter of knowledge. As ordinary hashes, an hash key is made by a certain amount of bytes for position, score, depth and best move. When on search, engine saves valuable entries on the PH file, depending on search depth. It's subjected to some rules, as shallower depth score cannot overwrite a deeper one, or root score is more valuable than non-root one.

The fact of having faster search depends on forward propagation (as in Critter). The min-maxing process for real "learning" is backward propagation. In Critter, the option exists (resolve score drops); in SF PA GTB id doesn't, so it depends on hash hits and it's not always working.

Critter has another advantage, though. It forward-propagates with multipv too.

as far as I know, hash key is the description of the position, and is part of the same structure, where other members of the structure as score, depth, bound, etc., would belong.

The hash key structure changed since the times of SF PA GTB. Each entry is 16 bytes size now. I think 8 for position (zorbrist-compressed FEN), 2 for score, 2 for depth and 4 for best move, if I remember correctly. Three years ago they were 32 bytes total but I don't know the "distribution" of bytes. In any case there were all data of an EPD and it allowed to build a routine with the console command "importepd <filename.epd>" for storing additional data into PH file.

[MikeB]

latest dev-SF-McB

first using just 2 cores and 2048 MB hash

[d]8/pp3p1k/6pp/7P/P5P1/3K4/1PP5/8 b - a3 0 1

Code: Select all

dep	score	nodes	time	(not shown:  tbhits	knps	seldep)
 39	+17.31 	3.63G	9:57.43	Kg7 a5 Kf6 Ke4 Kg5 c4 Kf6 Kd4 Ke6 a6 b6 Ke4 Kf6 Kd5 Ke7 Ke5 f6+ Kf4 g5+ Kf5 Kd6 b4 Kc6 b5+ Kc5 Kxf6 Kd4 Kg7 Ke5 Kxh6 Kf4 Kg6 Kxg4 h6 Kf4 h7 g4 h8=Q g3 Qb8+ Kg4 Qxa7 g2 Qxb6 
 39	+13.27?	3.17G	8:36.80	f5 a5? 
 39	+10.01?	2.16G	5:50.47	f5 a5? 
 39	+7.42?	1.76G	4:50.54	f5 a5? 
 39	+5.36?	1.57G	4:22.06	f5 a5? 
 39	+3.72?	1.45G	4:03.24	f5 a5? 
 39	+2.43?	1.34G	3:45.00	f5 a5? 
 39	+1.42?	1.18G	3:20.28	f5 a5? 
 39	+0.62?	1.10G	3:05.60	f5 a5? 
 39	  0.00?	859.5M	2:21.28	f5 a5? 
 39	 -0.48?	762.1M	2:05.01	f5 a5? 
 39	 -0.85?	660.2M	1:47.42	f5 a5? 
 39	 -1.13?	621.3M	1:41.69	f5 a5? 
 39	 -1.34?	505.1M	1:21.01	f5 gxf5? 
 39	 -1.50?	480.3M	1:16.86	f5 gxf5? 
 39	 -1.60?	456.6M	1:13.32	f5 gxf5? 
 39	 -1.68?	426.5M	1:08.55	f5 gxf5? 
 38	 -1.75?	374.7M	1:00.06	f5 gxf5? 
 38	 -1.86?	344.2M	0:55.28	f5 gxf5? 
 38	 -1.93?	316.4M	0:51.17	f5 gxf5? 
 37	 -2.00 	305.0M	0:49.40	f5 gxf5 gxh5 Ke4 Kg7 a5 Kf6 a6 bxa6 Kf4 a5 c4 h4 c5 h3 c6 h2 c7 h1=Q c8=Q Qf1+ Kg3 Qd3+ Kh2 Qe2+ Kg1 Qd1+ Kh2 Qd2+ Kg1 Qe3+ Kh2 Qf2+ Kh1 Qf3+ Kh2 Qf4+ Kg1 Qxf5 Qd8+ Ke6 Qc8+ Ke5 Qc3+ Kf4 Qc1+ Kg3 Qc3+ Kh4 Kh2 Qf4+ Kg2 Qe4+ Kf2 
 36	 -2.00!	259.6M	0:41.62	f5! 
 

here using 18 threads on a 12 core Mac Pro (2010) with 4096 MB hash

Code: Select all

dep	score	nodes	time	(not shown:  tbhits	knps	seldep)
 43	+128.35 	18.0G	7:21.20	Kg7 a5 Kf6 Ke4 Ke6 c4 Kf6 Kd5 Ke7 b4 g5 Ke5 f6+ Kf5 Kd7 Kxf6 Kd6 b5 Kc5 Kg6 Kb4 Kxh6 Kc5 Kxg5 Kd6 c5+ Kxc5 h6 Kd6 h7 Ke6 h8=Q Kd7 Qg7+ Ke6 Qxb7 Kd6 Qxa7 
 42	+128.35 	17.7G	7:12.78	Kg7 a5 Kf6 Ke4 Ke6 c4 Kf6 Kd5 Ke7 b4 g5 Ke5 f6+ Kf5 Kd7 Kxf6 Kd6 b5 Kc5 Kg6 Kb4 Kxh6 Kc5 Kxg5 Kd6 c5+ Kxc5 h6 Kd6 h7 Ke6 h8=Q Kd7 Qg7+ Ke6 Qxb7 Kd6 Qxa7 
 41	+128.35 	17.3G	7:01.79	Kg7 a5 Kf6 Ke4 Ke6 c4 Kf6 Kd5 Ke7 b4 g5 Ke5 f6+ Kf5 Kd7 Kxf6 Kd6 b5 Kc5 Kg6 Kb4 Kxh6 Kc5 Kxg5 Kd6 c5+ Kxc5 h6 Kd6 h7 Ke6 h8=Q Kd7 Qg7+ Ke6 Qxb7 Kd6 Qxa7 
 40	+128.35 	17.2G	7:00.34	Kg7 a5 Kf6 Ke4 Ke6 c4 Kf6 Kd5 Ke7 b4 g5 Ke5 f6+ Kf5 Kd7 Kxf6 Kd6 b5 Kc5 Kg6 Kb4 Kxh6 Kc5 Kxg5 Kd6 c5+ Kxc5 h6 Kd6 h7 Ke6 h8=Q Kd7 Qg7+ Ke6 Qxb7 Kd6 Qxa7 
 39	+128.35 	16.8G	6:48.53	Kg7 a5 Kf6 Ke4 Ke6 c4 Kf6 Kd5 Ke7 b4 g5 Ke5 f6+ Kf5 Kd7 b5 Kd6 Kxf6 Kc5 Kg6 Kb4 Kxh6 Kc5 Kxg5 Kd6 c5+ Kd7 c6+ bxc6 bxc6+ Kd8 c7+ Kc8 a6 Kd7 c8=N Kc7 Nxa7 
 38	+128.35 	16.3G	6:36.10	Kg7 a5 Kf6 Ke4 Ke6 c4 Kf6 Kd5 Ke7 b4 g5 Ke5 f6+ Kf5 Kd7 b5 Kd6 Kxf6 Kc5 Kg6 Kb4 Kxh6 Kc5 Kxg5 Kd6 c5+ Kd7 c6+ bxc6 bxc6+ Kc7 a6 Kd8 c7+ Kc8 h6 Kd7 c8=Q+ Kxc8 
 37	+128.34 	15.9G	6:26.65	Kg7 a5 Kf6 Ke4 Ke7 c4 Ke6 b4 g5 c5 a6 b5 axb5 c6 Kd6 cxb7 Kc7 a6 b4 Kd3 b3 Kc3 Kb8 Kxb3 Kc7 b8=Q+ Kxb8 Kc4 Kc8 Kd4 Kc7 Ke5 Kd7 Kf6 Ke8 a7 Kd7 Kxf7 Kc7 Kg6 Kb6 
 37	+128.26?	13.3G	5:18.65	Kg7 a5? 
 37	+128.18?	13.3G	5:17.30	Kg7 a5? 
 36	+128.11 	13.2G	5:16.61	Kg7 a5 Kf6 Ke4 Ke6 c4 Kd6 b4 Ke7 Ke5 g5 Kf5 f6 b5 b6 a6 Kd6 Kxf6 Kd7 Kg7 Kc7 Kg6 Kd7 Kxh6 Kd6 Kg6 Kc5 Kf5 Kd4 h6 Kc5 Kg6 Kb4 Kf6 Kc5 Kf5 Kb4 Kxg5 
 36	+120.09?	9.80G	3:54.67	f5 a5? 
 36	+95.32?	9.73G	3:52.82	f5 a5? 
 36	+75.52?	9.65G	3:51.00	f5 a5? 
 36	+59.69?	8.57G	3:26.06	f5 a5? 
 36	+47.05?	7.62G	3:03.84	f5 a5? 
 36	+36.95?	7.49G	3:00.48	f5 a5? 
 36	+28.88?	7.21G	2:52.97	f5 a5? 
 36	+22.45?	6.88G	2:44.24	f5 a5? 
 36	+17.31?	5.49G	2:09.12	f5 a5? 
 36	+13.22?	4.98G	1:57.03	f5 a5? 
 36	+9.96?	3.61G	1:22.98	f5 a5? 
 36	+7.37?	2.86G	1:07.20	f5 a5? 
 36	+5.31?	2.43G	0:58.38	f5 a5? 
 36	+3.68?	2.13G	0:51.78	f5 a5? 
 36	+2.39?	1.86G	0:45.62	f5 a5? 
 36	+1.37?	1.65G	0:40.83	f5 a5? 
 36	+0.57?	1.48G	0:36.69	f5 a5? 
 36	 -0.04?	1.26G	0:31.13	f5 a5? 
 36	 -0.53?	1.12G	0:27.64	f5 a5? 
 36	 -0.90?	948.0M	0:23.67	f5 a5? 
 36	 -1.18?	899.5M	0:22.57	f5 a5? 
 36	 -1.39?	863.8M	0:21.74	f5 a5? 
 36	 -1.54?	841.0M	0:21.23	f5 a5? 
 36	 -1.65?	759.9M	0:19.34	f5 a5? 
 36	 -1.72?	729.7M	0:18.52	f5 a5? 
 35	 -1.80!	586.8M	0:14.79	f5! 
 35	 -1.69!	539.0M	0:13.59	f5! 
 35	 -1.62!	538.1M	0:13.56	f5! 
 34	 -1.55 	504.8M	0:12.62	f5 gxf5 gxh5 Ke4 Kg7 a5 Kf6 a6 bxa6 Kf4 a5 c4 h4 c5 h3 c6 h2 c7 h1=Q c8=Q Qf1+ Kg3 Qd3+ Kg2 Qe4+ Kh2 Qe5+ Kh1 Qd5+ Kh2 Qd2+ Kh1 Qe1+ Kg2 Qe2+ Kh1 Qf3+ Kh2 Qf2+ Kh1 Qh4+ Kg2 Qg5+ Kh2 Qe3 b3 Kg5 Qf8 Qe5+ Kg2 Qe2+ Kg1 Qg4+ Kh2  

[Lyudmil Tsvetkov]

hash contains no knowledge, it just saves time by avoiding repeated calculations, so nothing very promising to expect from that.

you don't need Monte Carlo to randomise games, just running in busy environment or using multiple cores.

It's not matter of knowledge. As ordinary hashes, an hash key is made by a certain amount of bytes for position, score, depth and best move. When on search, engine saves valuable entries on the PH file, depending on search depth. It's subjected to some rules, as shallower depth score cannot overwrite a deeper one, or root score is more valuable than non-root one.

The fact of having faster search depends on forward propagation (as in Critter). The min-maxing process for real "learning" is backward propagation. In Critter, the option exists (resolve score drops); in SF PA GTB id doesn't, so it depends on hash hits and it's not always working.

Critter has another advantage, though. It forward-propagates with multipv too.

as far as I know, hash key is the description of the position, and is part of the same structure, where other members of the structure as score, depth, bound, etc., would belong.

The hash key structure changed since the times of SF PA GTB. Each entry is 16 bytes size now. I think 8 for position (zorbrist-compressed FEN), 2 for score, 2 for depth and 4 for best move, if I remember correctly. Three years ago they were 32 bytes total but I don't know the "distribution" of bytes. In any case there were all data of an EPD and it allowed to build a routine with the console command "importepd <filename.epd>" for storing additional data into PH file.

well, storing the score is certainly the most important element, but that should not help a lot with incorrect eval, as most engines have, as in this case the score itself, lower/upper bounds, etc. will be wrong, misleading the engine in its search decisions.

so, I guess, the only thing that would certainly hold up in that hash structure would be the reached depth.

my point is not to over-estimate the boost to strength persistent hash can give.

[Rodolfo Leoni]

well, storing the score is certainly the most important element, but that should not help a lot with incorrect eval, as most engines have, as in this case the score itself, lower/upper bounds, etc. will be wrong, misleading the engine in its search decisions.

so, I guess, the only thing that would certainly hold up in that hash structure would be the reached depth.

my point is not to over-estimate the boost to strength persistent hash can give.

Score and depth are strictly related. Correct scoring is engine strenght related as well, and that's why the stronger the engine, the better the learning.

I don't think somebody could think to make an engine stronger with PHs, as tree enlarges very much since opening stages. If you run millions (or, possibly, zillions) games then you could have a stronger engine. It can take several lifetimes......

Engine with PHs is useful to players only in infinite mode; player makes moves, takes them back, tries variations, and so on. The old style analysis. The fact score can propagate helps player to keep track of good and bad variations. As a tool, it's user-oriented rather than engine-oriented. Then, sometimes one wants engine to give some hints about a position but it's sleep time (as it was for the experiment I made). Automatic playing is far less effective but it can give some clues too.

[Rodolfo Leoni]

BTW, I enjoy very much this conversation... It's a +++ on your credibility, and you're showing a better side of yourself. I encourage you to continue this way and... Thanks!