DTM50

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

User avatar
Nordlandia
Posts: 2821
Joined: Fri Sep 25, 2015 9:38 pm
Location: Sortland, Norway

Re: DTM50

Post by Nordlandia »

6-men nalimov on samsung 970 evo plus 2tb. For server analysis (Let's Check in Fritz or Chessbase).

Read 3,000 MB/s / Write 2,800 MB.

Probe speed of 3-4 is fine based on my testing so far (Houdini 6.03)

I clearly notice some EGTB improvements of 970 evo plus 2tb compared to 960 evo 256 GB version. Better syzygy enhancements.
Joost Buijs
Posts: 1563
Joined: Thu Jul 16, 2009 10:47 am
Location: Almere, The Netherlands

Re: DTM50

Post by Joost Buijs »

I use Nalimov 6-men on the old Samsung 970 EVO 2TB, with my own engine it is fast enough to probe at depth 1 without causing too much slowdown.

A few years back I generated 6-men syzygy WDL and DTZ which took 5 days on my old 5960x, however I never came to use it. I still want to try to combine syzygy WDL with Nalimov DTM, maybe syzygy WDL probes fast enough to use it in quiescence, this is probably what Houdini does too.
User avatar
Nordlandia
Posts: 2821
Joined: Fri Sep 25, 2015 9:38 pm
Location: Sortland, Norway

Re: DTM50

Post by Nordlandia »

Joost Buijs wrote: Tue Jul 02, 2019 7:06 am I use Nalimov 6-men on the old Samsung 970 EVO 2TB, with my own engine it is fast enough to probe at depth 1 without causing too much slowdown.

A few years back I generated 6-men syzygy WDL and DTZ which took 5 days on my old 5960x, however I never came to use it. I still want to try to combine syzygy WDL with Nalimov DTM, maybe syzygy WDL probes fast enough to use it in quiescence, this is probably what Houdini does too.
Per Houdini manual:

The only drawback of Syzygy bases is that they do not contain the exact Distance to Mate information. If you require an exact mating distance in a position you can use the Nalimov table bases (only available in the Houdini Pro version). Houdini can work with both Syzygy and Nalimov table bases simultaneously. When the root position is found in the Nalimov bases, the exact mate information will be used and the complete mating PV will be shown.
Joost Buijs
Posts: 1563
Joined: Thu Jul 16, 2009 10:47 am
Location: Almere, The Netherlands

Re: DTM50

Post by Joost Buijs »

Nordlandia wrote: Tue Jul 02, 2019 6:25 pm
Joost Buijs wrote: Tue Jul 02, 2019 7:06 am I use Nalimov 6-men on the old Samsung 970 EVO 2TB, with my own engine it is fast enough to probe at depth 1 without causing too much slowdown.

A few years back I generated 6-men syzygy WDL and DTZ which took 5 days on my old 5960x, however I never came to use it. I still want to try to combine syzygy WDL with Nalimov DTM, maybe syzygy WDL probes fast enough to use it in quiescence, this is probably what Houdini does too.
Per Houdini manual:

The only drawback of Syzygy bases is that they do not contain the exact Distance to Mate information. If you require an exact mating distance in a position you can use the Nalimov table bases (only available in the Houdini Pro version). Houdini can work with both Syzygy and Nalimov table bases simultaneously. When the root position is found in the Nalimov bases, the exact mate information will be used and the complete mating PV will be shown.
It is not very clear from the manual what Houdini exactly does, my guess is that with syzygy Houdini can probe somewhat closer to the leaves because with WDL you read less data and maybe the probing code of syzygy is somewhat better streamlined, also Nalimov does not take distance to conversion into consideration so you can use syzygy to determine if a win (or loss) is cursed or not.

Besides the smaller size I don't see a very big advantage in using syzygy over Nalimov, the only difficulty is that you need permission from both Eugene Nalimov and Andrew Kadatch to incorporate the Nalimov probing code into your engine, this is somewhat difficult to get.
User avatar
MikeB
Posts: 4889
Joined: Thu Mar 09, 2006 6:34 am
Location: Pen Argyl, Pennsylvania

Re: DTM50

Post by MikeB »

Joost Buijs wrote: Tue Jul 02, 2019 11:06 pm
Nordlandia wrote: Tue Jul 02, 2019 6:25 pm
Joost Buijs wrote: Tue Jul 02, 2019 7:06 am I use Nalimov 6-men on the old Samsung 970 EVO 2TB, with my own engine it is fast enough to probe at depth 1 without causing too much slowdown.

A few years back I generated 6-men syzygy WDL and DTZ which took 5 days on my old 5960x, however I never came to use it. I still want to try to combine syzygy WDL with Nalimov DTM, maybe syzygy WDL probes fast enough to use it in quiescence, this is probably what Houdini does too.
Per Houdini manual:

The only drawback of Syzygy bases is that they do not contain the exact Distance to Mate information. If you require an exact mating distance in a position you can use the Nalimov table bases (only available in the Houdini Pro version). Houdini can work with both Syzygy and Nalimov table bases simultaneously. When the root position is found in the Nalimov bases, the exact mate information will be used and the complete mating PV will be shown.
It is not very clear from the manual what Houdini exactly does, my guess is that with syzygy Houdini can probe somewhat closer to the leaves because with WDL you read less data and maybe the probing code of syzygy is somewhat better streamlined, also Nalimov does not take distance to conversion into consideration so you can use syzygy to determine if a win (or loss) is cursed or not.

Besides the smaller size I don't see a very big advantage in using syzygy over Nalimov, the only difficulty is that you need permission from both Eugene Nalimov and Andrew Kadatch to incorporate the Nalimov probing code into your engine, this is somewhat difficult to get.
If you just want the mate score:
http://www.k4it.de/?topic=egtb&lang=en
Image
Joerg Oster
Posts: 937
Joined: Fri Mar 10, 2006 4:29 pm
Location: Germany

Re: DTM50

Post by Joerg Oster »

Joost Buijs wrote: Tue Jul 02, 2019 11:06 pm
Nordlandia wrote: Tue Jul 02, 2019 6:25 pm
Joost Buijs wrote: Tue Jul 02, 2019 7:06 am I use Nalimov 6-men on the old Samsung 970 EVO 2TB, with my own engine it is fast enough to probe at depth 1 without causing too much slowdown.

A few years back I generated 6-men syzygy WDL and DTZ which took 5 days on my old 5960x, however I never came to use it. I still want to try to combine syzygy WDL with Nalimov DTM, maybe syzygy WDL probes fast enough to use it in quiescence, this is probably what Houdini does too.
Per Houdini manual:

The only drawback of Syzygy bases is that they do not contain the exact Distance to Mate information. If you require an exact mating distance in a position you can use the Nalimov table bases (only available in the Houdini Pro version). Houdini can work with both Syzygy and Nalimov table bases simultaneously. When the root position is found in the Nalimov bases, the exact mate information will be used and the complete mating PV will be shown.
It is not very clear from the manual what Houdini exactly does, my guess is that with syzygy Houdini can probe somewhat closer to the leaves because with WDL you read less data and maybe the probing code of syzygy is somewhat better streamlined, also Nalimov does not take distance to conversion into consideration so you can use syzygy to determine if a win (or loss) is cursed or not.

Besides the smaller size I don't see a very big advantage in using syzygy over Nalimov, the only difficulty is that you need permission from both Eugene Nalimov and Andrew Kadatch to incorporate the Nalimov probing code into your engine, this is somewhat difficult to get.
Maybe this post viewtopic.php?f=7&t=60833&p=679321&hili ... es#p679321 is of some interest for you.
Jörg Oster
Joost Buijs
Posts: 1563
Joined: Thu Jul 16, 2009 10:47 am
Location: Almere, The Netherlands

Re: DTM50

Post by Joost Buijs »

Joerg Oster wrote: Wed Jul 03, 2019 10:01 am Maybe this post viewtopic.php?f=7&t=60833&p=679321&hili ... es#p679321 is of some interest for you.
Thanks! Looks interesting.
syzygy
Posts: 5557
Joined: Tue Feb 28, 2012 11:56 pm

Re: DTM50

Post by syzygy »

Joost Buijs wrote: Tue Jul 02, 2019 11:06 pm Besides the smaller size I don't see a very big advantage in using syzygy over Nalimov, the only difficulty is that you need permission from both Eugene Nalimov and Andrew Kadatch to incorporate the Nalimov probing code into your engine, this is somewhat difficult to get.
They are smaller, so more is cached in RAM (via the OS's system/page cache), and they are compressed in small blocks which allows decompression on the fly (so no need to use RAM for caching large decompressed blocks). This makes them much faster on the same hardware, which allows probing much closer to the leaves. They also initialise a lot faster (initialising 6-men Nalimov on startup of the engine seems to have taken about half an hour in the past, probably it is faster on modern hardware but still).

In addition, 7-men syzygy TBs are now available. With SSDs becoming cheaper and cheaper, having 7-men WDL TBs on SSD starts to become a practical possibility (unfortunately RAM prices are lagging behind).
User avatar
Nordlandia
Posts: 2821
Joined: Fri Sep 25, 2015 9:38 pm
Location: Sortland, Norway

Re: DTM50

Post by Nordlandia »

I have ordered 128 GB ram for 795 euros. Corsair Vengeance LPX DDR4 2933MHz 128GB (CMK128GX4M8Z2933C16). So what kind of performance can i expect out of it, considering 128 GB ram costs half if not twice as much.

Loading 6-men nalimov on 970 evo plus 2tb still impose a short delay of about 20 seconds or so. On my sata external backup it was 1-2 minutes +/-
Joost Buijs
Posts: 1563
Joined: Thu Jul 16, 2009 10:47 am
Location: Almere, The Netherlands

Re: DTM50

Post by Joost Buijs »

Nordlandia wrote: Sat Jul 06, 2019 5:17 pm I have ordered 128 GB ram for 795 euros. Corsair Vengeance LPX DDR4 2933MHz 128GB (CMK128GX4M8Z2933C16). So what kind of performance can i expect out of it, considering 128 GB ram costs half if not twice as much.

Loading 6-men nalimov on 970 evo plus 2tb still impose a short delay of about 20 seconds or so. On my sata external backup it was 1-2 minutes +/-
I hope your processor and mainboard support 128 GB RAM.

On my old 970 EVO the initial delay to load 6-men Nalimov is 40 seconds, subsequent tries thereafter are about 10 seconds, Windows 10 seems to cache a lot. The load-time also depends upon how many CPU threads you want to use, with support for 1 thread it loads a lot faster than with support for 20 which I use at default. In the past it was faster, after Spectre mitigations it became slower. My mainboard is getting a bit outdated (Asus X99 Deluxe II), more recent mainboards with X299 chipsets will probably do better.