How Long to generate 5 men Tablebases

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

mhalstern
Posts: 484
Joined: Wed Nov 18, 2009 1:09 am

How Long to generate 5 men Tablebases

Post by mhalstern »

I will shortly have a pc built around an i7-920 with 8 gig of ram.

How long will it take to:

1 - Generate Tablebases with tbgen?
2 - Generate Gaviotta's Tablebases with the Built in Tablebase generator?

Thanks
User avatar
michiguel
Posts: 6401
Joined: Thu Mar 09, 2006 8:30 pm
Location: Chicago, Illinois, USA

Re: How Long to generate 5 men Tablebases

Post by michiguel »

mhalstern wrote:I will shortly have a pc built around an i7-920 with 8 gig of ram.

How long will it take to:

1 - Generate Tablebases with tbgen?
2 - Generate Gaviotta's Tablebases with the Built in Tablebase generator?

Thanks
I think it depends on several factors, but if you dedicate all cores (4) for the task, I estimate that it could be more than 24 hrs but certainly less than 48 hrs. Honestly, I do not the answer with accuracy. In fact, it would be great if you test it. I think it took me less than 3 days in a dual 2.4 GHz with 2 Gb RAM when I first generated them. But I did it in parts and validating the files in between, so I did not record the time. You can always start, and if you want to quit, press Control C and stop it. You can start later and the generation will resume from where it stopped (you only lose the info about the particular file that was being generated and not saved).

Maybe I should implement an elegant way to pause it. Anyway, there is a huge room for improvement in the generation speed and that is something I will have to address before attacking the 6-pc problem.

Miguel
mhalstern
Posts: 484
Joined: Wed Nov 18, 2009 1:09 am

Re: How Long to generate 5 men Tablebases

Post by mhalstern »

michiguel wrote:
mhalstern wrote:I will shortly have a pc built around an i7-920 with 8 gig of ram.

How long will it take to:

1 - Generate Tablebases with tbgen?
2 - Generate Gaviotta's Tablebases with the Built in Tablebase generator?

Thanks
I think it depends on several factors, but if you dedicate all cores (4) for the task, I estimate that it could be more than 24 hrs but certainly less than 48 hrs. Honestly, I do not the answer with accuracy. In fact, it would be great if you test it. I think it took me less than 3 days in a dual 2.4 GHz with 2 Gb RAM when I first generated them. But I did it in parts and validating the files in between, so I did not record the time. You can always start, and if you want to quit, press Control C and stop it. You can start later and the generation will resume from where it stopped (you only lose the info about the particular file that was being generated and not saved).

Maybe I should implement an elegant way to pause it. Anyway, there is a huge room for improvement in the generation speed and that is something I will have to address before attacking the 6-pc problem.

Miguel
I'd love to test the speed. I'll post the results and detailed specs about my hardware.

I've never used Gaviotta. Will the built in tablebase generator dedicate all 4 cores by default, or do I need to specify a setting?

In addition, for Chess Engine performance, I will have hyperthreading disabled. If I enable hyperthreading and have another 4 "virtual" cores, will the generation be faster?

Thanks
User avatar
michiguel
Posts: 6401
Joined: Thu Mar 09, 2006 8:30 pm
Location: Chicago, Illinois, USA

Re: How Long to generate 5 men Tablebases

Post by michiguel »

mhalstern wrote:
michiguel wrote:
mhalstern wrote:I will shortly have a pc built around an i7-920 with 8 gig of ram.

How long will it take to:

1 - Generate Tablebases with tbgen?
2 - Generate Gaviotta's Tablebases with the Built in Tablebase generator?

Thanks
I think it depends on several factors, but if you dedicate all cores (4) for the task, I estimate that it could be more than 24 hrs but certainly less than 48 hrs. Honestly, I do not the answer with accuracy. In fact, it would be great if you test it. I think it took me less than 3 days in a dual 2.4 GHz with 2 Gb RAM when I first generated them. But I did it in parts and validating the files in between, so I did not record the time. You can always start, and if you want to quit, press Control C and stop it. You can start later and the generation will resume from where it stopped (you only lose the info about the particular file that was being generated and not saved).

Maybe I should implement an elegant way to pause it. Anyway, there is a huge room for improvement in the generation speed and that is something I will have to address before attacking the 6-pc problem.

Miguel
I'd love to test the speed. I'll post the results and detailed specs about my hardware.

I've never used Gaviotta. Will the built in tablebase generator dedicate all 4 cores by default, or do I need to specify a setting?
Just type "cores 4" before you do "tbgen 5". You can get more details here:
http://sites.google.com/site/gaviotache ... blebases-1

In addition, for Chess Engine performance, I will have hyperthreading disabled. If I enable hyperthreading and have another 4 "virtual" cores, will the generation be faster?

Thanks
That is a great question! I do not know... maybe you can try it with tbgen 4 to get a quick answer. With hyperthreading on, you may try cores 8 vs cores 4 an see what happens! There is a good chance it will help a little since this is not like a chess engine search.

Miguel
User avatar
jshriver
Posts: 1342
Joined: Wed Mar 08, 2006 9:41 pm
Location: Morgantown, WV, USA

Re: How Long to generate 5 men Tablebases

Post by jshriver »

I suspect it would be a lot quicker for you as that's a nice machine :) I've been running it 24/7. this friday will be 2 weeks and looks like it'll def be done by then.

The machine I'm generating them on is a lowly 1.3ghz Atom CPU with 1gig of ram. Not a cruncher machine, but I like that it's low power so I can let it sit in a corner and offload from my main system.

Once a compression scheme is set I'll upload what I have to my website for people to grab. Looks like the uncompressed 3-4-5man set is around 35gigs. Which I could technically host but I'm starting to hit my limits now and would rather wait.

-Josh
User avatar
jshriver
Posts: 1342
Joined: Wed Mar 08, 2006 9:41 pm
Location: Morgantown, WV, USA

Re: How Long to generate 5 men Tablebases

Post by jshriver »

Wanted to also note the 3-4 took nearly no time at all (hours?) the most time has been spent on the kppkx set it seems.

-Josh
User avatar
beachknight
Posts: 3533
Joined: Tue Jan 09, 2007 8:33 pm
Location: Antalya, Turkey

Re: How Long to generate 5 men Tablebases

Post by beachknight »

jshriver wrote:Wanted to also note the 3-4 took nearly no time at all (hours?) the most time has been spent on the kppkx set it seems.

-Josh
Hi Josh,

I'd like to use 3-4 B tbs. How may I create them in my computer?

Best,

PS: Or a download link would be better.
hi, merhaba, hallo HT
User avatar
jshriver
Posts: 1342
Joined: Wed Mar 08, 2006 9:41 pm
Location: Morgantown, WV, USA

Re: How Long to generate 5 men Tablebases

Post by jshriver »

Check out the authors website:

http://sites.google.com/site/gaviotachessengine/

Grab the latest version of his engine, and run it.
Once it's running type this command in:

tbgen 4

Let it burn for a couple hours and it'll create everything for you. The resulting files will be in a folder called gtb.
User avatar
beachknight
Posts: 3533
Joined: Tue Jan 09, 2007 8:33 pm
Location: Antalya, Turkey

Re: How Long to generate 5 men Tablebases

Post by beachknight »

jshriver wrote:Check out the authors website:

http://sites.google.com/site/gaviotachessengine/

Grab the latest version of his engine, .
I did this part :)
and run it.
Once it's running type this command in:

tbgen 4

Let it burn for a couple hours and it'll create everything for you. The resulting files will be in a folder called gtb.
Thanks. I'll do that overnight or tomorrow morning.

Best,
hi, merhaba, hallo HT
User avatar
beachknight
Posts: 3533
Joined: Tue Jan 09, 2007 8:33 pm
Location: Antalya, Turkey

Re: How Long to generate 5 men Tablebases

Post by beachknight »

beachknight wrote:
jshriver wrote:Check out the authors website:

http://sites.google.com/site/gaviotachessengine/

Grab the latest version of his engine, .
I did this part :)
and run it.
Once it's running type this command in:

tbgen 4

Let it burn for a couple hours and it'll create everything for you. The resulting files will be in a folder called gtb.
Thanks. I'll do that overnight or tomorrow morning.

Best,
Just done.

In total, there are now 35 files in gtb folder, 174 MB in size.

Should check with tbcheck.exe, but how?

Best,

PS: Creating 4 B tbs took 15 minutes on my computer.
How many hours would it take for 5 B tbs?
hi, merhaba, hallo HT