Hi Ed,
I have a file pgn of 937 Mb with approximately 1.150.000 games.
I have turned this file to a file dat, the process is correct, now I have a database of 289 Mb.
I want to turn this database a EOC, I want to include all the games in the EOC (60 ply) and I have the problem that in phase I it gives an error (Insuficient memory) when it has found 5588243 positions of 5592400 maximum and then "Error during the EOC buiding process". Prodeo also gives an error and finishes when I try to process million.pgn to EOC. Problem of memory?
Prodeo has hash of 48 Mb, I have a computer with 1 Gb de RAM.
Best,
Pedro
EOC databases now as open source
Moderators: hgm, Rebel, chrisw
-
- Posts: 1056
- Joined: Fri Mar 10, 2006 6:07 am
- Location: Basque Country (Spain)
Re: EOC databases now as open source
Hi Pedro,
Simply increase the hash table size to 256/512Mb
Start the program with the Wx paramater for instance: prodeo w6
Here is the list of hash table sizes:
Good luck!
Ed
Simply increase the hash table size to 256/512Mb
Start the program with the Wx paramater for instance: prodeo w6
Here is the list of hash table sizes:
Code: Select all
w0= 32Mb | w1= 48Mb | w2= 64Mb | w3= 96Mb | w4= 128Mb
w5= 192Mb | w6= 256Mb | w7= 384Mb | w8= 512Mb | w9= 768Mb
wa= 1Gb | wb= 1.5Gb| wc= 2Gb | wd= 3Gb
Ed
pedrox wrote:Hi Ed,
I have a file pgn of 937 Mb with approximately 1.150.000 games.
I have turned this file to a file dat, the process is correct, now I have a database of 289 Mb.
I want to turn this database a EOC, I want to include all the games in the EOC (60 ply) and I have the problem that in phase I it gives an error (Insuficient memory) when it has found 5588243 positions of 5592400 maximum and then "Error during the EOC buiding process". Prodeo also gives an error and finishes when I try to process million.pgn to EOC. Problem of memory?
Prodeo has hash of 48 Mb, I have a computer with 1 Gb de RAM.
Best,
Pedro
-
- Posts: 1056
- Joined: Fri Mar 10, 2006 6:07 am
- Location: Basque Country (Spain)
Re: EOC databases now as open source
Perfect.
Thank you, Ed.
Best,
Pedro
Thank you, Ed.
Best,
Pedro
Re: EOC databases now as open source
Similar problem here :ed wrote:Hi Pedro,
Simply increase the hash table size to 256/512Mb
Start the program with the Wx paramater for instance: prodeo w6
Here is the list of hash table sizes:Good luck!Code: Select all
w0= 32Mb | w1= 48Mb | w2= 64Mb | w3= 96Mb | w4= 128Mb w5= 192Mb | w6= 256Mb | w7= 384Mb | w8= 512Mb | w9= 768Mb wa= 1Gb | wb= 1.5Gb| wc= 2Gb | wd= 3Gb
Ed
pedrox wrote:Hi Ed,
I have a file pgn of 937 Mb with approximately 1.150.000 games.
I have turned this file to a file dat, the process is correct, now I have a database of 289 Mb.
I want to turn this database a EOC, I want to include all the games in the EOC (60 ply) and I have the problem that in phase I it gives an error (Insuficient memory) when it has found 5588243 positions of 5592400 maximum and then "Error during the EOC buiding process". Prodeo also gives an error and finishes when I try to process million.pgn to EOC. Problem of memory?
Prodeo has hash of 48 Mb, I have a computer with 1 Gb de RAM.
Best,
Pedro
I have Windows XP and 2 GB of ram. In order to build a huge EOC, I set this hash table to 'prodeo wa' for 1 Gb. It still shows 768 Mb in the params.txt file and in task manager. Is this a bug or am I wrong ?
Re: EOC databases now as open source
I have built a database from 3.900.000 games, its size is 656 Mb. Protools immediatly crashes when I want to create EOC.
-
- Posts: 388
- Joined: Sun Dec 21, 2008 6:57 pm
- Location: Washington, DC
Re: EOC databases now as open source
Hi Ed! Nice to see you stop by I just wanted to take a moment to thank you for posting the web page of yours with the detailed description of many facets of how Rebel works. It gave me a lot of good ideas.
In particular, I am using your approach of doing a full eval at every internal node and making up for the cost (hopefully) by using the amount of useful information you can aquire from it. One thing surprised me though. I liked the idea of using the knowledge of which pieces were hanging to improve move ordering by giving a bonus for moving a hanging piece. Much to my surprise, that bonus always wound up hurting in node-count-do-specific-depth tests no matter how small I made the bonus. I haven't found any move ordering for non-captures non-killers that beats straight piece-square-table ordering.
Anyway, thanks again! And, of course, always glad to hear about any new ideas/discoveries!
Cheers,
Greg
In particular, I am using your approach of doing a full eval at every internal node and making up for the cost (hopefully) by using the amount of useful information you can aquire from it. One thing surprised me though. I liked the idea of using the knowledge of which pieces were hanging to improve move ordering by giving a bonus for moving a hanging piece. Much to my surprise, that bonus always wound up hurting in node-count-do-specific-depth tests no matter how small I made the bonus. I haven't found any move ordering for non-captures non-killers that beats straight piece-square-table ordering.
Anyway, thanks again! And, of course, always glad to hear about any new ideas/discoveries!
Cheers,
Greg
-
- Posts: 3697
- Joined: Tue Jul 31, 2007 4:26 pm
Re: EOC databases now as open source
just so you guys know
you are replying to a post Ed made two years ago
Time Fly's Regards
Stev
you are replying to a post Ed made two years ago
Time Fly's Regards
Stev
-
- Posts: 388
- Joined: Sun Dec 21, 2008 6:57 pm
- Location: Washington, DC
Re: EOC databases now as open source
Whoops! I saw "May 9" and thought it was current. Of course, since it's only May 8 perhaps that should have been a clue!Steve B wrote:just so you guys know
you are replying to a post Ed made two years ago
Time Fly's Regards
Stev
-
- Posts: 3697
- Joined: Tue Jul 31, 2007 4:26 pm
Re: EOC databases now as open source
Greg Strong wrote:Whoops! I saw "May 9" and thought it was current. Of course, since it's only May 8 perhaps that should have been a clue!Steve B wrote:
you never know
Ed still posts here from time to time
he could reply and you can pick up the discussion right where he left off.. two years ago
Back To The Future Regards
Steve
-
- Posts: 1342
- Joined: Wed Mar 08, 2006 9:41 pm
- Location: Morgantown, WV, USA
Re: EOC databases now as open source
What exactly is an EOC database? Curious because from my understanding it's an opening game database.
Is it similar to my goldentree project?
http://olympuschess.com/goldentree/
-Josh
Is it similar to my goldentree project?
http://olympuschess.com/goldentree/
-Josh