Learn from Database for Book Question

[ChessAddict]

I'm trying to do the "learn from database" option for my Openings Book. I'm trying to learn from 147,000 games. Problem is it's taking forever for it to learn from the games. It's showing a time of 30hours to complete the learning process. Is there something wrong why it's taking so long? I have a new computer. If i can remember the other computer i had did not take this long? Please help. Thanks.

[Mark Mason]

Hi Arnold,

Bear in mind that the Fritz GUI "Learn From Database" is an extremely complex function as you are effectively simulating the effect of the book actually playing every one of the games you have used to make it, and updating all the weightings accordingly for every move in the whole tree.

The length of time it takes depends very much on hardware speed but also size of book - not just number of games but also depth (eg. when you compile is it to 30, 50, 70, 100 deep etc). My current book is about 40,000 moves compiled at Absolute 70 and a full learn (ticking all four boxes - white and black wins & losses) - takes about 30 minutes on a Q6600 quad core with 3Gb RAM. To speed up the Learn I would suggest these options:

a) Reduce the number of games in the book, are you sure you really need 137,000 ? From your games database try filtering out and removing all games below a certain elo so that you have less games - but higher quality. Or, filter and remove all games below say 35 moves - this will take out spurious early finishes often due to things like disconnect. Again this will give you less games - but higher quality,
b) Reduce the depth - if you compile to 100, try 50. If you compile to 70 try 50.
c) Rather than learn all at once (wins, losses, white, black) try breaking it down to stages - run it for white first then afterward for black. This may help.
d) Borrow a faster computer from a friend to do the Learn
e) Invest in a faster computer...you know it makes sense !!

Good Luck,

Mark

[ChessAddict]

Hi,

I do have a fast computer. I have a Qx9650 with 2gb of corsair cmx 1333mhz sticks. When i check the the task manager while i'm doing the learn database function it shows the cpu usage at 0-2 percent usage? It's like it's not even using the processor? I'm not quite sure whats going on? I'm also just learning from white games and then trying to do the black games later but the white games are taking sooo long!!

[ChessAddict]

I don't know what i did but now the processor is working the "learn from database" function and is much faster than before. Wonder why it did not work before?

[ChessAddict]

I'm having the same problem again as the learning function is only learning one game at a time? It's not using the processor because i checked the task manager. I tried unloading and reloading the shredder chess program and still the same thing. Any suggestions on what i should do? Thanks.

[ChessAddict]

I'm kind of stumped as to why the program will not use any processor power for the "learn from database" function?

[Bill Rogers]

My laptop has an internal wireless modem which I found out stays on all the time. When I found a way to turn it off my computer started to FLY!
This may be your problem too.
In any event I would suggest start the program at night while you are sleeping.
Bill

[CRoberson]

Hi,

I do have a fast computer. I have a Qx9650 with 2gb of corsair cmx 1333mhz sticks. When i check the the task manager while i'm doing the learn database function it shows the cpu usage at 0-2 percent usage? It's like it's not even using the processor? I'm not quite sure whats going on? I'm also just learning from white games and then trying to do the black games later but the white games are taking sooo long!!

Assuming that the database operations (all book work is db work plus)
are not threaded, then the max cpu usage from it is 25% on a Qxnnnn.
Secondly, database ops are primarily disk ops which do not consume
cpu time - they put the process into a stall/wait state.

I recently sped up some large book work by 25x. It was simple.
I put all the files on a ramdisk.

[ChessAddict]

Well i put the database on a usb stick but still very slow and no processor usage. I also checked the modem and made sure that it was disabled and still the same problem. Wonder why it worked the one time? My database is 439mb size. Is this maybe why it's so slow? Thanks.