I've been working recently on diminishing Prédateur's memory footprint and was quite successful :
- the latest public Prédateur (Pred 2.0) uses about 450 MB of RAM, including 128 MB of hashtables,
- whereas my latest beta, Pred 2.1 now uses around 223 MB of RAM, including 128 MB of hashtables.
To do that, I changed lots of data structures, especially my previously HUGE HistoryKillers array, that was around 150 MB big and is now 0.9 MB big.
Problem is :
223 - 128 = 95 MB
I also have a 8 MB pawnhash, so :
95-8 = 87 MB of non-hash memory use
BUT... When I sum up all my other arrays, they're about 3 MB big in total. So where does the rest of the memory use come from ?
As I saw no other way to reduce any more my memory use, I made some experiments and found something strange :
Prédateur with a 128 MB Hashtable : 97 MB of non-hash memory use
Prédateur with a 64 MB Hashtable : 63 MB of non-hash memory use
Prédateur with a 8 MB Hashtable : 32 MB of non-hash memory use
Clearly, the "non-hahs" memory use actually increases when the hash's size does. So there must be something wrong with the way I compute my hashtable's size. Problem is, I double-checked everything and can't find what's wrong. Here are the details of my implementation :
Here's a hash "slot" (my hash tables is a dynamical array made of such records) :
Code: Select all
Hash = record
ZobristKey : Int64; //8 bytes
Depth : SmallInt; //2 bytes
Flag : ShortInt; //1 bytes
Score : SmallInt; //2 bytes
Move : array[1..3] of ShortInt; //3 bytes
end; //Total : 16 bytes
(Source, for a better layout : http://wiki.freepascal.org/Variables_and_Data_Types )From the FPC manual
integer types
Type Range Bytes
Byte 0 .. 255 1
Shortint -128 .. 127 1
Smallint -32768 .. 32767 2
Word 0 .. 65535 2
Integer smallint or longint 2 or 4
Cardinal longword 4
Longint -2147483648 .. 2147483647 4
Longword 0..4294967295 4
Int64 -9223372036854775808 .. 9223372036854775807 8
QWord 0 .. 18446744073709551615 8
Then, when I set the hashtable's size to 128 MB (for instance) I compute the number of slots that way :
NbOfSlots := (128 * 1024 * 1024) div 16;
And I set it to this length doing a SetLength :
SetLength(HashTable, NbOfSlots);
So, what is wrong? Are the datatypes of a different size than advertised on the FreePascal's wiki, or is there some additional memory used to handle dynamical arrays of records ?
I'm stuck!