Low hash size in Droidfish

[Gurcan Uckardes]

I searched a lot but didn't find a clear answer. I use Chess for Android for automated games in my Rapidroid experiment and Droidfish for manual games vs non-Android engines.
I've used 256MB hash per engine with Chess for Android without problems since thousands of games but when playing with Droidfish, same engines can't get more than 64MB, regardless of my setting.
The devices have 2GB ram and ~1GB free
I think in my case one single engine may be easily allowed to take 512MB at least. Why 64MB max?
Did anyone had the same issue?
Is there sonething missed in the algorhytm?

[petero2]

I searched a lot but didn't find a clear answer. I use Chess for Android for automated games in my Rapidroid experiment and Droidfish for manual games vs non-Android engines.
I've used 256MB hash per engine with Chess for Android without problems since thousands of games but when playing with Droidfish, same engines can't get more than 64MB, regardless of my setting.
The devices have 2GB ram and ~1GB free
I think in my case one single engine may be easily allowed to take 512MB at least. Why 64MB max?
Did anyone had the same issue?
Is there sonething missed in the algorhytm?

There is code in DroidFish that limits the hash table size to the value returned by Runtime.getRuntime().maxMemory(). This function returns the maximum amount of memory that a java program can use. I impose the same limit for engine processes, because at the time I implemented configurable hash table size (July 2012), without this limit some devices behaved very badly. The bad behavior included lockups, reboots and killing of the engine process.

I don't know if current android devices behave better when a process tries to use a very large amount of memory.

[Ferdy]

In my tab with 1 gb mem I can set to a max of 128 mb.

[Gurcan Uckardes]

Ferdinand did you check whether your setting is practiced by the gui? I use memorybar 1.2 which shows the free ram in the statusbar permanently. I woke up to the issue when i noticed that 64 or 128 or more hash settings go always to the same free ram once the engine is loaded and run.
Only when i reduce to 32MB, i start to see more free ram. This means the gui has some kind of "more than necessary" safety based limitation. It checks the user setting and decides what to do.

[Gurcan Uckardes]

There is code in DroidFish that limits the hash table size to the value returned by Runtime.getRuntime().maxMemory(). This function returns the maximum amount of memory that a java program can use. I impose the same limit for engine processes, because at the time I implemented configurable hash table size (July 2012), without this limit some devices behaved very badly. The bad behavior included lockups, reboots and killing of the engine process.

I don't know if current android devices behave better when a process tries to use a very large amount of memory.

Peter, on my 1gb tablets (rockchip android 4.1, intel 3745 kitkat) 128mb is no problem with Chess for Android. On 2gb tablets (samsung Note 8.0), 256mb is no problem.
CfA handles two engines and two hashes simultaneously. That means Droidfish can allocate double of it because it loads one engine.
My suggestion is to recheck the formula in the code. I have 7 devices and i confirm 256mb is fine for 1gB ram, 512mb for 2GB.
The only case i can't speculate is Lollipop. I only use 32bits.
I must remind also that i don't use tablebases. My confirmed values must be rechecked with tablebases as well. But i don't think tablebase and other caches consume too much.
At worst, the formula can be improved to take into account tablebase settings.

[petero2]

There is code in DroidFish that limits the hash table size to the value returned by Runtime.getRuntime().maxMemory(). This function returns the maximum amount of memory that a java program can use. I impose the same limit for engine processes, because at the time I implemented configurable hash table size (July 2012), without this limit some devices behaved very badly. The bad behavior included lockups, reboots and killing of the engine process.

I don't know if current android devices behave better when a process tries to use a very large amount of memory.

Peter, on my 1gb tablets (rockchip android 4.1, intel 3745 kitkat) 128mb is no problem with Chess for Android. On 2gb tablets (samsung Note 8.0), 256mb is no problem.
CfA handles two engines and two hashes simultaneously. That means Droidfish can allocate double of it because it loads one engine.
My suggestion is to recheck the formula in the code. I have 7 devices and i confirm 256mb is fine for 1gB ram, 512mb for 2GB.
The only case i can't speculate is Lollipop. I only use 32bits.
I must remind also that i don't use tablebases. My confirmed values must be rechecked with tablebases as well. But i don't think tablebase and other caches consume too much.
At worst, the formula can be improved to take into account tablebase settings.

I think it will be hard to determine larger safe values for all android devices supported by DroidFish. What I could do for the next release is to create a new setting called something like "allow very large hash tables", with a warning that enabling this option could make DroidFish and/or the android OS unstable.

In the meantime, here is a development version that disables the hash size limitation:

DroidFish_largehash.apk

Note that this version also moves the number of threads setting to the engine UCI options dialog, mostly to be able to have different settings for different network engines.

In case someone wants to rebuild this DroidFish version from source code, here is the patch I used:

Code: Select all

diff --git a/DroidFish/src/org/petero/droidfish/engine/ExternalEngine.java b/DroidFish/src/org/petero/droidfish/engine/ExternalEngine.java
index be3ff57..db1275d 100644
--- a/DroidFish/src/org/petero/droidfish/engine/ExternalEngine.java
+++ b/DroidFish/src/org/petero/droidfish/engine/ExternalEngine.java
@@ -209,6 +209,7 @@ public class ExternalEngine extends UCIEngineBase {
 
     /** Reduce too large hash sizes. */
     private final int getHashMB(int hashMB) {
+        /*
         if (hashMB > 16) {
             int maxMem = (int)(Runtime.getRuntime().maxMemory() / (1024*1024));
             if (maxMem < 16)
@@ -216,6 +217,7 @@ public class ExternalEngine extends UCIEngineBase {
             if (hashMB > maxMem)
                 hashMB = maxMem;
         }
+        */
         return hashMB;
     }
 

[Ferdy]

Ferdinand did you check whether your setting is practiced by the gui? I use memorybar 1.2 which shows the free ram in the statusbar permanently. I woke up to the issue when i noticed that 64 or 128 or more hash settings go always to the same free ram once the engine is loaded and run.
Only when i reduce to 32MB, i start to see more free ram. This means the gui has some kind of "more than necessary" safety based limitation. It checks the user setting and decides what to do.

I am using OS monitor. I place it in notification area, I just pull this down to see, the cpu usage, free ram, battery power remaining, and other info like which process takes a lot of cpu, and memory. If I open the full window I can see more processes running, including seeing how many memory each process are using, how much cpu they are using and other process info. For Droidfish, I can see 2 process id's, one for the gui which uses around 35 mb and one for the engine which could change if you change the hash size of the program (my max is 128mb). I monitored two things, the overall free memory, and the actual memory used by the engine.
I cannot find memorybar 1.2? but there is memorybar simple 1.1.2, which can display free memory. I tried it and works the same way with OS monitor with regards to free memory display.

[Gurcan Uckardes]

Thanks Ferdinand, Memorybar you try is ok. The same.

Now i will try Droidfish dev version to see the outcome. Thanks to Peter.

[Gurcan Uckardes]

Thanks Ferdinand, Memorybar you try is ok. The same.

Now i will try Droidfish dev version to see the outcome. Thanks to Peter.

...same subject revisited, with a link to v1.59 large hash: HERE