Komodo 9 vs Komodo 10 memory difference

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Paul Bedrey
Posts: 303
Joined: Thu Mar 09, 2006 10:46 am
Location: Saratoga Springs New York

Komodo 9 vs Komodo 10 memory difference

Post by Paul Bedrey » Sun Jun 12, 2016 9:05 am

I noticed that Komodo 10 runs in Arena with a significant memory difference than Komodo 9. Arena reports 4096 for Komodo 9 and all other engines but only 3181 for Komodo 10. I double checked it with Windows Task manager and it agrees with Arena. Is this normal? Can I change it? Under UCI options I have set a common hash size to 4096. When loading out Komodo 10 it reports hash table is now set at 4096 but memory is reported as 3181 mb. Up till now I thought memory and hash were the same. Are they?

Modern Times
Posts: 2346
Joined: Thu Jun 07, 2012 9:02 pm

Re: Komodo 9 vs Komodo 10 memory difference

Post by Modern Times » Sun Jun 12, 2016 9:16 am

The Komodo documentation explains this.
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

Paul Bedrey
Posts: 303
Joined: Thu Mar 09, 2006 10:46 am
Location: Saratoga Springs New York

Re: Komodo 9 vs Komodo 10 memory difference

Post by Paul Bedrey » Sun Jun 12, 2016 10:34 am

Komodo 10 documentation explains how to calculate hash size. My question is why Komodo 10 ignores the 4096 number I set in the UCI option I set in Arena. All other engines including Komodo 9 use my settings. I've read your and other test results and there is no question that Komodo 10 is at least 50 points stronger than Komodo 9 yet I have not had that experience and was wondering why.

Modern Times
Posts: 2346
Joined: Thu Jun 07, 2012 9:02 pm

Re: Komodo 9 vs Komodo 10 memory difference

Post by Modern Times » Sun Jun 12, 2016 11:16 am

The explanation is in the documentation....
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

mjlef
Posts: 1419
Joined: Thu Mar 30, 2006 12:08 pm
Contact:

Re: Komodo 9 vs Komodo 10 memory difference

Post by mjlef » Sun Jun 12, 2016 11:43 am

Komodo does things differently than other engines. One change since early Komodo 9 versions is I made the size of a hash entry smaller. The smaller the entry the faster the access. Most engines hash entries are power of 2. Like 8 or 16 bytes. Komodo uses 12 bytes, so hash allocation uses he formula 12 * 2^n. So 12, 24, 48, 96, 192... are the hash suzes used. Some GUIs only allow powers of 2. So Komodo will use the largest size that fits in the GUIs hash request that fits the formula above. Think of it as getting 25% more hash in the same space as old Komodo.a nd it is about 2% faster.

Mark

Karol Majewski
Posts: 94
Joined: Sat Jul 09, 2011 2:18 pm

Re: Komodo 9 vs Komodo 10 memory difference

Post by Karol Majewski » Sun Jun 12, 2016 12:43 pm

mjlef wrote:Komodo does things differently than other engines. One change since early Komodo 9 versions is I made the size of a hash entry smaller. The smaller the entry the faster the access. Most engines hash entries are power of 2. Like 8 or 16 bytes. Komodo uses 12 bytes, so hash allocation uses he formula 12 * 2^n. So 12, 24, 48, 96, 192... are the hash suzes used. Some GUIs only allow powers of 2. So Komodo will use the largest size that fits in the GUIs hash request that fits the formula above. Think of it as getting 25% more hash in the same space as old Komodo.a nd it is about 2% faster.

Mark
Mark, shouldn't you then set default Hash value to 96 or 192 instead of 128?

Paul Bedrey
Posts: 303
Joined: Thu Mar 09, 2006 10:46 am
Location: Saratoga Springs New York

Re: Komodo 9 vs Komodo 10 memory difference

Post by Paul Bedrey » Sun Jun 12, 2016 5:37 pm

OK thanks Mark as long as I know that I don't have a crippled engine. I noticed that Komodo 10 was using roughly 80% of whatever hash I set. I verified that Komodo 9.42 is doing the same too. I run head to to head matches between Komodo and Stockfish using 2 machines entering moves by hand. It's time consuming since I'm using 30 minute 3 sec time control with ponder on. I've stopped the match twice so far because I just can't believe the lopsided result. I'm using the Silver testsuite so maybe it'll even out eventually. Since each engine plays both sides I never had a result this strange before. I even broke out a 3rd quad playing no ponder games all other conditions the same and it is showing a very similar result. In reported head to head matches Stockfish usually is scoring 52% but I'm getting an unbelievable 62-63% difference. I've played hundreds of games with Komodo 9 the past year and never had that happen so I was surprised by this Komodo 10 result. I had assumed by using a testsuite rather than a book I was lowering statistical randomness. I guess I was wrong.

User avatar
hgm
Posts: 23385
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: Komodo 9 vs Komodo 10 memory difference

Post by hgm » Sun Jun 12, 2016 6:26 pm

mjlef wrote:Think of it as getting 25% more hash in the same space as old Komodo.a nd it is about 2% faster.
Actually in the described case it is more like you get the same hash in 25% smaller memory. This still of course benefits from the 2% speed increase, but leaving memory unused is of no benefit.

You would of course benefit enormously when users set all engine's hash size to 12*2^n, as that would reduce opponents that need powers of 2 for hash size to use 8*2^n, i.e. 33% less memory than Komodo. (And they would likely have larger entries too.)

Can't you use all the memory the user allows you? It is not that more expensive to derive an index in an arbitrary range from the hash key. Just an extra multiply and shift.

zullil
Posts: 5455
Joined: Mon Jan 08, 2007 11:31 pm
Location: PA USA
Full name: Louis Zulli

Re: Komodo 9 vs Komodo 10 memory difference

Post by zullil » Mon Jun 13, 2016 1:10 am

mjlef wrote: hash allocation uses he formula 12 * 2^n. So 12, 24, 48, 96, 192... are the hash suzes used.
Mark
Gee, I thought it was 3 * 2^n. :wink:

User avatar
hgm
Posts: 23385
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: Komodo 9 vs Komodo 10 memory difference

Post by hgm » Mon Jun 13, 2016 8:40 am

SzG wrote:Can it be set to 3 GB?
Probably. But not to 3 bytes. But I guess the semantics of the UCI Hash option would prevent that anyway.

Post Reply