UCI Hash Usage Rules

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

MOBMAT
Posts: 385
Joined: Sat Feb 04, 2017 11:57 pm
Location: USA

UCI Hash Usage Rules

Post by MOBMAT »

The "hash" option in UCI is used to set the size of the program's hash table. Most programs make use of hash tables.Some programs also have additional hashes/caches for eval, pawns, TBs, etc.

I recall reading somewhere (sorry, I can't provide a link), where it was stated that the hash size specified by UCI is suppose to represent the upper limit of memory use. So, if one adds up the sizes of all the hashes, it shouldn't exceed the value specified by the hash value.
  • Is this an accurate interpretation of the "rule"?
    Do programs (read yours) really adhere to this?
    Is it actually enforced by any any GUI/Tournament software?
    What is the common sense approach to this making it fair?
    Should I just use the size for my main hash size and do whatever I want with the other hashes/caches?
i7-6700K @ 4.00Ghz 32Gb, Win 10 Home, EGTBs on PCI SSD
Benchmark: Stockfish15.1 NNUE x64 bmi2 (nps): 1277K
AndrewGrant
Posts: 1750
Joined: Tue Apr 19, 2016 6:08 am
Location: U.S.A
Full name: Andrew Grant

Re: UCI Hash Usage Rules

Post by AndrewGrant »

As far as I know, the main Hash UCI option is meant to only talk about the primary Transposition Table.

Any other large tables, sized dynamically, I would list seperate options for. Some engines have an Eval Cache.

As for "Upper Bound", I think this is nonsense. Once you start capping the top memory usage, you have to start thinking about all the tiny "tables" that are in use. For example, I have Counter Move Tables, Killer Move Tables, History Tables, Counter Move History Tables, Pawn Tables (Static size), ...
#WeAreAllDraude #JusticeForDraude #RememberDraude #LeptirBigUltra
"Those who can't do, clone instead" - Eduard ( A real life friend, not this forum's Eduard )
Ras
Posts: 2487
Joined: Tue Aug 30, 2016 8:19 pm
Full name: Rasmus Althoff

Re: UCI Hash Usage Rules

Post by Ras »

MOBMAT wrote: Mon Jul 02, 2018 12:23 amSo, if one adds up the sizes of all the hashes, it shouldn't exceed the value specified by the hash value.
Correct. But usually, the main hash tables are the only important memory consumer. Static pawn hash tables are typically below 100 kB, which is small in comparison to the 1 MB resolution that you can set via UCI. More doesn't make sense for the pawn hash table because the hit rate doesn't really increase beyond roughly 10k entries. Other static caches are only 1-2kB or so, not worth considering.

Of course, if you had other hashing buffers of several MB, then you would need to take them into account.
Rasmus Althoff
https://www.ct800.net
User avatar
hgm
Posts: 27787
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: UCI Hash Usage Rules

Post by hgm »

AndrewGrant wrote: Mon Jul 02, 2018 1:07 am As far as I know, the main Hash UCI option is meant to only talk about the primary Transposition Table.

Any other large tables, sized dynamically, I would list seperate options for. Some engines have an Eval Cache.
This is wrong. The UCI specs clearly mention 'hash tables', plural. Insignificant tables don't have to be counted, but when an engine starts to use significant memory for other purposes (such as keeping bitbases in memory), it should come from the value indicated by the Hash option.
User avatar
Matthias Gemuh
Posts: 3245
Joined: Thu Mar 09, 2006 9:10 am

Re: UCI Hash Usage Rules

Post by Matthias Gemuh »

hgm wrote: Mon Jul 02, 2018 9:24 am
AndrewGrant wrote: Mon Jul 02, 2018 1:07 am As far as I know, the main Hash UCI option is meant to only talk about the primary Transposition Table.

Any other large tables, sized dynamically, I would list seperate options for. Some engines have an Eval Cache.
This is wrong. The UCI specs clearly mention 'hash tables', plural. Insignificant tables don't have to be counted, but when an engine starts to use significant memory for other purposes (such as keeping bitbases in memory), it should come from the value indicated by the Hash option.
ChessGUI has a parameter "Reduce Mem" that reduces Hash value of engines that steal too much extra Hash.
My engine was quite strong till I added knowledge to it.
http://www.chess.hylogic.de
elcabesa
Posts: 855
Joined: Sun May 23, 2010 1:32 pm

Re: UCI Hash Usage Rules

Post by elcabesa »

To make an example Vajolet has a footprint in memory of 3.1MB plus the hash table. In the 3.1MB are counted all variables, pawn hash table, magic move initialization tables, search history tables, quiet moves stat tables and so on. I don't know what it's considered fair, but I think that if you have an additional table of 64 MB somewhere... It should be counted and it's not trascurable.
Stockfish seems to have a footprint of 8 MB on my machine for example
DustyMonkey
Posts: 61
Joined: Wed Feb 19, 2014 10:11 pm

Re: UCI Hash Usage Rules

Post by DustyMonkey »

It is up to the user to decide the value of every parameter. UCI is not defined with "fairness" built in because thats wholly subjective. A tournament organizer would be responsible for "fairness" and that extends beyond just the hash value.
User avatar
hgm
Posts: 27787
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: UCI Hash Usage Rules

Post by hgm »

The point is that the parameters should have a fixed meaning for the user to sensibly set them. If an engine would use 640MB hash when the user specifies 64MB, it is simply not compliant. The idea of these standard options is that the GUI can apply them to all engines, having the same effect in each case.
DustyMonkey
Posts: 61
Joined: Wed Feb 19, 2014 10:11 pm

Re: UCI Hash Usage Rules

Post by DustyMonkey »

Most engines use 1 thread more than is specified by the user. This thread is merely for the UI in hopefully all of these engines and is considered by many to be inconsequential.

Haven't seen that debate.

Some engines ponder on the ponder move specifically, while others ponder on all the possible opponent moves.

Haven't seen that debate either.

Shouldn't be seeing this one. UCI is not a chess engine virtual machine. You guys are going to really hate it as engines with network evaluators grow in number. These will easily have hundreds of megabytes of built-in feature vectors.
User avatar
hgm
Posts: 27787
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: UCI Hash Usage Rules

Post by hgm »

' Threads' is not a UCI standard option, and engines that define it specifically define it as the number of search threads. So it is weird that you think anyone should object to that they use that number of search threads.

UCI specs allow explicitly to do whatever you want in ponder time. So again, you seem to have no point. To be non-compliant w.r.t. pondering, the engine would have to use significant CPU while pondering is off. An engine that would just run a task to flush the caches as quickly as possible while the opponent is thinking, to slow him down in a no-ponder game, would spark a debate, don't you think? The excuse "what are you complaining about, I am not pondering" is not likely to be accepted.