Windows and large hash tables

[gordonr]

I have 64 GB RAM and currently using Windows 11 Pro. But I'm noticing an issue...

i) I run Stockfish for long analysis with 32 GB hash tables. I stop the search but keep the process with its hash tables
ii) I run Komodo Dragon for long analysis with 16 GB hash tables. I stop the search but keep the process with its hash tables
iii) I run LC0 for long analysis and use about 4 GB of memory (seen in task manger)

So a total of 32 + 16 + 4 = 52 GB out of 64 GB. Nothing else is running.

But what I'm often seeing is that during (iii), according to task manager, it looks like the memory for (i) and (ii) is being swapped out (?) since the percentage drops to 0% for Stockfish and Komodo processes. And the overall RAM usage drops low too.

Then when I stop LC0 and go back to the Stockfish process to tell it to search again, the nodes per second is initially crippled for minutes. It's as if the process is initially running much slower due to swapping memory back in? After maybe 5 minutes or so, the KNPS reaches a normal figure.

I've also tried reducing the total memory needed by reducing Komodo's hash tables to 8 GB (so now using a total of 44 out of 64 GB) and I still see the issue. I'm certain that I'm not using all of my RAM and allowing enough RAM for Windows itself (I think 4 GB or so is needed). It appears as if a process that has been idle, for a few hours, Windows is then doing something with its memory usage?!

In case it's relevant, I'm starting the engines processes via some of my own code. It could be that I'm not giving them a correct priority setting or similar.

I can't find anything useful online or while browsing Windows settings.

Thanks for any thoughts.

[Modern Times]

Could be something to do with the caching of Syzygy tablebases, especially 6-men if you're using those. That can take up huge amounts of RAM. As an experiment, try analyses without using them and see if the problem still happens.

[Jouni]

Have You activated large pages in W11?

[gordonr]

Could be something to do with the caching of Syzygy tablebases, especially 6-men if you're using those. That can take up huge amounts of RAM. As an experiment, try analyses without using them and see if the problem still happens.

Good point - I hadn't considered that. I am using 6-men. However, this also happens when analysing opening positions with relatively little TB hits. I'm assuming Windows task manager is showing me the total memory usage for the engine process, including hash tables, tablebases, etc. But I will experiment with Syzygy disabled.

Thanks.

[gordonr]

Have You activated large pages in W11?

No I hadn't but I have started experimenting with it. Now when I run Stockfish with 32 GB hashtables, Windows task manager is showing under 1 GB of memory being used for Stockfish?! However, task manager's total memory is showing around 58% which looks right but the individual processes don't add up to this total?! Does task manager get "confused" when using large tables? Stockfish appears to be running ok - just its individual memory usage looking strange in task manager.

I'm still in the process of making sure I've got large pages working ok before I try reproducing my initial problem.

Thanks

[Jouni]

RubiChess tells You, if LP are activated: "Allocation of memory uses large pages."

[Modern Times]

I'm assuming Windows task manager is showing me the total memory usage for the engine process, including hash tables, tablebases, etc. But I will experiment with Syzygy disabled.

Thanks.

No it isn't. If the OS is caching tablebases, then that memory usage won't show as part of the memory usage of the engine itself.

I've had issues before with long time control games using 6-men. You need a lot of RAM. You could try lower hash sizes for the engine and see if that helps.

[Modern Times]

I have 64 GB RAM and currently using Windows 11 Pro. But I'm noticing an issue...

i) I run Stockfish for long analysis with 32 GB hash tables. I stop the search but keep the process with its hash tables
ii) I run Komodo Dragon for long analysis with 16 GB hash tables. I stop the search but keep the process with its hash tables
iii) I run LC0 for long analysis and use about 4 GB of memory (seen in task manger)

So a total of 32 + 16 + 4 = 52 GB out of 64 GB. Nothing else is running.

If I read this right, you have 52GB of engine RAM in use, plus the O/S will be using RAM on top. You've got less than 10GB of headroom far caching of 6-men tablebases, simply not enough for long analyses.

[gordonr]

I have 64 GB RAM and currently using Windows 11 Pro. But I'm noticing an issue...

i) I run Stockfish for long analysis with 32 GB hash tables. I stop the search but keep the process with its hash tables
ii) I run Komodo Dragon for long analysis with 16 GB hash tables. I stop the search but keep the process with its hash tables
iii) I run LC0 for long analysis and use about 4 GB of memory (seen in task manger)

So a total of 32 + 16 + 4 = 52 GB out of 64 GB. Nothing else is running.

If I read this right, you have 52GB of engine RAM in use, plus the O/S will be using RAM on top. You've got less than 10GB of headroom far caching of 6-men tablebases, simply not enough for long analyses.

Ok, I had misunderstood this in terms of what task manager was telling me. I've now observed the total RAM usage growing significantly with a long analysis even if the individual process figures aren't showing it. Thanks for highlighting this.

I now think I will terminate each engine process that is idle. The potential benefit of maintaining the hashtables between searches probably isn't worth me reducing the hash size and/or disabling tablebases.

I'm also going ahead with Jouni's suggestion of using large pages in Windows. I'd heard of these before but never investigated.

Thanks to everyone from their input. I've learned a bit more