Memory question

[Kempelen]

I admit one shoult not let the engine to change its internat TT table size, but would be nice if it could send to the GUI a message (like when sending nps or other internal data) when engine detect performance is going down due to hardware inneficience. That is the reason I was asking for a way to know when engine is using swap a lot.

[Codesquid]

Code: Select all

// In MiB
int get_system_memory()
{
	uint64_t pages = sysconf(_SC_PHYS_PAGES);
	uint64_t page_size = sysconf(_SC_PAGE_SIZE);
	return pages * page_size / 1024 / 1024;
}

And for the obscure Windows platform some strange people are using :

Code: Select all

int get_system_memory()
{
	MEMORYSTATUSEX status;
	status.dwLength = sizeof(MEMORYSTATUSEX);

	GlobalMemoryStatusEx( &status );

	return static_cast<int>&#40;status.ullTotalPhys / 1024 / 1024&#41;;
&#125;

[bob]

Code: Select all

// In MiB
int get_system_memory()
{
	uint64_t pages = sysconf(_SC_PHYS_PAGES);
	uint64_t page_size = sysconf(_SC_PAGE_SIZE);
	return pages * page_size / 1024 / 1024;
}

And for the obscure Windows platform some strange people are using :

Code: Select all

int get_system_memory()
{
	MEMORYSTATUSEX status;
	status.dwLength = sizeof(MEMORYSTATUSEX);

	GlobalMemoryStatusEx( &status );

	return static_cast<int>&#40;status.ullTotalPhys / 1024 / 1024&#41;;
&#125;

I am not sure that SC_PHYS_PAGES is available on all unix systems, based on the linux man page. Also, one has to somehow figure out how many of those pages are usable, since quite a few are dedicated to the O/S and its proc tables...

I'd think that if a program MUST do this (and I think it is a bad idea, just as assuming that the engine can use all available cores is a bad idea) the best approach is to choose a hash size, run a test, and then double hash and re-run until the program notices the nps cratering, which it will eventually when it starts to page.

Just because someone CAN do something does not mean they should do it. You CAN put your foot under a running lawn mower, even though the warning decal on the top tells you not to. But SHOULD you?

[ZirconiumX]

You could always try the Cyclone Dynamic Hash.

http://users.telenet.be/chesslogik/dynamic_hash.htm

Of course, you don't have to follow my advice.

Matthew:out

[bob]

You could always try the Cyclone Dynamic Hash.

http://users.telenet.be/chesslogik/dynamic_hash.htm

Of course, you don't have to follow my advice.

Matthew:out

Actually it would appear you followed MY advice.

This is not new at all. It has been in Crafty since version 19.10, which was released early in 2004. All it needs to know is the time control, which it gets from xboard, and it will compute a pretty optimal setting for hash and pawn hash. You simply need to tell it the MAX memory it is allowed to use. It will use optimal settings up to that limit.

I don't think assuming "all of memory is available" is very wise, as once you start paging, things to badly, and quickly...

For Crafty, the command is:

"adaptive NPS min-hash-memory max-hash-memory min-phash-memory max-phash-memory"

Once you enter that, the time control chosen will set the hash size optimally within those bounds...

[ZirconiumX]

So there you have it, a solution for xboard, and for UCI.

Matthew:out

[bob]

So there you have it, a solution for xboard, and for UCI.

Matthew:out

Not a "solution". The idea was to use available memory, which is a bad idea. My "solution" requires that the human specify a max memory size, not having the program look to see what is available and use everything...

My point was that the "solution" you mentioned in your program was not new, it has existed for 8+ years, but it doesn't try to grab all of memory, just what you say it can use, which is not what was originally being discussed here..