Hash Sizes... Again.

[Uri Blass]

It is not only speed and with more hash I can expect better results at the same depth because of more nodes that you prune.

I know some endgame position(fine70) when programs can see the best move at smaller depth with hash relative to not using hash.

[d]8/k7/3p4/p2P1p2/P2P1P2/8/8/K7 w - - 0 1 bm Kb1

[shrapnel]

The only tests that I know that people did to test different hash is from the stockfish framework

Here are the relevant results

http://tests.stockfishchess.org/tests/v ... 0b12db5ac4

32 mbytes is probably the best hash for 15+0.05(15 seconds per game+0.05 seconds per move) it could beat 128 mbytes by the following result

ELO: 2.59 +-2.0 (95%) LOS: 99.5%
Total: 40000 W: 6800 L: 6502 D: 26698

16 mbytes seems to be the same strength as 128 mbytes

http://tests.stockfishchess.org/tests/v ... 18089440c3

ELO: 0.44 +-2.0 (95%) LOS: 67.1%
Total: 40000 W: 6662 L: 6611 D: 26727

8 mbytes seems to be only slightly weaker

http://tests.stockfishchess.org/tests/v ... 1df44848cb

ELO: -2.19 +-2.0 (95%) LOS: 1.4%
Total: 40000 W: 6558 L: 6810 D: 26632

only with 4 mbytes stockfish found a big reduction in playing strength

http://tests.stockfishchess.org/tests/v ... 531a6b3404

ELO: -13.91 +-2.0 (95%) LOS: 0.0%
Total: 40000 W: 5833 L: 7434 D: 26733

Hi
That is indeed interesting !
But no one seems to have checked out what happens when Hash reaches Gigabyte Sizes in LTC games.
I'll check it out later,when I have more time.
Thank You

[dark_wizzie]

Uri, I'm guessing we can't just double it forever?

If 15 second game needs 16mb minimum...
30 second, 32mb
1 minute, 64mb
2 minutes, 128mb
4 minutes, 256mb
8 minutes, 512mb
16 minutes, 1024mb?

[hgm]

'Need' is a relative concept.

More proper to say is that "for 15-sec games, it is pointless to have more than 16MB hash", etc. I.e. you would not get any speedup from increasing the amount of hash above those 16 MB, because you already have more hash entries than the size of the search tree, so it will always fit in its entirety.

That doesn't mean it wouldn't work with less hash memory. Just that it is somewhat slower. If you use 4MB in stead of 16, it might run, say, 16% slower. Which would on averge cost you 16 Elo, which, against the same opponent(s), would make you score about 2 percent less. Which means the difference would only get noticeable (i.e. above the statistical noise due to luck) after some 400 games. Hardly the end of the world, considering you gave it 4 times less than what it 'needs'.

[Uri Blass]

Uri, I'm guessing we can't just double it forever?

If 15 second game needs 16mb minimum...
30 second, 32mb
1 minute, 64mb
2 minutes, 128mb
4 minutes, 256mb
8 minutes, 512mb
16 minutes, 1024mb?

I think that we need testing to know.
Maybe at some time control doubling stop to be productive when we double the hash size.

I suggest the following tests for people who have computer time

1)15 seconds
1 mb vs 8 mb 10000 games
2)30 seconds
2 mb vs 16 mb 10000 games
3)1 minute
4 mb vs 32 mb 10000 games
4)2 minutes
8 mb vs 64 mb 10000 games
5)4 minutes
16 mb vs 128 mb 10000 games
6)8 minutes
32 mb vs 256 mb 10000 games
7)16 minutes per game
64 mb vs 512 mb 10000 games
8)32 minutes per game
128 mb vs 1024 mb 10000 games
9)64 minutes per game
256 mb vs 2048 mb 10000 games
10)128 minutes per game
512 mb vs 4096 mb 10000 games

I think that we never saw an evidence that there is a rating improvement in the last cases and even if there is a rating improvement then if it goes down significantly from clearly more than 20 elo to clearly less than 10 elo then it may suggest that maybe at some point more hash is simply bad for all time controls.

[APassionForCriminalJustic]

If I am going to play an LTC game of let's say 120 minute\60 second increment then I think I will stick with 16 gigabytes as opposed to 1 gigabyte. Plain and simple. Using Komodo 8 at 120\60, it is quite clear that 16 gigabytes really amps up the NPS. There is nothing of a slowdown, not with my hardware (2660 v2 20 core dual socket). Eric, if you are going to spend that kind of money on a CPU then you might as well buy the most amount of RAM that you can afford. Some of the tests you propose would just take way too long.

Do you know how much of Ronald de Mans tablebases that I cache into the RAM during my engine games on ICC? With 64 gigabytes we are talking about up to 40 gigabytes (then 16 gigabytes for hash table utilization). Of course the caching of the tablebases is not immediate. Nevertheless, we all know that access to the random access memory beats access to the SSDs. Hence, there must be some obvious gain of ELO here.

There is always need for better hardware for computer chess entities. Authors might as well make the RAM useful for their engines than not. The engine not needing to research the same transpositions again must matter. In my tests, Komodo 8 really does a nice job with 16 gigabytes LTC. But I do not know as to how far in ELO the advantage actually occurs.

APassionforCriminalJustice

[Uri Blass]

I do not claim that there is no advantage but only that we do not know it
and it may be dependent on the engine.

Number of nodes per second does not tell the full story
and I am not sure that same hash is better even with the same number of nodes.

Chess programs have late move reductions(that are based on the history of the search and not only on the position).
In theory it is possible that reductions that are productive with 64 mbytes start to be counter productive with big hash because big hash cause the history of the search to be different so it is better to have smaller hash in order not to reduce the advantage of the existing reductions.

The best solution in this case may be to use the size of hash to decide how many plies to reduce but I know of no programs that even tested it.

What is possible to do is to test the optimal reduction with very small hash like 1 mbyte hash and with 64 mbyte hash and if you find for example that it is better to reduce more with 1 mbytes hash then try to do less LMR with bigger hash.

Note that
I do not claim that it is better to reduce more with small hash and it may be the opposite but only that it is not obvious that the same reduction that are best for one hash size are best for another hash size.

[dark_wizzie]

I do not claim that there is no advantage but only that we do not know it
and it may be dependent on the engine.

Number of nodes per second does not tell the full story
and I am not sure that same hash is better even with the same number of nodes.

Chess programs have late move reductions(that are based on the history of the search and not only on the position).
In theory it is possible that reductions that are productive with 64 mbytes start to be counter productive with big hash because big hash cause the history of the search to be different so it is better to have smaller hash in order not to reduce the advantage of the existing reductions.

The best solution in this case may be to use the size of hash to decide how many plies to reduce but I know of no programs that even tested it.

What is possible to do is to test the optimal reduction with very small hash like 1 mbyte hash and with 64 mbyte hash and if you find for example that it is better to reduce more with 1 mbytes hash then try to do less LMR with bigger hash.

Note that
I do not claim that it is better to reduce more with small hash and it may be the opposite but only that it is not obvious that the same reduction that are best for one hash size are best for another hash size.

Without such tests I dunno how much ram to get for the 5980x.

[Zenmastur]

More hash is better is an an unproved opinion.

I do not know of a rating list that tested Komodo8 or Stockfish
with 16 GB and with 1 GB to show that the 16 GB version has a better rating at some time control.

Maybe not, but I'm not even a little bit convinced that less memory is better!

I agree. When you enter the parameter region where the tree no longer fits in hash in its entirety, a good hash implementation loses only about 8 Elo per hash-memory halving.

As I understand it, detecting an 8 ELO difference takes quite a few games. A single test of, say 10,000 games, would be insufficient. It's seems more likely that many test with different hash sizes and time controls would need to be done with 30k-40k games per test.

So, if this hasn't been extensively tested how could you possibly know that a program will only lose 8 ELO when the hash is halved? Sounds like your making the answers up as you go a long.

But there are other issues than just the amount of RAM. With too large a table size you will get more misses in the TLB cache, and even in higher levels of the TLB cache, slowing the engine down. It is not difficult at all to lose the tiny 8 Elo advantage by a RAM doubling to a slowdown of the engine.

First, until we know that 8 ELO is, in fact a valid number this statement doesn't mean much. Second, what percentage TLB misses are required to cause the program to run at 95% of its nominal speed? Third, all nodes are not created equal. i.e. counting nodes isn't likely to give a good answer to this question. The thing that really matters is the quality of the line of play ( or root move if you like) that the program produces in a given time. Nothing else matters.

Well, I systematically investigated how the size of the search tree varied with hash size, in order to evaluate various replacement schemes. As this is almost exclusively a speed effect, it doesn't make much sense to test it in games. Just recording nodes to depth for some 1000 positions would be good enough.

I don't follow what you are saying. What exactly did you investigate? What were the results and you conclusions? Why do you think recording nodes of search to some depth will help?

It is not only speed and with more hash I can expect better results at the same depth because of more nodes that you prune.
I know some endgame position(fine70) when programs can see the best move at smaller depth with hash relative to not using hash.

[d]8/k7/3p4/p2P1p2/P2P1P2/8/8/K7 w - - 0 1 bm Kb1

Sound like you're making an argument to remove TT tables all together.

As far as fine70 or any other position is concerned depth is irrelevant, time to solution is what makes a difference. Or time to best response if no exact solution exists.

I think that we need testing to know.
Maybe at some time control doubling stop to be productive when we double the hash size.

I suggest the following tests for people who have computer time

1)15 seconds
1 mb vs 8 mb 10000 games
2)30 seconds
2 mb vs 16 mb 10000 games
3)1 minute
4 mb vs 32 mb 10000 games
4)2 minutes
8 mb vs 64 mb 10000 games
5)4 minutes
16 mb vs 128 mb 10000 games
6)8 minutes
32 mb vs 256 mb 10000 games
7)16 minutes per game
64 mb vs 512 mb 10000 games
8)32 minutes per game
128 mb vs 1024 mb 10000 games
9)64 minutes per game
256 mb vs 2048 mb 10000 games
10)128 minutes per game
512 mb vs 4096 mb 10000 games

I think that we never saw an evidence that there is a rating improvement in the last cases and even if there is a rating improvement then if it goes down significantly from clearly more than 20 elo to clearly less than 10 elo then it may suggest that maybe at some point more hash is simply bad for all time controls.

Sounds like a lot of work, too much in fact. Having a large suite ( a few hundred to a few thousand ) of test positions for which the game theoretical value of all moves is known would greatly speed up the testing process. While it's a lot of work to find or generate these positions, it only has to be done once. Then you can run the tests as many times as required and the scoring of the generated moves for each position is exact and trivial to conduct. In the mean time, it might be good to test these theories on a standard test suite.

I do not claim that there is no advantage but only that we do not know it and it may be dependent on the engine.

Number of nodes per second does not tell the full story
and I am not sure that same hash is better even with the same number of nodes.

Chess programs have late move reductions(that are based on the history of the search and not only on the position). In theory it is possible that reductions that are productive with 64 mbytes start to be counter productive with big hash because big hash cause the history of the search to be different so it is better to have smaller hash in order not to reduce the advantage of the existing reductions.

The best solution in this case may be to use the size of hash to decide how many plies to reduce but I know of no programs that even tested it.

What is possible to do is to test the optimal reduction with very small hash like 1 mbyte hash and with 64 mbyte hash and if you find for example that it is better to reduce more with 1 mbytes hash then try to do less LMR with bigger hash.

Note that I do not claim that it is better to reduce more with small hash and it may be the opposite but only that it is not obvious that the same reduction that are best for one hash size are best for another hash size.

Hmmm... not sure what to make of these comments. If your not searching a position because its been pruned it won't have any impact on the hash table. So what is your point?

Without such tests I dunno how much ram to get for the 5980x.

This depends a lot on what you plan on doing with the system. I too am planning a system upgrade. The system I have now, I built for low power consumption. Less power equals less heat. I was tired of walking into a computer room that felt like a sauna and sounded like a hurricane for all the fans. While the system is quiet and produces little heat it has trouble producing high nps. About 8m nps in opening and middle-game climbing to a max of ~22m nps in some endgames using SF.

Although I haven't decided on the exact specs, it will be either a Skylake or broadwell CPU. Probably Broadwell since I don't want to wait for Skylake if its introduced too much later. One thing I have decided is that at a minimum it will have 32gb to start, and its more likely to have 64-128 gb depending on my budget when these CPU's become available. I also want enough dimm slots so I can increase this later if I feel the need. I made the mistake when I built my current system of not getting enough dimm slots on the MB. It only has 4 and I put 4 4gb dimms in them. The MB can Handle 32MB but I would have to buy all new memory to do that, since it's DDR3 I don't want to waste my time or money on older technology memory. The one thing I did do right was I put relatively fast memory in the system. When I wanted more speed I jumped the memory speed from 1,333 to 1,866 and jacked the CPU from 3.4 ghz to 4.7 ghz. This made a huge difference in NPS. I had the CPU at 5.1 ghz but I couldn't keep the cpu cool on the low cost cooler I have. The new system will have lots of very fast ram, lots of dimm slots, lots of cores and lots of cooling! And I may get more than one of them!

Then it will be back to the sauna again.

Regards,

Forrest