I think I stand by what I said in the past.IIRC with the XOR trick I measured nearly no collision, since the writes are atomic. It doesn't change the fact that everything surrounding the hash table is inherently unsafe - funnily this does not even change using safe Rust only, although one might trick oneself into thinking so. Hash key collisions happen often enough at TCEC timecontrols and hardware, such that a lot of engines (including FabChess) heavily crashed in pre-season TCEC testing. Of course this is a different kind of unsafe than we usually envision in the context of Rust, but a non crashing engine is ultimately the goal we want to achieve. The easiest solution is to treat everything coming out of the hash table as complete garbage(mainly we can check if TTMove bytes correspond to a move enconding and the move contained in the move encoding is valid in the position). Then one can also easily disregard any kinds of data races - simply because validness of TTMove will be checked anyway(the score might change, but I think we can handwave that - or can we?).mvanthoor wrote: ↑Sat Sep 12, 2020 11:38 pm
According to FabianVDW (of FabChess, which is also an engine in Rust), mutex locking works, but is too slow if an engine uses more than 8 threads. Then the overhead of the locks/unlocks becomes so big that more threads don't add any more speed. He says that "racey access to the hash table is necessary." That can be done with unsafe Rust, but it is not the preference, as this will drop you into manual memory management territory: YOU make sure that everything is correct instead of the compiler being able to check it for you. If you wrap code by unsafe { }, the compiler won't help you at all.
If the problem starts with 8 threads only, I think I'll first implement proper locked hash table sharing (when the time for SMP comes for my engine... after I hit at least 2850 ELO on a single thread), and if it isn't fast enough when running more than 4 threads, I'll look into lockless/unsafe Rust then.
Nonetheless, my feeling is that when thread-> infinity the strategy of allowing data races is suboptimal.