SF 191 Why not clear TT?

[mcostalba]

Actually with my tester I could easily do this - assuming Stockfish honors the clear_hash command or whatever it's called. And I think someone said it does.

SF honors all the UCI commands, so also "Clear hash", anyhow this is yet another indirect test.

I still think that if someone is really interested to clarify this point the correct approach is a direct measure, tweaking the TT entry to store a game ID, as I have presented above.

[Ralph Stoesser]

Actually with my tester I could easily do this - assuming Stockfish honors the clear_hash command or whatever it's called. And I think someone said it does.

SF honors all the UCI commands, so also "Clear hash", anyhow this is yet another indirect test.

I still think that if someone is really interested to clarify this point the correct approach is a direct measure, tweaking the TT entry to store a game ID, as I have presented above.

Agreed, have tried it. There are a few hits (~0.02% in case I do not repeat games with interchanged colors) from previous games, but the hit rate seems not to increase over time. (1000 1 min games)

I have a question regarding TT. Has someone ever tried to store two or more moves in the TT, for better move ordering?

[mcostalba]

Agreed, have tried it. There are a few hits (~0.02% in case I do not repeat games with interchanged colors) from previous games, but the hit rate seems not to increase over time. (1000 1 min games)

I have a question regarding TT. Has someone ever tried to store two or more moves in the TT, for better move ordering?

Thanks for this test ! Did you test also with interchanged colors ? And how big was the TT size ?

Regarding your question, yes I have tried to implement this but I gave up because code was becoming too messy for my taste. In particular merging of an already exsisting entry with a second new cut-off move was a bit tricky, so I stopped implementation, actually I didn't spend a lot of time on it and if it turns out to have some _potential_ I think I could respin that and fully implement it.

[Ralph Stoesser]

Thanks for this test ! Did you test also with interchanged colors ? And how big was the TT size ?

TT size was 256 MB. I have also tried with interchanged colors, but only a few games. The hit rate was better. Maybe I'll come back to this issue later (testing code is saved).

Regarding your question, yes I have tried to implement this but I gave up because code was becoming too messy for my taste. In particular merging of an already exsisting entry with a second new cut-off move was a bit tricky, so I stopped implementation, actually I didn't spend a lot of time on it and if it turns out to have some _potential_ I think I could respin that and fully implement it.

The move store/replacement code is slightly expensive and tricky, but first experiments with two TT moves look promising. Unfortunately a move needs 17 bit instead of 16 bit. A 16 bit requirement would fit much better w.r.t the storage of several TT moves. The promotion piece type in the move encoding demands 3 bits, but 2 bits should be enough.

[mcostalba]

The move store/replacement code is slightly expensive and tricky, but first experiments with two TT moves look promising. Unfortunately a move needs 17 bit instead of 16 bit. A 16 bit requirement would fit much better w.r.t the storage of several TT moves. The promotion piece type in the move encoding demands 3 bits, but 2 bits should be enough.

How do you handle 2 TT moves for the same position with different depths ?

I have an idea on how to implement 2 TT moves per position in a clean way (although I don't know how efficient), anyhow I am interested to see your results before start coding...

[mcostalba]

A 16 bit requirement would fit much better w.r.t the storage of several TT moves. The promotion piece type in the move encoding demands 3 bits, but 2 bits should be enough.

In case you need to squeeze a move you can also consider to multiplex en-passant and castle flags. They should be unambigously detected given the 'from' square; if it is on rank 8/1 is a castle, otherwise an en-passant.

Of course code gets a bit messier, but if for your implementation you need an extra bit this could be viable.

[Ralph Stoesser]

The move store/replacement code is slightly expensive and tricky, but first experiments with two TT moves look promising. Unfortunately a move needs 17 bit instead of 16 bit. A 16 bit requirement would fit much better w.r.t the storage of several TT moves. The promotion piece type in the move encoding demands 3 bits, but 2 bits should be enough.

How do you handle 2 TT moves for the same position with different depths ?

I have an idea on how to implement 2 TT moves per position in a clean way (although I don't know how efficient), anyhow I am interested to see your results before start coding...

currently I think a scheme similar to history counters could be the goal. If a move is cut-off move at depth d add d*d to the move ordering score, but then we would need additional score counters, therefore this is probably too expensive...

Anyway, my first try looks like this

Code: Select all

void TranspositionTable::store(const Key posKey, Value v, ValueType t, Depth d, Move m, Value statV, Value kingD) {

  int c1, c2, c3;
  TTEntry *tte, *replace;
  uint32_t posKey32 = posKey >> 32; // Use the high 32 bits as key
  Move m2 = MOVE_NONE;

  tte = replace = first_entry(posKey);
  for (int i = 0; i < ClusterSize; i++, tte++)
  {
      if (!tte->key() || tte->key() == posKey32) // empty or overwrite old
      {
          if (m == MOVE_NONE)
          {
        	  // preserve any existing ttMoves
              m  = tte->move();
              m2 = tte->move2();
          }
          else if (m != tte->move())
          {
        	  // overwrite first ttMove?
        	  if (d >= tte->depth())
        	  {	  
        		  // yes, also store the previous first ttMove in slot #2
        		  m2 = tte->move();
        	  }
        	  else
        	  {	  
        		  // no, but store the new move in slot #2 in case it is empty
        		  if (tte->move2())
        			  m2 = tte->move2();
        		  else
        			  m2 = m;
        		  m = tte->move();
        	  }
          }

          tte->save(posKey32, v, t, d, m, m2, generation, statV, kingD);
          return;
      }

      if (i == 0)  // Replacing first entry is default and already set before entering for-loop
          continue;

      c1 = (replace->generation() == generation ?  2 : 0);
      c2 = (tte->generation() == generation ? -2 : 0);
      c3 = (tte->depth() < replace->depth() ?  1 : 0);

      if (c1 + c2 + c3 > 0)
          replace = tte;
  }
  replace->save(posKey32, v, t, d, m, m2, generation, statV, kingD);
}

Testing: 256 MB Hash, 1 Thread, tc 1+0
Score of test vs default: +131 -100 =280

so far ...

[mcostalba]

Anyway, my first try looks like this

I would use an "always replace strategy" as per killer moves:

Code: Select all

void TranspositionTable::store(const Key posKey, Value v, ValueType t, Depth d, Move m, Value statV, Value kingD) {

  Move m1, m2;
  int c1, c2, c3;
  TTEntry *tte, *replace;
  uint32_t posKey32 = posKey >> 32; // Use the high 32 bits as key

  m1 = m2 = MOVE_NONE;

  tte = replace = first_entry(posKey);

  for (int i = 0; i < ClusterSize; i++, tte++)
  {
      if (!tte->key() || tte->key() == posKey32) // empty or overwrite old
      {

          if (tte->key())
          {
                m1 = tte->move1();
                m2 = tte->move2();
          }

          if (m != MOVE_NONE && m != m1)
          {
                m2 = m1;
                m1 = m;
          }

          tte->save(posKey32, v, t, d, m1, m2, generation, statV, kingD);
          return;
      }

      if (i == 0)  // Replacing first entry is default and already set before entering for-loop
          continue;

      c1 = (replace->generation() == generation ?  2 : 0);
      c2 = (tte->generation() == generation ? -2 : 0);
      c3 = (tte->depth() < replace->depth() ?  1 : 0);

      if (c1 + c2 + c3 > 0)
          replace = tte;
  }
  replace->save(posKey32, v, t, d, m, m2, generation, statV, kingD);
}

[Ralph Stoesser]

Anyway, my first try looks like this

I would use an "always replace strategy" as per killer moves:

...

One thing probably worth to mention is the changed MovePicker implementation w.r.t. the second ttMove. In case I have a mateKiller I use it for the ttMoves[1].move, otherwise I use the second ttMove from TT.

No other code changes besides the obviously required ones.

Btw.: today I have bought a new fast computer only for testing purposes. No monitor, no keyboard, ssh and vi only. I have +14 Elo after 1000 1 minute games on my old machine. Already re-testing with 6 threads...

[mcostalba]

Anyway, my first try looks like this

I would use an "always replace strategy" as per killer moves:

...

One thing probably worth to mention is the changed MovePicker implementation w.r.t. the second ttMove. In case I have a mateKiller I use it for the ttMoves[1].move, otherwise I use the second ttMove from TT.

Yes, I figured out the same scheme because mateKiller is almost always MOVE_NONE.

Btw.: today I have bought a new fast computer only for testing purposes. No monitor, no keyboard, ssh and vi only. I have +14 Elo after 1000 1 minute games on my old machine. Already re-testing with 6 threads...

I have one QUAD without keyboard too ! It is a Windows 7 machine and I use the software keyboard provided by the OS even to log in. I just plug in a mouse to do all the preparatory stuff, then I start the match, switch off the monitor, unplug the mouse and that's all


BTW with your fast hardware I'd suggest something like 30"+0.1 (I guess you use cutechess-cli) and run 4K-5K games.