Speed vs ELO gain

[hgm]

What I mean with a node is generate the move, do the move on the board and undo the move on the board.
When I would just count the number of moves generated at the leaves of the tree it would be quite a lot faster than it is now.

It would be faster, but the nodes per second would be a lot slower. Moves are not nodes. What you count now is more accuratey described as moves/sec than as nodes/sec.

[Joost Buijs]

It would be faster, but the nodes per second would be a lot slower. Moves are not nodes. What you count now is more accuratey described as moves/sec than as nodes/sec.

Well I guess it is just a matter of opinion. This is the way I do it for almost 34 years now. I count each new position as a node. I don't know how others count their nodes. I don't even understand what your definition of a node exactly means.

[Joost Buijs]

I've programmed my engine "Sjakk" in Visual Basic (Express 2010). This is quite a slow language without support for inline statements, and I don't get any speedup when I compile it to 64 bit (not even on the move generation where I use bitboards). My engine on a 2.00 GHz computer does about 100 K N/S and the move generator (perft testing) does about 700 KN/S. This is very slow compared to the engines I compete with so I'm wondering if I should convert it to C. But I haven't programmed C in about 15 years so it will be a lot of hard work. But still if I could get a really large speed gain (some say 2-4 times) it might be worth it.

Is there a simple rule of thumb on how much ELO gain one gets for doubling the speed?

Regards
Jacob

Hi Jacob,

I have a very different opinion on this. In my tests Stockfish on 50% strength reached ~120elo less in both 1 minute and 3 minute games.

My other experiments with my weaker engine also confirmed that every 1 percent boost gave 1 elo strength increase at least.

Others should confirm this with their own tests.

Possibly you are right.
When I play a match with a huge number of games between my engine on 1 core and my engine on 6 cores - wich gives a speedup of 4 to 4.5 (time to depth) - I usually see a difference of around 180 ELO points.

[hgm]

Well I guess it is just a matter of opinion. This is the way I do it for almost 34 years now. I count each new position as a node. I don't know how others count their nodes. I don't even understand what your definition of a node exactly means.

But that was exactly my point. It is meaningless to compare nps numbers when you don't count nodes the same way. My definition of a node means a position where I do something, like move generation or evaluation. Positions were I do nothing do not occur in my engine. I don't even do the MakeMove to them. Why would I create a position where I am not going to do anything? That would be a pure waste of time.

[micron]

As a rule of thumb I always use 70 ELO for a speed doubling. It will also depend upon how fast your engine already is. Slow engines will probably gain more.

I have a very different opinion on this. In my tests Stockfish on 50% strength reached ~120elo less in both 1 minute and 3 minute games.
My other experiments with my weaker engine also confirmed that every 1 percent boost gave 1 elo strength increase at least.

My engine's strength gain, from doubling the time allocation, ranges from 140+ Elo in hyperbullet matches (e.g. 40/2 vs 40/1) down to ~100 Elo for leisurely matches (e.g. 40/64 vs 40/32 and 40/128 vs 40/64). Spandrel runs 5 MNps single-threaded from the starting position, and cannot be categorised as a 'slow engine'. (Weak perhaps, but not slow).

All engines show this effect: the Elo loss from halving the allocation is more or less constant at slow TCs, but as TC is made very fast the loss becomes disproportionate. At some point the engine starts losing on time.
Presumably this is due to fixed overheads such as string processing and i/o.

Robert P.

[Joost Buijs]

Well I guess it is just a matter of opinion. This is the way I do it for almost 34 years now. I count each new position as a node. I don't know how others count their nodes. I don't even understand what your definition of a node exactly means.

But that was exactly my point. It is meaningless to compare nps numbers when you don't count nodes the same way. My definition of a node means a position where I do something, like move generation or evaluation. Positions were I do nothing do not occur in my engine. I don't even do the MakeMove to them. Why would I create a position where I am not going to do anything? That would be a pure waste of time.

I mainly use perft to check the validity and speed of the move generator and the do_move/undo_move logic. It would be useless to do an evaluation or something else at the leaves.

With full evalution at each node I usually see something like 4000 kn/s on a single 3.3 gHz. core, thats about 7 times slower than without evaluation.

[hgm]

For instance my engine runs perft at ~28000 kN/S on a single 3.3 GHz. core. This is with full fledged do_move() and undo_move().

It depends on what you count as nodes. If a node is defined as a move generation your perft speed would probably be quite a bit lower.

Sure, perft should not use evaluation. But my original remark, quoted above, was intended to address your remark about the do/undo_move quoted above that.

The point I tried to make is that when you would _not_ do these, it would certainly not be justified to count the position as a 'node'. You have never even visited that position, it was not created on the internal board, and you certainly dd not do any work there. The only thing you did is generate the move to it in the parent position.

Now generating a move is usually not counted as a node. At least not while you are searching. About half the QS nodes generate several captures, subjects each of those captures to SEE, sees that they are all bad, and prunes them. You count all that for 1 QS node. Not for 5, if you had 5 bad captures.

So when you don't Make/UnMake the counted moves in a perft, the nps of that perft will be quite a bit lower than when you do make them. At least when you do count MakeMoves/sec rather than MoveGens/sec.

[Joost Buijs]

About half the QS nodes generate several captures, subjects each of those captures to SEE, sees that they are all bad, and prunes them. You count all that for 1 QS node. Not for 5, if you had 5 bad captures.

It looks like this is going to be an endless academic disscussion.

I don't know how you do it, but in my program I don't have to make a move before it gets pruned away by SEE or whatever. It would be rather inefficient to make a move and see that it's bad en then retract it.

Anyway I was talking about how I count nodes in my perft routine, this has nothing to do with a normal search.

[hgm]

I am not sure what we are discussing about. For one, you just exactly repeat what I said before: one does not make moves to positions where one doesn't want to do anything, which sugest we agree.

But the point was, do you count moves that you prune (without making them, obviously), as nodes? And if you don't do that in a Chess program, why would you do it in a perft? So I would think that in a perftprogram where you do _not_ make and unmake the counted moves, but only count them, you would not count the moves as nodes. And I am not even sure you disagree with that...

[Joost Buijs]

I am not sure what we are discussing about. For one, you just exactly repeat what I said before: one does not make moves to positions where one doesn't want to do anything, which sugest we agree.

But the point was, do you count moves that you prune (without making them, obviously), as nodes? And if you don't do that in a Chess program, why would you do it in a perft? So I would think that in a perftprogram where you do _not_ make and unmake the counted moves, but only count them, you would not count the moves as nodes. And I am not even sure you disagree with that...

The whole point is that I only count moves after making them on the board, moves that are pruned are not counted because they will never be made.

In perft I make the moves at the leaves because I normally run a procedure that checks everything for sanity, the board structure, hash keys, incremental evaluation etc. Thats why I'm counting the nodes in perft the same way as in my normal search. Just lazyness.

In the end we both mean the same thing.