reasonable nps

[AndrewShort]

I'm trying to determine what a reasonable nps is. In other words, I want to make sure my engine is doing what it should be doing, with regard to speed, at least on my hardware.

Perhaps there is a rule of thumb out there - you should be getting x nps for each GHz, or something like that..

My machine is a 2 year old Sony VAIO laptop, running 32-bit Windows Vista, Intel Core 2 CPU, T5600, 1.83 Ghz, 2 GB RAM.

My engine is written in VB.NET, which is a compiled language. It's not the old interpreted Basic. It's compiled.

I use a 128 MB hash, and I use 1d array with "border squares" for the board (array of piece indeces to a piece list), pseudo-legal move generation, PVS, null-move, killers, history heuristic, ID with aspiration windows, IID, MVV/LVA, check extension, piece square tables, and a rather simple eval function. I presently have no SEE, and no futiliity pruning of any kind, no lazy eval, no LMR. Also, I fail hard, not soft. No pondering.

Also, my engine runs on a separate thread from my own GUI. But, the engine itself does not split the tree among multiple threads.

If I include hash hits in the node count (which most of you seem to be doing), I am getting roughly 450,000 to 650,000 nps, depending upon the position, but more usually around 500,000 nps. In that famous Fine's Chess endings high-hash position with all the locked pawns, I solve a depth 26 search (includes depth 1-25, too) in less than half a second (.497 sec), and the nps in that startup position is just over 500,000 nps.

Given the parameters of my hardware and engine, does that seem reasonable? Or have I just publicly humiliated myself?

[bob]

I'm trying to determine what a reasonable nps is. In other words, I want to make sure my engine is doing what it should be doing, with regard to speed, at least on my hardware.

Perhaps there is a rule of thumb out there - you should be getting x nps for each GHz, or something like that..

My machine is a 2 year old Sony VAIO laptop, running 32-bit Windows Vista, Intel Core 2 CPU, T5600, 1.83 Ghz, 2 GB RAM.

My engine is written in VB.NET, which is a compiled language. It's not the old interpreted Basic. It's compiled.

I use a 128 MB hash, and I use 1d array with "border squares" for the board (array of piece indeces to a piece list), pseudo-legal move generation, PVS, null-move, killers, history heuristic, ID with aspiration windows, IID, MVV/LVA, check extension, piece square tables, and a rather simple eval function. I presently have no SEE, and no futiliity pruning of any kind, no lazy eval, no LMR. Also, I fail hard, not soft. No pondering.

Also, my engine runs on a separate thread from my own GUI. But, the engine itself does not split the tree among multiple threads.

If I include hash hits in the node count (which most of you seem to be doing), I am getting roughly 450,000 to 650,000 nps, depending upon the position, but more usually around 500,000 nps. In that famous Fine's Chess endings high-hash position with all the locked pawns, I solve a depth 26 search (includes depth 1-25, too) in less than half a second (.497 sec), and the nps in that startup position is just over 500,000 nps.

Given the parameters of my hardware and engine, does that seem reasonable? Or have I just publicly humiliated myself?

It is very difficult to compare. Crafty, for example, on a single core-2 CPU at 2.0ghz runs about 2.5M nodes per second, but it can go up significantly in the right kinds of positions. It depends on how clever you were with data structures, how much (or how little) memory traffic you produce, how complex or simple your evaluation is, etc...

[Michel]

This seems reasonable. I am getting 550000k with Toga II 14b5c on a 1.6MHz Pentium M (in the middle game).

Of course you cannot really compare nps between engines. Rybka has a very low nps and yet is very strong.

[YL84]

Hi,
my engine is also written in VB.net, has no hashtable, and only get 100000 to 150000 nps on a core duo 2, 2Ghz, with the full (though poor) eval.
You have done a very good job
Just to let you know,
Yves

[AndrewShort]

Hi,
my engine is also written in VB.net, has no hashtable, and only get 100000 to 150000 nps on a core duo 2, 2Ghz, with the full (though poor) eval.
You have done a very good job
Just to let you know,
Yves

Merci Yves, et bonne chance

[bob]

This seems reasonable. I am getting 550000k with Toga II 14b5c on a 1.6MHz Pentium M (in the middle game).

Of course you cannot really compare nps between engines. Rybka has a very low nps and yet is very strong.

550 billion nodes per second seems very fast.

[Tommy]

Hey, that's interesting. I appear to be at a similar development stage with my new chess engine which happens to written in FreeBASIC. I don't have null move implemented yet as you do, but I do have a SEE function.

I was getting 440,000 NPS from the opening position with SEE enabled, and 520,000 NPS without SEE from an Athlon XP 3200+ (2.2Ghz). It appears to me you are getting reasonably good performance for an engine written in BASIC. My evaluation function only has piece square and some code to recognise drawn positions.

Can you post that position you talk about with the locked pawns. I'm afraid I'm not familiar with it and would like to try it out on my engine.

There are a few other chess engines written in BASIC you could download and make comparisons if you want. There is Minimax and Kanguruh written in PowerBASIC, Larsen VB, Gringo and Pentagon written in VB6. I think Milady was written in VB.Net. Probably a few others that I cannot thing of right now.

Cheers,
Tom.

[kranium]

Dr. Hyatt,

Is there (as far as you know) a direct correlation (or any relationship) betwen NPS and strength?

I'm often concerned if a change I make lowers NPS.

I know that some engines are strong despite a low NPS.
I also know that other engines acheive an extraordinarily high NPS (like Crafty)

I hate to see the engine lose node count...
Am I unecessarily concerned?

I'm asking you because i realize that you likely have an extremely keen insight into this matter..

Many thanks,
Norm

[AndrewShort]

...
Can you post that position you talk about with the locked pawns. I'm afraid I'm not familiar with it and would like to try it out on my engine.
...

Set your engine to White, and set side to move to White.
White King on a1
White pawns on a4, d4, d5, f4
Black King on a7
Black pawns on a5, d6, f5

A depth 26 search or less should show Ka1-b1 as White's move which wins a pawn (with iterative deepening, that means you've searched depth 1-26). And it should take less than a second. It takes me less than half a second. I think Crafty does it in something like 5 milliseconds, which is incredible. If your engine has null move, make sure null move is not allowed in such a position, otherwise you might not solve it in 26 ply or less.

Frankly I'm not sure if Ka1-b1 is the only proper move to win a pawn. I don't have multi-PV, so I wouldn't know and I'm too lazy to figure that one out.

[Dann Corbit]

Dr. Hyatt,

Is there (as far as you know) a direct correlation (or any relationship) betwen NPS and strength?

I'm often concerned if a change I make lowers NPS.

I know that some engines are strong despite a low NPS.
I also know that other engines acheive an extraordinarily high NPS (like Crafty)

I hate to see the engine lose node count...
Am I unecessarily concerned?

I'm asking you because i realize that you likely have an extremely keen insight into this matter..

Many thanks,
Norm

Branching factor is far more important than NPS.

But you should not unnecessarily have a slow move generator or evaluation.

If you want to improve strength, there is a bigger win in branching factor than in speeding up NPS. But there is still something to gain from how fast you can generate moves and evaluate them.