What is the Ideal Output for Understanding a Chess Engine?

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
rfadden

What is the Ideal Output for Understanding a Chess Engine?

Post by rfadden » Mon Apr 07, 2008 2:09 pm

In another thread we were discussing the various ways of tallying a Node Count. Here I would like to raise this topic as a separate question.

I realize this topic may come up repeatedly but I have not been reading CCC for a long time and I am hoping that people find this topic worth chatting about here.

So there was mention of counting how many times one enters the recursive search routines in a chess program, and there was some mention of not including Qu search and also of course the statement that Qu search counts must be included.

I am also wondering: if we tend to only look at one number then why not simply look at the number of calls to Eval during search?

I'd really like to hear the thoughts of various people here. I'd like to hear the thoughts of experienced folks on this topic of tallying calls to Eval.
I'm further wondering if we could take these reasons and turn them into an idea something like: Tally two entities, one is calls to eval and the other is a tally of calls to the recursive search functions. All kinds of thoughts pop up like possible interest in comparing these two numbers.

Wouldn't it be typical for someone to reply here that "This has all been discussed previously, and just about everyone agreed the answer is X." Well if someone will say that, the I would like to know what everyone has agreed to. I want to know about "X" the agreed upon technique for Node Count or the concensus on what this should be.

If people are not agreeing on X, then I think the general discussion on figuring out the best technique for X would be educational.

Thanks,
Rick

User avatar
Bill Rogers
Posts: 3562
Joined: Thu Mar 09, 2006 2:54 am
Location: San Jose, California

Re: What is the Ideal Output for Understanding a Chess Engin

Post by Bill Rogers » Mon Apr 07, 2008 3:41 pm

One possible problem with you idea is that during a 10 ply search 10 nodes are generated before on evaluation is done. The Eval is done at the top of the search.

Gerd Isenberg
Posts: 2127
Joined: Wed Mar 08, 2006 7:47 pm
Location: Hattingen, Germany

Re: What is the Ideal Output for Understanding a Chess Engin

Post by Gerd Isenberg » Mon Apr 07, 2008 5:45 pm

Usually a node is entered with a call of a recursive search/qsearch-routine - or equivalently a jump to an entry point inside an iterative search. Cycles per node is a fine measure, or taking energy consumption of certain processor systems into account, nano-joule per node and cycle ;-)

For an inner eye following proposal: absolute numbers and some ratios, differentiated by ply (let say 0==root ...50) and thread/process of the search, all numbers distinct after each search iteration as well as aggregated for effective branching factors:

Code: Select all

nodes, 
pv-nodes, 
zerowindow-nodes, 
re-searches, 
qnodes, 
horizon-nodes, 
frontier-nodes, 
pre-frontier-nodes, 
pre-pre-frontier-nodes, 
leaf-nodes, 
nullmove-cuts, 
cut-nodes,
all-nodes, 
cuts-onMove[moveNr 1..N], 
cuts-onHashMove, 
cuts-onKiller[killerIndex],
expected-cut-nodes, 
expected-cut-nodes-becomes-all, 
expected-all-nodes, 
expected-all-nodes-becomes-cut, 
fail-high-on-hashprobe,
fail-low-on-hashprobe,
exacthit-on-hashprobe,
nodes-below-nullmove,
nodes-in-IID,
not to mention the number of various (fractional) extensions, reductions or prunings, mates, stalemates, repetitions, recognizer- and egtb-hits etc.. Calls to lazy-eval versus eval. Number of stores, probes and hits and sufficient hits of various caches since eval may be cached in main transposition table as well as in dedicated eval hash.

Probably more interesting for the programmer than for the average user, anyway we only need to log it, and to visualize it with a neat configurable online chart application for the inner eye. Chess-System Tal was quite nice with that respect. Unfortunately updating all those counters will take some time...

Cheers,
Gerd

Dann Corbit
Posts: 10130
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: What is the Ideal Output for Understanding a Chess Engin

Post by Dann Corbit » Mon Apr 07, 2008 7:13 pm

The ideal way to understand a chess engine is to look at the moves that it makes.

jwes
Posts: 778
Joined: Sat Jul 01, 2006 5:11 am

Re: What is the Ideal Output for Understanding a Chess Engin

Post by jwes » Mon Apr 07, 2008 8:26 pm

f5

tmokonen
Posts: 1033
Joined: Sun Mar 12, 2006 5:46 pm
Location: Vancouver

Re: What is the Ideal Output for Understanding a Chess Engin

Post by tmokonen » Mon Apr 07, 2008 11:01 pm

jwes wrote:f5
Is this the chessic equivalent of the number 42?

Tony

User avatar
Ovyron
Posts: 2750
Joined: Tue Jul 03, 2007 2:30 am

Re: What is the Ideal Output for Understanding a Chess Engin

Post by Ovyron » Mon Apr 07, 2008 11:07 pm

tmokonen wrote:
jwes wrote:f5
Is this the chessic equivalent of the number 42?

Tony
As in:

"42 is the answer but nobody knows the question"

"f5 is the best move, but nobody knows the position"

:?:

tmokonen
Posts: 1033
Joined: Sun Mar 12, 2006 5:46 pm
Location: Vancouver

Re: What is the Ideal Output for Understanding a Chess Engin

Post by tmokonen » Mon Apr 07, 2008 11:10 pm

Ovyron wrote:
tmokonen wrote:
jwes wrote:f5
Is this the chessic equivalent of the number 42?

Tony
As in:

"42 is the answer but nobody knows the question"

"f5 is the best move, but nobody knows the position"

:?:
I think you've just figured out the answer to Life, the Universe, Chess and Everything :)

bob
Posts: 20580
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: What is the Ideal Output for Understanding a Chess Engin

Post by bob » Tue Apr 08, 2008 1:05 am

rfadden wrote:In another thread we were discussing the various ways of tallying a Node Count. Here I would like to raise this topic as a separate question.

I realize this topic may come up repeatedly but I have not been reading CCC for a long time and I am hoping that people find this topic worth chatting about here.

So there was mention of counting how many times one enters the recursive search routines in a chess program, and there was some mention of not including Qu search and also of course the statement that Qu search counts must be included.

I am also wondering: if we tend to only look at one number then why not simply look at the number of calls to Eval during search?

I'd really like to hear the thoughts of various people here. I'd like to hear the thoughts of experienced folks on this topic of tallying calls to Eval.
I'm further wondering if we could take these reasons and turn them into an idea something like: Tally two entities, one is calls to eval and the other is a tally of calls to the recursive search functions. All kinds of thoughts pop up like possible interest in comparing these two numbers.

Wouldn't it be typical for someone to reply here that "This has all been discussed previously, and just about everyone agreed the answer is X." Well if someone will say that, the I would like to know what everyone has agreed to. I want to know about "X" the agreed upon technique for Node Count or the concensus on what this should be.

If people are not agreeing on X, then I think the general discussion on figuring out the best technique for X would be educational.

Thanks,
Rick
Calls to Evaluate() will almost exactly match the number of calls to Quiesce(), since Evaluate() is called at the top of Quiesce() to set the "stand-pat" score before searching any moves there...

My only comment is that nodes ought to be reported consistently, and in the literature, a "new position" is a new node. You could count calls to MakeMove() since that leads to a new position. You could count calls to Search() / Quiesce() which ought to be the same number. Anything else (Evaluate() calls as an example) will be an example of a number that is less useful since it omits what might be large parts of the tree. For example, chessmaster is very selective, which means not counting the normal search nodes would greatly reduce the total nodes searched number, making it even harder to compere nodes between programs.

I would always suggest sticking with the traditional definition of "node" unless your name happens to be Donald Knuth, Claude Shannon, etc... Then one might have the "reputation" necessary to re-define an accepted term.

If a definition becomes vague, it becomes useless.

User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 6:43 pm

Node counting

Post by sje » Tue Apr 08, 2008 1:29 am

For its traditional A/B search, Symbolic has a simple convention for node counting: the counter is incremented for each time a legal move is made on the analysis board. It is guaranteed that some sort of score will result from this although it may not result in a full evaluation.

The only non legal moves tried are those pulled from a table that were legal in some other position, but not in the current one. These are not counted. Nearly all moves are legal as the move generators produce only legal moves.

For its cognitive search, Symbolic keeps the whole tree in memory and so the node count is simply the total number of nodes in the tree graph. A purist might consider this an undercount as a given node may be visited more than once for different reasons.

Post Reply