Doubts with NebiyuChess and NebiyuCheckers in console mode.

[Ajedrecista]

Hello:

I have used NebiyuChess and NebiyuCheckers. I discovered them yesterday (version 1.42). At last I have a perft counter for checkers! And also perft estimates (MonteCarlo and UCT). Thank you very much, Mr. Shawul.

Here is the help displayed by NebiyuCheckers:

Code: Select all

help
. -- Send a search status update while analyzing.
? -- Move now.
accepted -- Response to the feature command.
analyze -- Enter analyze mode.
computer -- Inform Nebiyu that the opponent is also a computer chess engine.
d -- Debugging command to print the current board.
easy -- Turn off pondering.
eht -- Set evaluation hash table size in megabytes.
exit -- Leave analysis mode.
force -- Set the engine to play neither color ('force mode'). Stop clocks.
go -- Leave force mode and set the engine to play the color that is on move.
hard -- Turn on pondering (thinking on the opponent's time or permanent brain).
help -- Produce this listing of supported commands.
history -- Debugging command to print game history.
ht -- Set hash table size in megabytes.
level -- level <MPS> <BASE> <INC> {Set time controls}.
log -- 'log on' turns log on and 'log off' turns log off.
mirror -- Debugging command to mirror the current board.
movelist -- Debugging command to print all possible moves for the current board.

mt/cores <N>-- Set the number of parallel threads of execution.
mt/cores auto -- Automatically detect number of threads
name -- Tell nebiyu who you are.
new -- Reset the board to the standard chess starting position.
nopost -- do not show thinking.
otim -- otim N {Set a clock that belongs to the opponent in centiseconds.}
paranoid -- use paranoid alpha-beta search for multi-player games
perft -- perft <depth> performs a move generation count to <depth>.
      -- perfmc <depth> monte-carlo perft
      -- perfuct <depth> UCT perft
      -- perfbf <depth> branching factor perft
pht -- Set pawn hash table size in megabytes.
post -- Show thinking.
protover -- protover <N> {Command sent immediately after the 'xboard' command.}
pvstyle -- syle for pv output.
quit -- The chess engine should immediately exit.
random -- This GnuChess command is ignored.
rejected -- Response to the feature command.
remove -- The user asks to back up full move.
resign <RESIGN_VALUE> -- Sets resign value.
result -- Sets the game result.
runeval -- runeval <epdfile> runs the evalution function on EPD file <epdfile>.
runsearch -- runsearch <epdfile> runs the engine search on EPD file <epdfile>.
score -- score runs the evaluation function on the current position.
sd -- sd <DEPTH> {The engine should limit its thinking to <DEPTH> ply.}
setboard -- setboard <FEN> is used to set up FEN position <FEN>.
selfplay -- run a self play game
seed -- set random number seed to time(NULL)
st -- st <TIME> {Set time controls search time in seconds.}
time -- time <N> {Set a clock that belongs to the engine in centiseconds.}
undo -- The user asks to back up one half move.
xboard -- Request xboard mode (the default).

Some perft parameters are wrong when they are displayed in help (NebiyuChess has the same issue):

Code: Select all

<000000087766> perfmc 5
Illegal move: perfmc
Illegal move: 5
<000000092063> perftmc 5
1.{7.339500e+003} 7.339500e+003 +- 0.000000e+000 963 0.00 sec
2.{7.439500e+003} 7.389500e+003 +- 3.535534e+001 1926 0.00 sec
3.{7.519000e+003} 7.432667e+003 +- 4.240043e+001 2889 0.00 sec

[...]

Correct are perftmc (instead of perfmc), perftuct (instead of perfuct) and perftbf (instead of perfbf). The log file is very useful.

In NebiyuCheckers, I noticed that moves are in chess-like notation instead of checkers notation (i.e. c3-d4 instead of the usual 11-15). Nothing wrong with this.

When I set FEN strings (using P for white checkers, p for black checker, Q for white king and q for black king), it is like white does the first move, when in checkers first move is done by black/red (at least in English draughts / American checkers). Some examples with an Edinburgh opening (9-13 or g3h4):

Code: Select all

setboard 1p1p1p1p/p1p1p1p1/1p1p1p1p/8/7P/P1P1P3/1P1P1P1P/P1P1P1P1 b - - 0 1
perft 1
h6g5 1
f6e5 1
f6g5 1
d6c5 1
d6e5 1
b6a5 1
b6c5 1
nodes 7
time 0.01 sec
setboard 1p1p1p1p/p1p1p1p1/1p1p1p1p/8/7P/P1P1P3/1P1P1P1P/P1P1P1P1 w - - 0 1
perft 1
h4g5 1
e3f4 1
e3d4 1
c3d4 1
c3b4 1
a3b4 1
h2g3 1
f2g3 1
nodes 8
time 0.00 sec
setboard 1P1P1P1P/P1P1P1P1/1P1P1P1P/8/7p/p1p1p3/1p1p1p1p/p1p1p1p1 w - - 0 1
perft 1
nodes 0
time 0.00 sec
setboard 1P1P1P1P/P1P1P1P1/1P1P1P1P/8/7p/p1p1p3/1p1p1p1p/p1p1p1p1 b - - 0 1
perft 1
h4g3 1
nodes 1
time 0.00 sec

So, mentally changing colours in the FEN string is enough and perft works fine (also perft estimates)!

I want to post a MonteCarlo Perft(25) for the starting position of draughts, ran in an Intel Pentium D930 (3 GHz):

<000000006328> perftmc 25
1.{8.054163e+016} 8.054163e+016 +- 0.000000e+000 13043 0.00 sec
2.{1.031917e+017} 9.186665e+016 +- 8.007997e+015 26086 0.00 sec
3.{8.347236e+016} 8.906855e+016 +- 5.806970e+015 39129 0.00 sec
4.{9.055764e+017} 2.931955e+017 +- 1.768328e+017 52172 0.00 sec


[...]


38330.{7.299780e+016} 1.614287e+017 +- 1.189754e+015 499938099 0.00 sec
38331.{2.952520e+017} 1.614322e+017 +- 1.189728e+015 499951142 0.00 sec
38332.{7.008499e+016} 1.614298e+017 +- 1.189700e+015 499964185 0.00 sec
38333.{1.880235e+017} 1.614305e+017 +- 1.189669e+015 499977228 0.00 sec
38334.{7.498186e+016} 1.614282e+017 +- 1.189640e+015 499990271 0.00 sec
38335.{1.002307e+017} 1.614266e+017 +- 1.189610e+015 500003314 0.00 sec
time 2492.66 sec

It looks like the estimate automatically stops when ~ 5e+8 nodes are reached. The uncertainties (± ...) are calculated for a given confidence level... which? 95% (~ ± 1.96-sigma), ~ 95.45% (± 2-sigma)...

Some months ago, I did a clumsy estimate for Perft(25) of starting position of checkers, and I remind my post:

http://talkchess.com/forum/viewtopic.ph ... ht=#417256

1.6077835e+17 < Perft(25) < 1.6234776e+17

Perft(25) ~ 1.6156306e+17 ± 0.4857%

My estimated interval (which has a very big amplitude width) fits in this MonteCarlo estimate... good! I do not understand the numbers between {}... what do they mean?

Finally, I do not understand the majority of numbers in UCT perft. Here is an example with UCT Perft(6) for the initial position of standard chess:

<000000007968> perftuct 6
[st = 5557ms, mt = 29250ms , moves_left 10]
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1
[Tree not found]


(More data).


1.190648e+008 +- 9.225598e+003 1.190561e+008 +- 9.214804e+003 55380537 6.866e+007
1.190651e+008 +- 9.219804e+003 1.190557e+008 +- 9.208945e+003 55450537 6.866e+007
1.190655e+008 +- 9.214078e+003 1.190552e+008 +- 9.203171e+003 55520536 6.866e+007
1.190650e+008 +- 9.208346e+003 1.190549e+008 +- 9.197386e+003 55590536 6.866e+007
1.190646e+008 +- 9.202741e+003 1.190550e+008 +- 9.191470e+003 55660535 6.866e+007
1.190647e+008 +- 9.197141e+003 1.190548e+008 +- 9.185568e+003 55730532 6.866e+007
1.190646e+008 +- 9.191375e+003 1.190551e+008 +- 9.179980e+003 55800529 6.866e+007
1.190648e+008 +- 9.185530e+003 1.190548e+008 +- 9.174199e+003 55870529 6.866e+007
1.190648e+008 +- 9.179808e+003 1.190550e+008 +- 9.168212e+003 55940529 6.866e+007
1.190651e+008 +- 9.174221e+003 1.190553e+008 +- 9.162589e+003 56010527 6.866e+007

=========Cycles 4000000 : Total nodes 421 ==========
0 h2h3 4.459133e+006 +- 1.948821e+003 149980 | 4.463928e+006 +- 1.820180e+003 149980 | 20
0 h2h4 5.387655e+006 +- 1.873851e+003 181061 | 5.389036e+006 +- 1.899426e+003 181061 | 20
0 g2g3 5.346035e+006 +- 1.933404e+003 179517 | 5.343088e+006 +- 1.927081e+003 179517 | 20
0 g2g4 5.242365e+006 +- 1.787448e+003 176079 | 5.240746e+006 +- 1.818714e+003 176079 | 20
0 f2f3 4.403606e+006 +- 1.707394e+003 147909 | 4.402329e+006 +- 1.682486e+003 147909 | 20
0 f2f4 4.893502e+006 +- 1.891714e+003 164345 | 4.891509e+006 +- 1.886892e+003 164345 | 20
0 e2e3 9.727892e+006 +- 2.815736e+003 326848 | 9.728215e+006 +- 2.826694e+003 326848 | 20
0 e2e4 9.777382e+006 +- 2.824497e+003 328091 | 9.765203e+006 +- 2.800809e+003 328091 | 20
0 d2d3 8.075123e+006 +- 1.995088e+003 271190 | 8.071605e+006 +- 2.000694e+003 271190 | 20
0 d2d4 8.873434e+006 +- 2.469339e+003 298343 | 8.879802e+006 +- 2.479412e+003 298343 | 20
0 c2c3 5.415792e+006 +- 2.180262e+003 182088 | 5.419609e+006 +- 2.175782e+003 182088 | 20
0 c2c4 5.865389e+006 +- 2.142202e+003 197029 | 5.864322e+006 +- 2.142931e+003 197029 | 20
0 b2b3 5.313322e+006 +- 1.927097e+003 178320 | 5.307475e+006 +- 1.920451e+003 178320 | 20
0 b2b4 5.293518e+006 +- 1.894186e+003 177904 | 5.295085e+006 +- 1.940811e+003 177904 | 20
0 a2a3 4.462927e+006 +- 1.901898e+003 149943 | 4.462843e+006 +- 1.819508e+003 149943 | 20
0 a2a4 5.363022e+006 +- 1.869023e+003 180229 | 5.364293e+006 +- 1.883295e+003 180229 | 20
0 g1h3 4.876842e+006 +- 1.764898e+003 163945 | 4.879593e+006 +- 1.772126e+003 163945 | 20
0 g1f3 5.724630e+006 +- 1.676027e+003 192357 | 5.725267e+006 +- 1.715696e+003 192357 | 20
0 b1c3 5.707362e+006 +- 2.022007e+003 191690 | 5.705414e+006 +- 2.062977e+003 191690 | 20
0 b1a3 4.856202e+006 +- 1.918080e+003 163151 | 4.855982e+006 +- 1.912147e+003 163151 | 20
Perft 1.190651e+008 +- 9.174221e+003 1.190553e+008 +- 9.162589e+003

The real value is well-known (119,060,324). There are two intervals in each line (why two intervals?), for example in the last line:

Code: Select all

Perft 1.190651e+008 +- 9.174221e+003

Code: Select all

1.190553e+008 +- 9.162589e+003

I do not know which of these two intervals is the UCT perft estimate (and I also do not know which is the confidence level, again). At least I understand 6.866e+007 ~ sd·sqrt(n) ~ (9.174221e+003)·sqrt(56010527) that comes from the first interval (and not from the second one). There are more unknown things to me:

Code: Select all

[st = 5557ms, mt = 29250ms , moves_left 10]

st -- st <TIME> {Set time controls search time in seconds.} In help file. But I do not know if here means the same as in help file; the others are an enigma for me.

Total nodes 421. Could be Perft(0) + Perft(1) + Perft(2) = 1 + 20 + 400 = 421? It could be, in view of other example:

Code: Select all

setboard 7k/8/8/8/8/8/8/7K w - - 0 1
perft 2
h1g2  3
h1h2  3
h1g1  3
nodes 9
time 0.02 sec

I expected here Perft(0) + Perft(1) + Perft(2) = 1 + 3 + 9 = 13; running an UCT Perft(20) of this exact position:

<000000023859> setboard 7k/8/8/8/8/8/8/7K w - - 0 1
<000000032078> perftuct 20
[st = 5557ms, mt = 29250ms , moves_left 10]
7k/8/8/8/8/8/8/7K w - - 0 1
[Tree not found]
0 h1g2 2.464243e+015 +- 6.067268e+014 1 | 2.154377e+015 +- 5.677411e+014 1 | 3
0 h1h2 8.379384e+014 +- 1.513504e+014 1 | 9.171541e+014 +- 2.707289e+014 1 | 3
0 h1g1 8.869364e+014 +- 2.418442e+014 1 | 9.216036e+014 +- 1.750100e+014 1 | 3
Perft 4.189118e+015 +- 6.704573e+014 3.993135e+015 +- 6.528802e+014
=========Total nodes 13 ===================

3.158293e+015 +- 5.260321e+013 3.105139e+015 +- 5.052450e+013 351194 3.117e+016
3.142182e+015 +- 3.676298e+013 3.079028e+015 +- 3.532297e+013 701194 3.078e+016
3.138487e+015 +- 3.047089e+013 3.082393e+015 +- 2.922391e+013 1051194 3.124e+016
3.153693e+015 +- 2.680430e+013 3.095009e+015 +- 2.551514e+013 1401194 3.173e+016
3.154599e+015 +- 2.386489e+013 3.120638e+015 +- 2.323009e+013 1751194 3.158e+016
3.143931e+015 +- 2.162428e+013 3.125989e+015 +- 2.125007e+013 2101194 3.135e+016
3.145370e+015 +- 1.997270e+013 3.125999e+015 +- 1.961709e+013 2451194 3.127e+016
3.148596e+015 +- 1.871192e+013 3.121949e+015 +- 1.830659e+013 2801194 3.132e+016
3.149730e+015 +- 1.762100e+013 3.122079e+015 +- 1.729815e+013 3151194 3.128e+016
3.151179e+015 +- 1.670251e+013 3.124296e+015 +- 1.644107e+013 3501194 3.125e+016

=========Cycles 50000 : Total nodes 13 ==========
0 h1g2 1.481451e+015 +- 1.125667e+013 23746 | 1.483835e+015 +- 1.121598e+013 23746 | 3
0 h1h2 7.697804e+014 +- 8.335128e+012 12351 | 7.716894e+014 +- 8.442337e+012 12351 | 3
0 h1g1 8.999482e+014 +- 9.098728e+012 13905 | 8.687715e+014 +- 8.557885e+012 13905 | 3
Perft 3.151179e+015 +- 1.670251e+013 3.124296e+015 +- 1.644107e+013
3.146157e+015 +- 1.591470e+013 3.125776e+015 +- 1.564924e+013 3851194 3.123e+016
3.145296e+015 +- 1.521424e+013 3.135627e+015 +- 1.506700e+013 4201194 3.118e+016
3.141297e+015 +- 1.458969e+013 3.137663e+015 +- 1.450638e+013 4551194 3.112e+016
3.145073e+015 +- 1.408547e+013 3.135334e+015 +- 1.401915e+013 4901194 3.118e+016
3.144242e+015 +- 1.363878e+013 3.142397e+015 +- 1.358594e+013 5251194 3.125e+016
3.144388e+015 +- 1.320024e+013 3.139563e+015 +- 1.313844e+013 5601194 3.124e+016
3.150654e+015 +- 1.284723e+013 3.141795e+015 +- 1.275360e+013 5951194 3.134e+016
3.151045e+015 +- 1.249253e+013 3.139484e+015 +- 1.237763e+013 6301194 3.136e+016
3.149835e+015 +- 1.215076e+013 3.137370e+015 +- 1.203394e+013 6651194 3.134e+016
3.149406e+015 +- 1.184797e+013 3.141212e+015 +- 1.174264e+013 7001194 3.135e+016

=========Cycles 100000 : Total nodes 13 ==========
0 h1g2 1.477891e+015 +- 7.939858e+012 47237 | 1.483757e+015 +- 7.940019e+012 47237 | 3
0 h1h2 7.730371e+014 +- 6.022072e+012 24548 | 7.711008e+014 +- 6.049451e+012 24548 | 3
0 h1g1 8.984776e+014 +- 6.408404e+012 28217 | 8.863542e+014 +- 6.184646e+012 28217 | 3
Perft 3.149406e+015 +- 1.184797e+013 3.141212e+015 +- 1.174264e+013

Total nodes 13 exactly. I also do not understand cycles (sorry for the big amount of questions). Returning to UCT Perft(6) and copying one random line:

Code: Select all

0 c2c4   5.865389e+006 +- 2.142202e+003    197029 | 5.864322e+006 +- 2.142931e+003    197029 |        20

The last number (20) seems Perft(1) of 1.- c4 (I mean, 20 possible moves for black now). The number 197029 seems the number of cycles that should correspond to 1.- c4 in Perft(6): real value is 5,866,666 out of 119,060,324 (~ 4.9275%), while here is 197,029 out of 4,000,019 (if I added well all this group of twenty numbers), which is ~ 4.9257%; the same for the rest of moves. It looks like NebiyuChess does its estimates and then divides the perft into all possible Perft(1) moves, applying a weight for each move. I do not really know how UCT Perft works and it will be a waste of time if somebody tries to explain it to me, so please refuse that attempt. I only want to know what means each number. Thanks in advance and sorry for the long output of this post.

Regards from Spain.

Ajedrecista.

[hgm]

In WB protocol the keyword 'white' is used to designate the side that moves first (for historic reasons; 'sente' would of course have been a better choice).

In cases where the user thing this is annoying, he can configure the user interface to rename the colors, by supplying a translation file (e.g. checkerese.lng) redefining the strings "White:" and "Black:" to take care of the clock display, and use the option -colorNickNames "xy" with xy the substitutes for wb. This only works for input; when WinBoard outputs FENs, it will always use the standard. The option is meant as an aid to understand non-compliant FENs (e.g. because they are in another language).

[Daniel Shawul]

Hi Jesus
I am glad you find it useful.

Correct are perftmc (instead of perfmc), perftuct (instead of perfuct) and perftbf (instead of perfbf). The log file is very useful.

Thanks. Will fix that in next version. I know documentation is lacking and also I haven't touched that code since we stopped searching for perft estimators.

white does the first move

Winboard assumes white to move first for any game (maybe PGN problem ??) but you can set black to move to Nebiyu as you correctly figured out.

It looks like the estimate automatically stops when 5e+8 nodes are searched

You are right. That was the stopping criteria for our comparisons 500 million nodes. You can try to set it through "nodes" parameter but I haven't tested it! I do not remeber a lot of stuff that i did now.

Finally, I do not understand the majority of numbers in UCT perft. Here is an example with UCT Perft(6) for the initial position of stand

Unlike other people i compute two perfts in parallel so what you see is.
mean1 +- sd1 mean2 +- sd2 nodes "Michel's formula of work"
You should only pay attention to the first mean, the second one is used for guiding and it can give pretty bad results.
One cycle constitutes moving from the root node down to a leaf node.
Total nodes is the number of nodes stored in memory. For example when its 421 i.e perft(2) it means the first 2 plies are in memory.
The result for each of the root moves is displayed once in a while.

The last column in the individual move results (i.e 20) the size of the subtree stored in memory for each root move. I have used that for perft estimation in the past.

I guess that is it. If you have more questions, I would be glad to help.
regards,
Daniel

[Ajedrecista]

Hello!

@Muller: Thank you very much for your fast answer. All the stuff you had written is very interesting and useful. I am not annoyed with the fact that 'white' is considered the first-mover (just as the chess-like notation for the game of checkers); I was simply curious of that fact. It is good to know why it was considered in that way. My solution of mentally change colours is fine for me!

I did not reply inmediately because I wanted to wait for Shawul's reply and then post a reply for both of you.

@Daniel: Thank you very much for your answer. It is good that this tiny bug on documentation will be fixed in the next version. As I have told to Mr. Muller: it is the same for me if white or black is considered the first-mover, I was simply curious. Again, my solution works fine for me!

I think that I will not try nodes parameter: 5e+8 nodes seem enough for me. Thanks for your suggestion.

Regarding UCT, it is surprising to me the fact of computing two parallel perfts, but I have understood: (the first mean) ± (the first standard deviation) is the estimate to which I have to be aware.

Thanks for your definition of cycle. My guess was correct regarding Total nodes. I wonder if the number of plies stored in memory can be raised from 2 to 3 or 4, for example. Your info is very valuable. Thanks again.

Regards from Spain.

Ajedrecista.

[Daniel Shawul]

Thanks for your definition of cycle. My guess was correct regarding Total nodes. I wonder if the number of plies stored in memory can be raised from 2 to 3 or 4, for example. Your info is very valuable. Thanks again.

Yes that was a fixed number in the source code that I forgot to change. Well at first it was actually automatically calculated as it should be in a real UCT perfter but the performance was inferior. Generating the tree at startup (N plies) deep turned out to be better for the starting position of chess.
I will make these changes in the next version.

regards,
Daniel