uci protocol

[ericlangedijk]

I am writing my first uci engine.
I can find examples of the commands but no examples of an actual game going on.
What messages are sent when a game is going on?
Only moves?
Only FEN's?
Or a complete FEN including all moves? (which would be quite insane and verbose).

[op12no2]

Here's an example of the first few moves. As well as the chit-chat commands, the engine needs to be able to react to a position command with a FEN and move list - and a go command. The engine responds to the go command with info strings and finally a bestmove. On ucinewgame the engine does once-per-game init like reseting the TT.

Code: Select all

261 >lozza.js(0): uci
262 >lozzacand.js(1): uci
292 <lozza.js(0): id name Lozza 8
292 <lozza.js(0): id author Colin Jenkins
292 <lozza.js(0): uciok
292 >lozza.js(0): isready
292 <lozzacand.js(1): id name Lozza 8 (cand)
292 <lozza.js(0): readyok
292 <lozzacand.js(1): id author Colin Jenkins
293 <lozzacand.js(1): uciok
293 >lozzacand.js(1): isready
293 <lozzacand.js(1): readyok
Started game 1 of 200000 (lozza.js vs lozzacand.js)
293 >lozza.js(0): ucinewgame
293 >lozza.js(0): position fen r1bqk2r/1p2bppp/p1nppn2/6B1/4PP2/2N2N2/PPPQ2PP/R3KB1R w KQkq - 0 1
293 >lozzacand.js(1): ucinewgame
293 >lozzacand.js(1): position fen r1bqk2r/1p2bppp/p1nppn2/6B1/4PP2/2N2N2/PPPQ2PP/R3KB1R w KQkq - 0 1
293 >lozza.js(0): isready
307 <lozza.js(0): readyok
307 >lozza.js(0): go wtime 10100 btime 10100 winc 100 binc 100
348 <lozza.js(0): info depth 1 seldepth 5 score cp 98 nodes 24 time 40 nps 600 hashfull 0 pv e1c1
359 <lozza.js(0): info depth 2 seldepth 9 score cp 87 nodes 94 time 52 nps 1807 hashfull 0 pv e1c1 e8g8
363 <lozza.js(0): info depth 3 seldepth 14 score cp 91 nodes 254 time 55 nps 4618 hashfull 0 pv e1c1 e8g8 g2g4
365 <lozza.js(0): info depth 4 seldepth 14 score cp 84 nodes 406 time 57 nps 7122 hashfull 0 pv e1c1 e8g8 g5f6 e7f6
368 <lozza.js(0): info depth 5 seldepth 14 score cp 84 nodes 519 time 60 nps 8650 hashfull 0 pv e1c1 e8g8 g5f6 e7f6 d2d6
375 <lozza.js(0): info depth 6 seldepth 14 score cp 83 nodes 1060 time 67 nps 15820 hashfull 0 pv e1c1 e8g8 c1b1 b7b5 g5f6 e7f6
387 <lozza.js(0): info depth 7 seldepth 16 score cp 80 nodes 2103 time 79 nps 26620 hashfull 0 pv e1c1 e8g8 c1b1 d8a5 g2g4 b7b5 h1g1
403 <lozza.js(0): info depth 8 seldepth 19 score cp 78 nodes 4011 time 96 nps 41781 hashfull 0 pv e1c1 e8g8 c1b1 d8a5 g2g4 b7b5 h1g1 b5b4
424 <lozza.js(0): info depth 9 seldepth 19 score cp 74 nodes 6364 time 116 nps 54862 hashfull 0 pv e1c1 e8g8 c1b1 d8a5 g2g4 b7b5 g5f6 e7f6 d2d6
459 <lozza.js(0): info depth 10 seldepth 19 score upperbound 64 nodes 12034 time 152 nps 79171 hashfull 0 pv
474 <lozza.js(0): info depth 10 seldepth 19 score cp 61 nodes 15527 time 167 nps 92976 hashfull 1 pv e1c1 d8a5 c1b1 e8g8 g2g4 b7b5 g5f6 e7f6 d2d6 c8b7
483 <lozza.js(0): info depth 11 seldepth 19 score lowerbound 71 nodes 17660 time 175 nps 100914 hashfull 1 pv e1c1
484 <lozza.js(0): info depth 10 seldepth 19 score cp 61 nodes 17993 time 177 nps 101655 hashfull 1 pv e1c1 d8a5 c1b1 e8g8 g2g4 b7b5 g5f6 e7f6 d2d6 c8b7
537 <lozza.js(0): info depth 12 seldepth 20 score lowerbound 71 nodes 30660 time 230 nps 133304 hashfull 2 pv e1c1
561 <lozza.js(0): info depth 11 seldepth 20 score lowerbound 86 nodes 37243 time 253 nps 147205 hashfull 2 pv e1c1
563 <lozza.js(0): info depth 10 seldepth 20 score cp 65 nodes 37741 time 256 nps 147425 hashfull 3 pv e1c1 d8a5 d2e1 e8g8 e4e5 d6e5 f4e5 f6g4 g5e7 c6e7
668 <lozza.js(0): info depth 13 seldepth 23 score upperbound 55 nodes 59813 time 360 nps 166147 hashfull 4 pv
845 <lozza.js(0): info depth 13 seldepth 23 score upperbound 40 nodes 98384 time 537 nps 183210 hashfull 7 pv
887 <lozza.js(0): info depth 13 seldepth 23 score lowerbound 52 nodes 104984 time 579 nps 181319 hashfull 8 pv e1c1
887 <lozza.js(0): bestmove e1c1
887 >lozzacand.js(1): position fen r1bqk2r/1p2bppp/p1nppn2/6B1/4PP2/2N2N2/PPPQ2PP/R3KB1R w KQkq - 0 1 moves e1c1
887 >lozzacand.js(1): isready
889 <lozzacand.js(1): readyok
889 >lozzacand.js(1): go wtime 9620 btime 10100 winc 100 binc 100
935 <lozzacand.js(1): info depth 1 seldepth 8 score cp -87 nodes 43 time 46 nps 934 hashfull 0 pv e8g8
941 <lozzacand.js(1): info depth 2 seldepth 13 score cp -91 nodes 249 time 52 nps 4788 hashfull 0 pv e8g8 g2g4
944 <lozzacand.js(1): info depth 3 seldepth 13 score cp -84 nodes 441 time 54 nps 8166 hashfull 0 pv e8g8 g5f6 e7f6
945 <lozzacand.js(1): info depth 4 seldepth 13 score cp -84 nodes 481 time 56 nps 8589 hashfull 0 pv e8g8 g5f6 e7f6 d2d6
950 <lozzacand.js(1): info depth 5 seldepth 14 score cp -83 nodes 899 time 61 nps 14737 hashfull 0 pv e8g8 c1b1 b7b5 g5f6 e7f6
955 <lozzacand.js(1): info depth 6 seldepth 14 score cp -83 nodes 1216 time 65 nps 18707 hashfull 0 pv e8g8 c1b1 b7b5 g5f6 e7f6 d2d6
965 <lozzacand.js(1): info depth 7 seldepth 18 score cp -78 nodes 2497 time 75 nps 33293 hashfull 0 pv e8g8 c1b1 d8a5 g2g4 b7b5 h1g1 b5b4
986 <lozzacand.js(1): info depth 8 seldepth 18 score cp -74 nodes 5655 time 97 nps 58298 hashfull 0 pv e8g8 c1b1 d8a5 g2g4 b7b5 g5f6 e7f6 d2d6
1029 <lozzacand.js(1): info depth 9 seldepth 19 score cp -78 nodes 15426 time 140 nps 110185 hashfull 1 pv d8a5 a2a3 e8g8 c1b1 h7h6 g5h4 b7b5 h4f6 e7f6
1052 <lozzacand.js(1): info depth 10 seldepth 19 score cp -71 nodes 20612 time 163 nps 126453 hashfull 1 pv d8a5 a2a3 e8g8 c1b1 a8b8 d2e1 b7b5 g5f6 e7f6 d1d6
1099 <lozzacand.js(1): info depth 11 seldepth 20 score upperbound -81 nodes 31310 time 209 nps 149808 hashfull 2 pv
1117 <lozzacand.js(1): info depth 11 seldepth 20 score lowerbound -71 nodes 35688 time 228 nps 156526 hashfull 3 pv d8a5
1122 <lozzacand.js(1): info depth 10 seldepth 20 score cp -72 nodes 36776 time 232 nps 158517 hashfull 3 pv d8a5 d2e1 e8g8 c1b1 f8d8 g2g4 b7b5 g5f6 e7f6 e4e5
1154 <lozzacand.js(1): info depth 12 seldepth 21 score lowerbound -62 nodes 44231 time 264 nps 167541 hashfull 3 pv d8a5
1259 <lozzacand.js(1): info depth 11 seldepth 22 score cp -60 nodes 64153 time 369 nps 173856 hashfull 5 pv e8g8 e4e5 d6e5 f4e5 f6d5 h2h4 d8c7 c1b1 h7h6 c3d5 e6d5
1304 <lozzacand.js(1): bestmove e8g8
1304 >lozza.js(0): position fen r1bqk2r/1p2bppp/p1nppn2/6B1/4PP2/2N2N2/PPPQ2PP/R3KB1R w KQkq - 0 1 moves e1c1 e8g8
1304 >lozza.js(0): isready
1305 <lozza.js(0): readyok
1305 >lozza.js(0): go wtime 9620 btime 9785 winc 100 binc 100

[hgm]

'Insanely verbose' describes it pretty well.

Note that the isready/readyok exchange before evert go command in the example is not standard. Usually a position command is directly followed by a go command.

[ericlangedijk]

Thanks! Clear.
As I suspected

[ericlangedijk]

So... to circumvent a lot of the fen + moves could we use an optimizer inside the engine?
"if lastposition_string is almost the same as uci_command" only do the new move on our current board.
Instead of rebuilding the whole thing again?

[Steve Maughan]

With respect, you're trying to solve a problem that doesn't really exist. From a human perspective, sending the start position (or FEN) and all moves seems wasteful. But UCI is a stateless protocol, where the engine is not expected to remember anything (although it can choose to retain history tables etc). In practice the verbose communication has zero impact on playing strength. A decent engine can play a whole game with 1 second + 0.05 second increments without the verbose communication playing a significant role.

— Steve

[mar]

what the op proposes is something I did when switching Dany's Isa to uci (the motivation being buggy GUIs that claim to support xboard but not really, like banksia)

the point being Isa validates moves and the validation is a bit expensive (now doing only partial movegen per move)
it's probably ok to not validate the moves, but then the engine might crash on invalid input (GUIs won't send invalid input, but people in console might - actually there's Arena that will happily ask the engines to search in positions with 0 legal moves)

the GUIs might compress the move list by sending fen + moves since last irreversible move, but I guess nobody does this (perhaps it might even break some engines)
I'm not sure I'm sold on the idea that UCI really is a "stateless" protocol, definitely the engine still remembers all state since last search as the game progresses (of course keeps the TT)
and sending every move each time feels wasteful, if not stupid, doing useless work
no matter how much "zero impact it has", I hate doing useless work over and over again, YMMV
plus there might be even some IO overhead

[hgm]

'Statelessness' of the protocol is not the same as statelessness of the engine; the latter obviously has many states (thinking, pondering, waiting idly for input), and every option setting defines another state variable.

But even statelessness of the protocol is a myth; there obviously is a state where a 'go' command can be accepted, and one where it is not, and the 'position' command brings you from one into the other.

Checking legality, or even pseudo-legality of the input moves is not really necessary. But you can only meaningfully exploit that if you would never check any move for legality. If you already have a routine in the engine that does this it will be much faster than the IO overhead, and you might as well apply it to every move. Making sure an engine cannot get into a game state it cannot handle requires more than just checking moves; you would also have to check any FEN.

And if you have a routine for checking the acceptability of a position, you might as well have it operate only on the position that results after applying the moves. You can then use a very simple routine for applying the moves. Most moves can be taken at face value (i.e. remove any pieces in the origin and destination, place the moving piece (or the type indicated by the promotion suffix) in the destination (updating the hash key for each of these operations)). The only tricky cases are the moves with implied side effects: castling and e.p. capture. These are easily recognized, though (king moving 2 squares, or pawn moving diagonally to an empty square), and then you can perform the side effect (rook move or pawn removal).

Note that having to search positions without legal moves is an independent issue; you can get into such positions through a sequence of legal moves as well. You could try to detect such positions, and then reject those with an error message (through 'info string'). The UCI protocol specs do not tell what you should do here; you could argue that it would be OK to just ignore the 'go' command here ("a command that was not supposed to come at that time"), but it would probably be better to reply with a 'bestmove' command with an invalid move (like '0000', which at least is a move syntax allowed by UCI), justified by the "garbage-in-garbage-out principle".

All my engines are WB, but the funny thing is that internally they all work like UCI for taking moves back (the 'undo' and 'remove' commands): they remember the FEN from which they started, and then accumulate an array of all input moves since then. On reception of an undo or remove they just set up that initial position, and then replay as many moves from the array as needed by redirecting the input to the array. (E.g. by setting a variable that specifies how many of the stored moves should be played, having the input command take the move from the array instead of reading a line from standard input, decrementing this counter, if the latter is non-zero.) That is codewise a very simple way to implement taking back moves.

[ericlangedijk]

It also depends.. if we are in 'terminal' or 'user' mode entering fens and moves, which can be full of errors. But that is detectible I think. If we are in 'non terminal' or 'gui' mode we can skip validation and do some optimization (if we feel like it and I do).
I think I can handle 25.000 fen strings + 50 moves in one second, but still I feel a LOT of useless time is wasted if I do not optimize it.

[benvining]

Spending some time making sure than your FEN parsing function isn't incredibly inefficient might be worth it. But trying to keep a list of moves so that you can only make the new moves, I'm not sure that's worth it. As far as I understand, it is legal in UCI for the move list to deviate from the last one sent without a `ucinewgame` in between (for example if the ponder move wasn't played), which means that to correctly implement that you'd have to remember the starting FEN and keep a list of all moves played, and then for each position command you'd have to check if the starting position and the move list is the same as the ones from the last position command (except for the last move in the list of course). So whether you're doing incremental updates or fully refreshing your board with each position command, either way you'd need to parse the FEN string, which is likely the most expensive part; parsing moves from UCI notation and making them on the internal board should probably be quite cheap.