help me test multipv

[elcabesa]

The UCI specs do not say that the 'min' and 'max' parameters are required, right? So if there is no maximum to a spin option you could just leave it out.

He asked about compatibility with gui. His version doesn't work with Arena. It's not possible to setup multi pv.

[brtzsnr]

Got it. I fixed the two issues identified so far. Here is a new binary.

https://drive.google.com/open?id=0B2uKL ... UZnRnByV3M

[Ferdy]

Got it. I fixed the two issues identified so far. Here is a new binary.

https://drive.google.com/open?id=0B2uKL ... UZnRnByV3M

It works in Arena and Fritz, but there are issues in scoring below.

The score of the best move was wrong.
The score of the best move is always 0.
The score of 2nd pv and 3rd pv is always the same.

[d]rnbqkb1r/pppp1ppp/4pn2/4P3/8/8/PPPP1PPP/RNBQKBNR w KQkq - 0 3

Code: Select all

2016-06-20 17:32:27.328-->1:position startpos moves e2e4 g8f6 e4e5 e7e6
2016-06-20 17:32:27.328-->1:go infinite

2016-06-20 17&#58;32&#58;38.187<--1&#58;info depth 15 seldepth 15 multipv 1 score cp 0 nodes 15830160 time 10858 nps 1457842 pv e5f6 g7f6 d2d4 d7d5 c1e3 b8c6 c2c3 c6e7 f1d3 h8g8 e1f1 c8d7 b1d2 e7g6 g1f3
2016-06-20 17&#58;32&#58;42.955<--1&#58;info depth 15 seldepth 17 multipv 2 score cp 404 nodes 22496623 time 15625 nps 1439701 pv d2d4 f6e4 g1h3 h7h6 f1d3 d7d5 e5d6 e4d6 e1g1 b8c6 c2c3 f8e7 h3f4 e8g8 b1d2
2016-06-20 17&#58;32&#58;45.521<--1&#58;info depth 15 seldepth 17 multipv 3 score cp 404 nodes 26172058 time 18193 nps 1438575 pv g1f3 f6d5 d2d4 d5b6 f1d3 c7c5 e1g1 c5d4 c2c3 d4c3 b1c3 b8c6 a2a3 d7d5 c1g5

[brtzsnr]

Thanks Ferdinand! Your example is a very good test position. If fixed the score

https://drive.google.com/open?id=0B2uKL ... VluOUVxcWs

Code: Select all

info depth 0 seldepth 0 multipv 1 score cp 325 nodes 5 time 0 nps 151080 pv e5f6 d8f6
info depth 1 seldepth 1 multipv 1 score cp 325 nodes 67 time 0 nps 623464 pv e5f6 d8f6
info depth 1 seldepth 1 multipv 2 score cp 196 nodes 118 time 0 nps 848902 pv d2d4
info depth 1 seldepth 1 multipv 3 score cp 151 nodes 150 time 0 nps 1018834 pv b1c3
info depth 2 seldepth 2 multipv 1 score cp 325 nodes 479 time 0 nps 829079 pv e5f6 d8f6
info depth 2 seldepth 2 multipv 2 score cp 84 nodes 966 time 0 nps 1028513 pv b1c3 f6d5 c3d5 e6d5
info depth 2 seldepth 2 multipv 3 score cp 64 nodes 1226 time 1 nps 1118288 pv d2d4 f6d5
info depth 3 seldepth 3 multipv 1 score cp 384 nodes 2091 time 1 nps 1131915 pv e5f6 b8c6 f6g7 f8g7
info depth 3 seldepth 3 multipv 2 score cp 117 nodes 3879 time 3 nps 1236071 pv d2d3 f6d5 c2c4
info depth 3 seldepth 3 multipv 3 score cp 114 nodes 3941 time 3 nps 1247455 pv d2d4 f6e4 c1d2 e4d2 b1d2
info depth 4 seldepth 4 multipv 1 score cp 340 nodes 5177 time 4 nps 1213554 pv e5f6 d8f6 d2d4 b8c6
info depth 4 seldepth 4 multipv 2 score cp 84 nodes 7564 time 5 nps 1275781 pv b1c3 f6d5 c3d5 e6d5
info depth 4 seldepth 4 multipv 3 score cp 77 nodes 8242 time 6 nps 1304631 pv d2d4 f6d5 b1c3 c7c6 c3d5 c6d5
info depth 5 seldepth 6 multipv 1 score cp 413 nodes 9992 time 7 nps 1318522 pv e5f6 d8f6 d2d4 b8c6 c2c3
info depth 5 seldepth 6 multipv 2 score cp 106 nodes 14450 time 11 nps 1311516 pv d2d4 f6e4 b1c3 d7d5 c3e4 d5e4
info depth 5 seldepth 6 multipv 3 score cp 101 nodes 16291 time 12 nps 1283882 pv g2g3 f6d5 d2d4 f8b4 c2c3 b4e7
bestmove e5f6 ponder d8f6
readyok

[cdani]

Now it works in ChessBase!

[Ferdy]

https://drive.google.com/open?id=0B2uKL ... VluOUVxcWs

There are times that pv scores were not properly sorted. See pv's at depth 5, 13 and 15.

[d]rnbqkbnr/pppp1ppp/8/4p3/4P3/8/PPPP1PPP/RNBQKBNR w KQkq e6 0 2

Code: Select all

2016-06-21 13:10:21.334-->1:position startpos moves e2e4 e7e5
2016-06-21 13:10:21.344-->1:go infinite
2016-06-21 13&#58;10&#58;22.726<--1&#58;info depth 0 seldepth 0 multipv 1 score cp 0 nodes 2 time 0 nps 2000000 pv d2d4 e5d4 g1f3 c7c5 c2c3 b8c6 c3d4 c5d4 f3d4 g8f6 b1c3 d7d5 f1b5 f6e4 c3e4 d5e4 d4c6 d8d1 e1d1 b7c6 b5c6
2016-06-21 13&#58;10&#58;22.731<--1&#58;info depth 1 seldepth 1 multipv 1 score cp 79 nodes 49 time 0 nps 49000000 pv b1c3 b8c6 a2a3 c6d4 f1c4
2016-06-21 13&#58;10&#58;22.736<--1&#58;info depth 1 seldepth 1 multipv 2 score cp 42 nodes 92 time 0 nps 92000000 pv f1c4
2016-06-21 13&#58;10&#58;22.746<--1&#58;info depth 1 seldepth 1 multipv 3 score cp 36 nodes 124 time 0 nps 124000000 pv g1f3 b8c6 b1c3 g8f6 d2d4 e5d4 f3d4 f8c5 d4b3 c5d6 f2f4 d6b4 e4e5 f6e4
2016-06-21 13&#58;10&#58;22.751<--1&#58;info depth 2 seldepth 2 multipv 1 score cp 31 nodes 344 time 0 nps 344000000 pv d2d4 b8c6 d4e5 c6e5
2016-06-21 13&#58;10&#58;22.756<--1&#58;info depth 2 seldepth 2 multipv 2 score cp 0 nodes 500 time 1 nps 499950 pv b1c3 b8c6 a2a3 c6d4 f1c4
2016-06-21 13&#58;10&#58;22.761<--1&#58;info depth 2 seldepth 2 multipv 3 score cp 0 nodes 624 time 1 nps 623937 pv g1f3 g8f6 b1c3 b8c6 d2d4 e5d4 f3d4 f8c5 d4b3 c5d6 f2f4 d6b4 e4e5 f6e4
2016-06-21 13&#58;10&#58;22.765<--1&#58;info depth 3 seldepth 3 multipv 1 score cp 59 nodes 1491 time 2 nps 745462 pv b1c3 b8c6 a2a3 c6d4 f1c4
2016-06-21 13&#58;10&#58;22.770<--1&#58;info depth 3 seldepth 3 multipv 2 score cp 52 nodes 1985 time 2 nps 992450 pv d2d4 d7d5 d4e5 d5e4 d1d8 e8d8
2016-06-21 13&#58;10&#58;22.775<--1&#58;info depth 3 seldepth 3 multipv 3 score cp 41 nodes 2067 time 2 nps 1033448 pv g1f3 d7d5 d2d4 d5e4 f3e5
2016-06-21 13&#58;10&#58;22.782<--1&#58;info depth 4 seldepth 4 multipv 1 score cp 41 nodes 2535 time 3 nps 844943 pv g1f3 d7d5 d2d4 d5e4 f3e5
2016-06-21 13&#58;10&#58;22.786<--1&#58;info depth 4 seldepth 4 multipv 2 score cp 28 nodes 3115 time 3 nps 1038264 pv b1c3 b8c6 g1f3 d8f6
2016-06-21 13&#58;10&#58;22.791<--1&#58;info depth 4 seldepth 4 multipv 3 score cp 21 nodes 3776 time 4 nps 943929 pv d2d4 b8c6 d4e5 c6e5
2016-06-21 13&#58;10&#58;22.795<--1&#58;info depth 5 seldepth 5 multipv 1 score cp 49 nodes 5968 time 6 nps 994600 pv g1f3 d7d5 f3e5 d5e4 b1c3 f7f6 e5c4
2016-06-21 13&#58;10&#58;22.799<--1&#58;info depth 5 seldepth 5 multipv 2 score cp 36 nodes 6043 time 6 nps 1007099 pv d2d4 e5d4 c2c3 d7d5 e4d5
2016-06-21 13&#58;10&#58;22.803<--1&#58;info depth 5 seldepth 5 multipv 3 score cp 53 nodes 7134 time 7 nps 1019084 pv b1c3 b8c6 g1f3 f8c5 a2a3 a7a6 b2b4 c5e7
2016-06-21 13&#58;10&#58;22.808<--1&#58;info depth 6 seldepth 6 multipv 1 score cp 33 nodes 12794 time 14 nps 913804 pv d2d4 e5d4 g1f3 b8c6 f3d4 d7d5
2016-06-21 13&#58;10&#58;22.812<--1&#58;info depth 6 seldepth 6 multipv 2 score cp 8 nodes 13835 time 15 nps 922277 pv b1c3 b8c6 g1f3 f8c5 a2a3 g8f6
2016-06-21 13&#58;10&#58;22.816<--1&#58;info depth 6 seldepth 6 multipv 3 score cp 8 nodes 14994 time 16 nps 937072 pv g1f3 d7d5 f3e5 d5e4 f1c4 g8h6
2016-06-21 13&#58;10&#58;22.820<--1&#58;info depth 7 seldepth 7 multipv 1 score cp 92 nodes 25344 time 28 nps 905091 pv g1f3 b8c6 d2d4 e5d4 f3d4 c6d4 d1d4 c7c5
2016-06-21 13&#58;10&#58;22.825<--1&#58;info depth 7 seldepth 7 multipv 2 score cp 71 nodes 29744 time 33 nps 901281 pv b1c3 b8c6 g1f3 a7a6 d2d4 e5d4 f3d4
2016-06-21 13&#58;10&#58;22.829<--1&#58;info depth 7 seldepth 8 multipv 3 score cp 53 nodes 37437 time 42 nps 891306 pv d2d4 e5d4 g1f3 d7d5 d1d4 d5e4 d4e4 f8e7
2016-06-21 13&#58;10&#58;22.834<--1&#58;info depth 8 seldepth 9 multipv 1 score cp 41 nodes 59444 time 66 nps 900614 pv g1f3 b8c6 b1c3 a7a6 d2d4 e5d4 f3d4 g8f6 d4c6 d7c6 d1d8 e8d8
2016-06-21 13&#58;10&#58;22.838<--1&#58;info depth 8 seldepth 9 multipv 2 score cp 41 nodes 63076 time 70 nps 901034 pv b1c3 b8c6 g1f3 a7a6 d2d4 e5d4 f3d4 g8f6 d4c6 d7c6 d1d8 e8d8
2016-06-21 13&#58;10&#58;22.842<--1&#58;info depth 8 seldepth 9 multipv 3 score cp 15 nodes 122789 time 137 nps 896218 pv f1c4 g8f6 b1c3 b8c6 a2a3 b7b5 c4b5 a7a6
2016-06-21 13&#58;10&#58;22.847<--1&#58;info depth 9 seldepth 9 multipv 1 score cp 45 nodes 158185 time 177 nps 893649 pv g1f3 b8c6 b1c3 g8f6 d2d4 e5d4 f3d4 d7d5 f1b5 d5e4 d4c6 d8d1 c3d1 b7c6 b5c6
2016-06-21 13&#58;10&#58;22.851<--1&#58;info depth 9 seldepth 9 multipv 2 score cp 45 nodes 166618 time 184 nps 905480 pv b1c3 b8c6 g1f3 g8f6 d2d4 e5d4 f3d4 d7d5 f1b5 d5e4 d4c6 d8d1 c3d1 b7c6 b5c6
2016-06-21 13&#58;10&#58;22.856<--1&#58;info depth 9 seldepth 12 multipv 3 score cp 28 nodes 307772 time 339 nps 907830 pv d2d4 e5d4 c2c3 d7d5 d1d4 d5e4 d4e4 f8e7 f1b5 c7c6 b5d3
2016-06-21 13&#58;10&#58;22.860<--1&#58;info depth 10 seldepth 12 multipv 1 score cp 21 nodes 427733 time 483 nps 885524 pv g1f3 b8c6 b1c3 g8f6 d2d4 e5d4 f3d4 c6d4 d1d4 c7c5
2016-06-21 13&#58;10&#58;22.865<--1&#58;info depth 10 seldepth 12 multipv 2 score cp 21 nodes 449561 time 510 nps 881441 pv b1c3 b8c6 g1f3 g8f6 d2d4 e5d4 f3d4 c6d4 d1d4 c7c5
2016-06-21 13&#58;10&#58;22.869<--1&#58;info depth 10 seldepth 12 multipv 3 score cp 5 nodes 460535 time 522 nps 882200 pv d2d4 e5d4 g1f3 c7c5 c2c3 d8e7 c3d4 e7e4 c1e3 c5d4 f3d4
2016-06-21 13&#58;10&#58;22.874<--1&#58;info depth 11 seldepth 12 multipv 1 score cp 63 nodes 670398 time 773 nps 867218 pv g1f3 b8c6 b1c3 g8f6 d2d4 e5d4 f3d4 d7d5 e4d5 f6d5 d1e2 d5e7 d4c6 b7c6
2016-06-21 13&#58;10&#58;22.878<--1&#58;info depth 11 seldepth 12 multipv 2 score cp 63 nodes 698847 time 808 nps 864860 pv b1c3 b8c6 g1f3 g8f6 d2d4 e5d4 f3d4 d7d5 e4d5 f6d5 d1e2 d5e7 d4c6 b7c6
2016-06-21 13&#58;10&#58;22.883<--1&#58;info depth 11 seldepth 12 multipv 3 score cp 26 nodes 793390 time 910 nps 871807 pv d2d4 e5d4 g1f3 c7c5 c2c3 d8e7 c3d4 e7e4 c1e3 c5d4 b1d2 f8b4 f3d4
2016-06-21 13&#58;10&#58;22.888<--1&#58;info depth 12 seldepth 13 multipv 1 score cp 63 nodes 1177596 time 1376 nps 855762 pv g1f3 b8c6 b1c3 g8f6 d2d4 e5d4 f3d4 d7d5 e4d5 f6d5 d1e2 d5e7 d4c6 b7c6
2016-06-21 13&#58;10&#58;23.051<--1&#58;info depth 12 seldepth 13 multipv 2 score cp 34 nodes 1451161 time 1702 nps 852572 pv b1c3 g8f6 g1f3 f8b4 f1c4 b8c6 c3d5 b4c5 b2b4 f6d5 e4d5 c6b4 f3e5
2016-06-21 13&#58;10&#58;24.111<--1&#58;info depth 12 seldepth 13 multipv 3 score cp 33 nodes 2279379 time 2761 nps 825515 pv d2d4 e5d4 g1f3 c7c5 c2c3 d8e7 c3d4 e7e4 f1e2 c5d4 b1d2 f8b4 e1g1
2016-06-21 13&#58;10&#58;26.021<--1&#58;info depth 13 seldepth 15 multipv 1 score cp 54 nodes 3731449 time 4671 nps 798808 pv g1f3 g8f6 d2d4 f6e4 f1d3 d7d5 f3e5 f8b4 c2c3 b4d6 e1g1 d6e5 d4e5 e8g8
2016-06-21 13&#58;10&#58;27.337<--1&#58;info depth 13 seldepth 15 multipv 2 score cp 21 nodes 4824629 time 5987 nps 805804 pv b1c3 g8f6 g1f3 f8b4 f1c4 b8c6 c3d5 f6d5 e4d5 c6a5 c4e2 e5e4 f3e5
2016-06-21 13&#58;10&#58;27.548<--1&#58;info depth 13 seldepth 15 multipv 3 score cp 34 nodes 4993629 time 6197 nps 805767 pv d2d4 e5d4 g1f3 f8b4 c2c3 d4c3 b1c3 d8e7 e4e5 d7d6 c1f4 b4c3 b2c3 c8f5
2016-06-21 13&#58;10&#58;31.423<--1&#58;info depth 14 seldepth 16 multipv 1 score cp 13 nodes 8026719 time 10073 nps 796809 pv d2d4 e5d4 g1f3 b8c6 f3d4 g8f6 b1c3 f8e7 f1e2 e8g8 e1g1 d7d5 d4c6 b7c6 e4d5 f6d5
2016-06-21 13&#58;10&#58;31.920<--1&#58;info depth 14 seldepth 16 multipv 2 score cp 13 nodes 8408220 time 10570 nps 795434 pv g1f3 b8c6 b1c3 g8f6 d2d4 e5d4 f3d4 f8e7 f1e2 e8g8 e1g1 d7d5 d4c6 b7c6 e4d5 f6d5
2016-06-21 13&#58;10&#58;32.459<--1&#58;info depth 14 seldepth 16 multipv 3 score cp 13 nodes 8813924 time 11109 nps 793358 pv b1c3 g8f6 g1f3 b8c6 d2d4 e5d4 f3d4 f8e7 f1e2 e8g8 e1g1 d7d5 d4c6 b7c6 e4d5 f6d5
2016-06-21 13&#58;10&#58;40.942<--1&#58;info depth 15 seldepth 17 multipv 1 score cp 57 nodes 15311317 time 19593 nps 781463 pv b1c3 g8f6 g1f3 f8b4 f3e5 e8g8 f1e2 b4c3 d2c3 f8e8 e5d3 f6e4 e1g1 d7d5 f2f3
2016-06-21 13&#58;10&#58;43.090<--1&#58;info depth 15 seldepth 17 multipv 2 score cp 65 nodes 17031767 time 21740 nps 783421 pv g1f3 b8c6 b1c3 g8f6 d2d4 e5d4 f3d4 d7d5 d4c6 b7c6 e4d5 f6d5 c3d5 d8d5 c1d2
2016-06-21 13&#58;10&#58;56.802<--1&#58;info depth 15 seldepth 17 multipv 3 score cp 46 nodes 27675128 time 35453 nps 780613 pv c2c4 g8f6 b1c3 c7c5 g1f3 b8c6 d2d3 d7d6 h2h3 h7h6 f1e2 f8e7 e1g1 e8g8 a2a3
2016-06-21 13&#58;11&#58;22.492<--1&#58;info depth 16 seldepth 17 multipv 1 score cp 29 nodes 47368232 time 61142 nps 774718 pv g1f3 b8c6 f1c4 g8f6 e1g1 f6e4 b1c3 f8e7 c3e4 d7d5 c4b5 d5e4 b5c6 b7c6 f3e5 d8d5
2016-06-21 13&#58;11&#58;35.574<--1&#58;info depth 16 seldepth 17 multipv 2 score cp 11 nodes 57220125 time 74223 nps 770919 pv b1c3 b8c6 g1f3 g8f6 d2d4 e5d4 f3d4 f8b4 d4c6 d7c6 d1d8 e8d8 c1d2 h8e8 e1c1 c8g4
2016-06-21 13&#58;12&#58;02.384<--1&#58;info depth 16 seldepth 17 multipv 3 score cp 10 nodes 77757390 time 101034 nps 769610 pv d2d4 e5d4 g1f3 b8c6 f3d4 g8f6 b1c3 f8b4 d4c6 b7c6 e4e5 b4c3 b2c3 f6d5 c3c4 d5c3

[brtzsnr]

Should the pv scores be sorted? Especially at high depths you don't want to wait 3x just to sort the pv lines by score.

The PV lines are printed as soon as they are computed. Ideally the search returns the scores in a decreasing order, but the search is full of instabilities (hash, lmr, nullmove) and sometimes the second move is searched more or less deeply.

[hgm]

The way all UCI engines I have seen do this is resend all N PVs (sorted by score) whenever the search comes up with one new PV. The new PV replaces the PV for that same move at a previous depth, so that you would never have two PVs for the same move, but otherwise depths can be mixed. E.g. if multipv=4, and the first PV from the d=15 iteration comes in, you would send that, plus the best 3 PVs from the 14th iteration for different moves.

Remember that UCI is a protocol designed to send the same redundant information over and over again...

[Evert]

Should the pv scores be sorted? Especially at high depths you don't want to wait 3x just to sort the pv lines by score.

The PV lines are printed as soon as they are computed. Ideally the search returns the scores in a decreasing order, but the search is full of instabilities (hash, lmr, nullmove) and sometimes the second move is searched more or less deeply.

The spec doesn't really say:

Code: Select all

info multipv 
		this for the multi pv mode.
		for the best move/pv add "multipv 1" in the string when you send the pv.
		in k-best mode always send all k variants in k strings together.

However, it sortof implies that "multipv 1" is for the best move (so it should be sorted first) and it suggests that they should all be sent at once, not one at a time (so you can indeed sort them if you want).
Personally, I would say it's the responsibility of the GUI to sort the multiple PVs, not the responsibility of the engine (which is an argument that should also appeal to UCI fans), and the results should just be sent as they come in.
You can always make an option to change the behaviour if it causes problems in some GUIs.

On a different note, having a later line improve on an earlier one implies that the move ordering is wrong at the root. You should check if this occurs frequently (if it does there may be something to gain there).

[Evert]

The way all UCI engines I have seen do this is resend all N PVs (sorted by score) whenever the search comes up with one new PV. The new PV replaces the PV for that same move at a previous depth, so that you would never have two PVs for the same move, but otherwise depths can be mixed. E.g. if multipv=4, and the first PV from the d=15 iteration comes in, you would send that, plus the best 3 PVs from the 14th iteration for different moves.

Hilarious. I suppose that's another way to interpret the "always send k strings", but it seems unintended by the protocol, which does suggest that you should send all k strings at once at the end of the search (and thus wait for the iteration to finish) and sorted in the sense that the best move gets labelled multipv=1.

Which seems like a silly requirement as well, as I mentioned in my other post.