Arasan, repetition and insufficient material

[hgm]

Well, I am a strong believer in fixing problems where they ly, and never catering to bugs of other software.

[Ferdy]

I don't know if LB is still being actively developed or not, I switched to cutechess-cli long time ago.

I have been using cutechess-cli and winboard, but LB is cool in terms of info on ave time per move, ave depth, time losses, illegal and others.
I am testing how my engine performs after making some modifications on time management, including non-capture easy move.
So I monitor tpm, d, t. Generally engines are stable, except smt with t=4, arasan i=1 and Atlas t=1.

Code: Select all

Games Completed = 4000 of 4000 (Avg game length = 36.435 sec)
Settings = Gauntlet/64MB/15000ms+100ms/M 700cp for 3 moves, D 200 moves/EPD:C:\LB\TopGM_4move.epd(10000)
Time = 21199 sec elapsed, 0 sec remaining
 1.  Deuterium v2015.1.35.161 	1609.5/4000	1071-1852-1077  (L: m=39 t=0 i=0 a=1813)(D: r=591 i=276 f=158 s=24 a=28)(tpm=275.4 d=13.52 nps=841723)
 2.  Rodent 1.7 build 1       	124.0/286	82-120-84  	(L: m=0 t=0 i=0 a=120)	(D: r=27 i=33 f=14 s=2 a=8)	(tpm=218.6 d=13.50 nps=907695)
 3.  Arasan 17.5              	139.5/286	97-104-85  	(L: m=0 t=0 i=1 a=103)	(D: r=44 i=22 f=17 s=0 a=2)	(tpm=288.6 d=13.68 nps=1018159)
 4.  Atlas 3.80 x64           	161.0/286	115-79-92  	(L: m=0 t=1 i=0 a=78)	(D: r=58 i=21 f=11 s=2 a=0)	(tpm=281.6 d=13.55 nps=1660602)
 5.  cheng4 0.38              	168.5/286	129-78-79  	(L: m=0 t=0 i=0 a=78)	(D: r=48 i=16 f=11 s=1 a=3)	(tpm=306.5 d=13.41 nps=1351316)
 6.  Gaviota                  	165.0/286	130-86-70  	(L: m=1 t=0 i=0 a=85)	(D: r=30 i=24 f=12 s=1 a=3)	(tpm=263.4 d=10.86 nps=524366)
 7.  Quazar 0.4 x64           	163.0/286	124-84-78  	(L: m=0 t=0 i=0 a=84)	(D: r=51 i=13 f=12 s=1 a=1)	(tpm=284.8 d=13.01 nps=318571)
 8.  SmarThink 1.70           	177.0/286	145-77-64  	(L: m=0 t=4 i=0 a=73)	(D: r=30 i=17 f=9 s=5 a=3)	(tpm=306.5 d=14.44 nps=940694)
 9.  spark-1.0                	193.0/286	151-51-84  	(L: m=1 t=0 i=0 a=50)	(D: r=51 i=24 f=6 s=1 a=2)	(tpm=242.6 d=12.72 nps=1909953)
10.  Texel 1.05 64-bit        	235.5/286	211-26-49  	(L: m=0 t=0 i=0 a=26)	(D: r=27 i=11 f=11 s=0 a=0)	(tpm=271.5 d=12.50 nps=965755)
11.  Toga II 3.0              	172.0/286	142-84-60  	(L: m=0 t=0 i=0 a=84)	(D: r=29 i=18 f=12 s=1 a=0)	(tpm=280.6 d=11.98 nps=923718)
12.  Vajolet2 2.0             	156.0/285	116-89-80  	(L: m=0 t=0 i=0 a=89)	(D: r=55 i=16 f=5 s=2 a=2)	(tpm=268.0 d=13.60 nps=779983)
13.  Spike 1.4                	172.5/285	126-66-93  	(L: m=0 t=0 i=0 a=66)	(D: r=66 i=16 f=6 s=2 a=3)	(tpm=287.2 d=12.15 nps=971665)
14.  DiscoCheck 5.2.1         	194.0/285	158-55-72  	(L: m=1 t=0 i=0 a=54)	(D: r=27 i=15 f=25 s=5 a=0)	(tpm=285.9 d=14.95 nps=0)
15.  Nemo SP64o 1.0.1 Beta    	169.5/285	126-72-87  	(L: m=0 t=0 i=0 a=72)	(D: r=48 i=30 f=7 s=1 a=1)	(tpm=282.4 d=11.73 nps=1602544)

[jdart]

i=1 is an illegal move? Can you tell me what illegal move Arasan made? I routinely run it through thousands of games, although I test more in Winboard mode than UCI mode (it supports both).

--Jon

[Ferdy]

i=1 is an illegal move? Can you tell me what illegal move Arasan made? I routinely run it through thousands of games, although I test more in Winboard mode than UCI mode (it supports both).

--Jon

Yes i = illegal.

When given pos with 3 reps already, Arasan will send bestmove 0000. LB does not like this and consider this as illegal.

When given pos with drawn material, KBkb, Arasan will send bestmove 0000. Again LB does not like this.

See my first post.

LB only supports uci engines, and it expects Arasan to send a legal move after those positions. Most uci engines I know will send legal move if there is one. Perhaps LB waited just one more move before adjudicating the game as drawn.

[jdart]

Ok, as I noted, I don't think that's invalid engine behavior when playing a game. The GUI should adjudicate it as a draw (especially since UCI has no means of having the engine claim a draw).

--Jon

[jdart]

I have fixed this behavior (for analysis mode only) in the dev version:

https://github.com/jdart1/arasan-chess/ ... f1a5b3094c

--Jon

[hgm]

Why should it be wrong to claim an immediate draw in analysis mode, if a draw is the optimal result? Playing a move first seems as wrong to me as reporting a mate in 5 (in analysis only) when you can in fact give a mate in 1, just because "the user likes to have as long a PV as possible in analysis mode"...

[Evert]

Why should it be wrong to claim an immediate draw in analysis mode, if a draw is the optimal result? Playing a move first seems as wrong to me as reporting a mate in 5 (in analysis only) when you can in fact give a mate in 1, just because "the user likes to have as long a PV as possible in analysis mode"...

What's "wrong" is that UCI doesn't have a (well-defined and universally understood) way for the engine to say "Oy, the root position is an end-of-game position, there is no legal move to find here". The issue here is really that the GUI is not supposed to ask the engine to think in a position like this, and what "should" happen if it does is undefined. This really is a short-coming of the protocol, but one that still has to be dealt with somehow (because of course software has bugs and people get this stuff wrong).

Cue the usual rumblings about UCI vs CECP and who "should" be in control of what.

[Ferdy]

Why should it be wrong to claim an immediate draw in analysis mode, if a draw is the optimal result? Playing a move first seems as wrong to me as reporting a mate in 5 (in analysis only) when you can in fact give a mate in 1, just because "the user likes to have as long a PV as possible in analysis mode"...

Can you claim a draw in analysis mode for engines that implements uci protocol? When the gui sends "go infinite" to the engine, the gui is the boss.

Code: Select all

*go
	* infinite
		search until the "stop" command. Do not exit the search without being told so in this mode!

So the engine will send,
info depth 1 score cp 0 time 10 nodes 100 pv 0000 0000 0000 0000
then the gui will send the stop command, and the engine will reply with
bestmove 0000.

In Arasan's case there are still legal moves, I am not talking about game playing, but in analysis mode.

[hgm]

That only specifies you must not send 'bestmove' before receiving 'stop'. If you present the engine a mate-in-1 position, it will also be done searching or pondering in 1 microsecond, with a PV of 1 move, and nothing to do but wait for 'stop'. 'infinite' in that case apparently means only 1 micro-second or 1 move. In a situation where it is already mated, or already a draw, it could also think 0 microseconds and give a PV of 0 moves. There is nothing in the UCI specs that suggests otherwise. As long as it does not send 'bestmove' spontaneous.

There is no requirement to pad a PV with null moves. You also don't do that when you can give mate in one:

info depth 3 score mate 1 pv h5f7 0000 0000

There is no formal requirement that the PV must contain as many moves as the depth parameter. Or, indeed, any moves at all. With 'position-moves' you can give zero moves after 'moves'...