NICE 1.2

[Rebel]

I came to the conclusion that NICE is 95% programmer tool, hence I post it here.

Version 1.2 is a significant improvement.

1. NICE now can make predictions with with a precision of 1 elo. The idea to invest 20 minutes before you put an engine change into cute-chess looks attractive to me when NICE can report an elo gain or regression with a precision of 1 elo. It's to hope NICE turns out to be a good prophet

2. Reporting errors - Some engines (in fact a bit more than some) have the habit to surpress the mainlines of the early plies, some hide the first x plies, some hide the main lines the first 100ms and there are even those who use 1 full second. It's probably inspired by a speed gain of 1-2 millieseconds. When there is no main line NICE will report the number of cases found in the error column of the html and the result will be unreliable.

3. The time-bonus feature has been dropped. Someone said, it only adds noise. After a good look I had to conclude it is even worse, without it the results are more reliable.

http://rebel13.nl/misc/nice.html

[mar]

thanks Ed, that's really Nice
is there a way to utilize multiple threads (actually processes)? I only see 1 core utilized (1 engine instance)
also, how accurately can Nice detect say search improvements?

[Rebel]

thanks Ed, that's really Nice
is there a way to utilize multiple threads (actually processes)? I only see 1 core utilized (1 engine instance)

Edit the batch file you plan to use:

set THREADS=1

also, how accurately can Nice detect say search improvements?

I don't know. Well, yet... as most of the time I tested eval changes.

I suppose there is no real difference between the two, tests at 100ms, 250ms, 500ms and 1000ms show significant elo increments. I think the golden rule is to use a higher time control testing search changes.

[mar]

I tried to set THREADS to 8, but I it still only see 1 engine process

[Rebel]

I tried to set THREADS to 8, but I it still only see 1 engine process

Then I misunderstood you. I tried Cheng 4.39, created test.bat

Code: Select all

del debug*.*

set MT=100
set HASH=128
set THREADS=1
set PROTOCOL=uci
set EPD=epd\sf1.epd
set OPTIONS=MultiPV=1
set MRL=x1

set EXE=engines\Cheng4_4.39.exe
set NAME=Cheng4_4.39

mea.exe --engine %EXE% --name %NAME% --hash %HASH% --threads %THREADS% --protocol %PROTOCOL% --epd %EPD% --movetime %MT% --log --eoption %OPTIONS%
csv %MRL% %EPD% %OPTIONS% %NAME% %MT% %CCRL%
mrl %MRL% %MRL% %EPD% %MT%

exit

1. Started NICE.
2. Select "Split Run"
3. Select "test.bat"
4. Speed up by threads -> I typed 8
5. Exit program
6. Double click "@start.bat"

The "sf1.epd" is split into 8 parts of 5,000 position and runs in 8 threads.

My task manager shows 8 x Cheng.

[mar]

Oh, my bad I only ran the original batch file directly, didn't run nice.exe to generate the split batch, sorry

[Rebel]

Oh, my bad I only ran the original batch file directly, didn't run nice.exe to generate the split batch, sorry

50 push-ups or one hour in the corner

[JVMerlino]

Well I'm stumped. Either I can't make it work with Myrddin (which is an xboard engine) or NICE doesn't support xboard. Has anybody had success using NICE with any xboard engine?

[Rebel]

Well I'm stumped. Either I can't make it work with Myrddin (which is an xboard engine) or NICE doesn't support xboard. Has anybody had success using NICE with any xboard engine?

That is a question for Ferdy.

Meanwhile I did a quick run with Myrddin using the 1500 STS positions using the wb2uci adapter.

http://rebel13.nl/mea/Myrddin.html

Here is how.

The wb2uci.eng file

Code: Select all

[ENGINE]
Name = Myrddin_0.87
Author = John Merlino
Filename = wb2uci.exe

[OPTIONS]
Program = Myrddin_0.87.exe
InitString = BookOff/n
LevelExtend = none
Analyze = true
UseUndo = false
Logfile = false
LevelType = 2
SimulateHint = true
TerminateHard = false
; In WIN XP or Win NT  TerminateHard = true
Protocol = 1
Visible = Ponder,Extras

The batch file

Code: Select all

del debug*.*

set MT=1000
set HASH=128
set THREADS=1
set PROTOCOL=uci
set EPD=epd\sts.epd
set OPTIONS=MultiPV=1
set MRL=Myrddin

set EXE=engines\wb2uci.exe
set NAME=Myrddin_0.87

mea.exe --engine %EXE% --name %NAME% --hash %HASH% --threads %THREADS% --protocol %PROTOCOL% --epd %EPD% --movetime %MT% --log --eoption %OPTIONS%
csv %MRL% %EPD% %OPTIONS% %NAME% %MT% %CCRL%
mrl %MRL% %MRL% %EPD% %MT%

exit

Double click the batch file and it will run in one thread.

[JVMerlino]

It works - AND I found a (possible) bug in Myrddin! It does not accept "st" command values less than 1, because the protocol says it is used to "set the exact number of seconds per move". I guess, when I implemented that more than 10 years ago, I never considered decimal values. But the text of the protocol (and the example, which only uses the number 30) should probably be updated.

However, trying to do a Split Run via Myrddin.bat in NICE fails with "Error in line 8".

Thanks very much!