NICE 1.2

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
Rebel
Posts: 5157
Joined: Thu Aug 18, 2011 10:04 am

NICE 1.2

Post by Rebel » Tue May 19, 2020 8:09 am

I came to the conclusion that NICE is 95% programmer tool, hence I post it here.

Version 1.2 is a significant improvement.

1. NICE now can make predictions with with a precision of 1 elo. The idea to invest 20 minutes before you put an engine change into cute-chess looks attractive to me when NICE can report an elo gain or regression with a precision of 1 elo. It's to hope NICE turns out to be a good prophet :wink:

2. Reporting errors - Some engines (in fact a bit more than some) have the habit to surpress the mainlines of the early plies, some hide the first x plies, some hide the main lines the first 100ms and there are even those who use 1 full second. It's probably inspired by a speed gain of 1-2 millieseconds. When there is no main line NICE will report the number of cases found in the error column of the html and the result will be unreliable.

3. The time-bonus feature has been dropped. Someone said, it only adds noise. After a good look I had to conclude it is even worse, without it the results are more reliable.

http://rebel13.nl/misc/nice.html
90% of coding is debugging, the other 10% is writing bugs.

mar
Posts: 2111
Joined: Fri Nov 26, 2010 1:00 pm
Location: Czech Republic
Full name: Martin Sedlak

Re: NICE 1.2

Post by mar » Tue May 19, 2020 1:14 pm

thanks Ed, that's really Nice :)
is there a way to utilize multiple threads (actually processes)? I only see 1 core utilized (1 engine instance)
also, how accurately can Nice detect say search improvements?
Martin Sedlak

User avatar
Rebel
Posts: 5157
Joined: Thu Aug 18, 2011 10:04 am

Re: NICE 1.2

Post by Rebel » Tue May 19, 2020 4:52 pm

mar wrote:
Tue May 19, 2020 1:14 pm
thanks Ed, that's really Nice :)
is there a way to utilize multiple threads (actually processes)? I only see 1 core utilized (1 engine instance)
Edit the batch file you plan to use:

set THREADS=1
also, how accurately can Nice detect say search improvements?
I don't know. Well, yet... as most of the time I tested eval changes.

I suppose there is no real difference between the two, tests at 100ms, 250ms, 500ms and 1000ms show significant elo increments. I think the golden rule is to use a higher time control testing search changes.
90% of coding is debugging, the other 10% is writing bugs.

mar
Posts: 2111
Joined: Fri Nov 26, 2010 1:00 pm
Location: Czech Republic
Full name: Martin Sedlak

Re: NICE 1.2

Post by mar » Wed May 20, 2020 7:56 am

I tried to set THREADS to 8, but I it still only see 1 engine process
Martin Sedlak

User avatar
Rebel
Posts: 5157
Joined: Thu Aug 18, 2011 10:04 am

Re: NICE 1.2

Post by Rebel » Wed May 20, 2020 8:45 am

mar wrote:
Wed May 20, 2020 7:56 am
I tried to set THREADS to 8, but I it still only see 1 engine process
Then I misunderstood you. I tried Cheng 4.39, created test.bat

Code: Select all

del debug*.*

set MT=100
set HASH=128
set THREADS=1
set PROTOCOL=uci
set EPD=epd\sf1.epd
set OPTIONS=MultiPV=1
set MRL=x1

set EXE=engines\Cheng4_4.39.exe
set NAME=Cheng4_4.39

mea.exe --engine %EXE% --name %NAME% --hash %HASH% --threads %THREADS% --protocol %PROTOCOL% --epd %EPD% --movetime %MT% --log --eoption %OPTIONS%
csv %MRL% %EPD% %OPTIONS% %NAME% %MT% %CCRL%
mrl %MRL% %MRL% %EPD% %MT%

exit
1. Started NICE.
2. Select "Split Run"
3. Select "test.bat"
4. Speed up by threads -> I typed 8
5. Exit program
6. Double click "@start.bat"

The "sf1.epd" is split into 8 parts of 5,000 position and runs in 8 threads.

My task manager shows 8 x Cheng.
90% of coding is debugging, the other 10% is writing bugs.

mar
Posts: 2111
Joined: Fri Nov 26, 2010 1:00 pm
Location: Czech Republic
Full name: Martin Sedlak

Re: NICE 1.2

Post by mar » Wed May 20, 2020 9:00 am

Oh, my bad :oops: I only ran the original batch file directly, didn't run nice.exe to generate the split batch, sorry :)
Martin Sedlak

User avatar
Rebel
Posts: 5157
Joined: Thu Aug 18, 2011 10:04 am

Re: NICE 1.2

Post by Rebel » Wed May 20, 2020 3:01 pm

mar wrote:
Wed May 20, 2020 9:00 am
Oh, my bad :oops: I only ran the original batch file directly, didn't run nice.exe to generate the split batch, sorry :)
50 push-ups or one hour in the corner :D
90% of coding is debugging, the other 10% is writing bugs.

User avatar
JVMerlino
Posts: 1016
Joined: Wed Mar 08, 2006 9:15 pm
Location: San Francisco, California

Re: NICE 1.2

Post by JVMerlino » Mon May 25, 2020 1:11 am

Well I'm stumped. Either I can't make it work with Myrddin (which is an xboard engine) or NICE doesn't support xboard. Has anybody had success using NICE with any xboard engine?

User avatar
Rebel
Posts: 5157
Joined: Thu Aug 18, 2011 10:04 am

Re: NICE 1.2

Post by Rebel » Mon May 25, 2020 7:01 am

JVMerlino wrote:
Mon May 25, 2020 1:11 am
Well I'm stumped. Either I can't make it work with Myrddin (which is an xboard engine) or NICE doesn't support xboard. Has anybody had success using NICE with any xboard engine?
That is a question for Ferdy.

Meanwhile I did a quick run with Myrddin using the 1500 STS positions using the wb2uci adapter.

http://rebel13.nl/mea/Myrddin.html

Here is how.

The wb2uci.eng file

Code: Select all

[ENGINE]
Name = Myrddin_0.87
Author = John Merlino
Filename = wb2uci.exe

[OPTIONS]
Program = Myrddin_0.87.exe
InitString = BookOff/n
LevelExtend = none
Analyze = true
UseUndo = false
Logfile = false
LevelType = 2
SimulateHint = true
TerminateHard = false
; In WIN XP or Win NT  TerminateHard = true
Protocol = 1
Visible = Ponder,Extras
The batch file

Code: Select all

del debug*.*

set MT=1000
set HASH=128
set THREADS=1
set PROTOCOL=uci
set EPD=epd\sts.epd
set OPTIONS=MultiPV=1
set MRL=Myrddin

set EXE=engines\wb2uci.exe
set NAME=Myrddin_0.87

mea.exe --engine %EXE% --name %NAME% --hash %HASH% --threads %THREADS% --protocol %PROTOCOL% --epd %EPD% --movetime %MT% --log --eoption %OPTIONS%
csv %MRL% %EPD% %OPTIONS% %NAME% %MT% %CCRL%
mrl %MRL% %MRL% %EPD% %MT%

exit
Double click the batch file and it will run in one thread.
90% of coding is debugging, the other 10% is writing bugs.

User avatar
JVMerlino
Posts: 1016
Joined: Wed Mar 08, 2006 9:15 pm
Location: San Francisco, California

Re: NICE 1.2

Post by JVMerlino » Mon May 25, 2020 5:25 pm

It works - AND I found a (possible) bug in Myrddin! It does not accept "st" command values less than 1, because the protocol says it is used to "set the exact number of seconds per move". I guess, when I implemented that more than 10 years ago, I never considered decimal values. :oops: But the text of the protocol (and the example, which only uses the number 30) should probably be updated.

However, trying to do a Split Run via Myrddin.bat in NICE fails with "Error in line 8".

Thanks very much!

Post Reply