Open letter to chess programmers

[lucasart]

Thanks. I think the priority is i, it'd make the engine more useful to me.

I commited a patch to handle the stop command. If you get the latest source code from my git repo, it should work.

- it seems to work on Linux.
- it should work on any POSIX compliant operating system.
- according to the documentation I found on microsoft.com about the Windows APIs, it *should* work on Windows. but I didn't test it (I need to test it with Wine+Arena).

For skill level, how about using a nodes limit like:

Code: Select all

nodes = 2 ^ (8+skill_level)

in other words

Code: Select all

level 0 = 256 nodes
level 1 = 512 nodes
level 2 = 1024 nodes
...

I'm an extremely lazy programmer, so it looks like an appealing solution to me

[kinderchocolate]

Thanks. I think the priority is i, it'd make the engine more useful to me.

I commited a patch to handle the stop command. If you get the latest source code from my git repo, it should work.

- it seems to work on Linux.
- it should work on any POSIX compliant operating system.
- according to the documentation I found on microsoft.com about the Windows APIs, it *should* work on Windows. but I didn't test it (I need to test it with Wine+Arena).

For skill level, how about using a nodes limit like:

Code: Select all

nodes = 2 ^ (8+skill_level)

in other words

Code: Select all

level 0 = 256 nodes
level 1 = 512 nodes
level 2 = 1024 nodes
...

I'm an extremely lazy programmer, so it looks like an appealing solution to me

Thanks. I've seen the diffs I think your changes will work. The changes look very familiar, it was copied from Stockfish, wasn't it? I'll report to you if I have any questions.

[lucasart]

Thanks. I've seen the diffs I think your changes will work. The changes look very familiar, it was copied from Stockfish, wasn't it? I'll report to you if I have any questions.

No, this code was written by me:

* I wrote the POSIX version myself, simply looking at the documentation for the select() system call. select() is an extremely powerful function by the way. And what makes it even more powerful is that POSIX treats any kind of file descriptor the same (file on a disk, but also pipe, console, network socket, you name it...)
* The Windows version is based on Fruit as well as the documentation I could find on microsoft.com. I believe my implementation is somewhat better/simpler than the original Fruit one. The Windows case is a real mess, because the design of Windows is so flawed and stupid: Windows treats every kind of file descriptor completely differently, so there's no uniform way of doing this for all of them (console, pipe, file, etc.)

Stockfish does this in a completely different way, using threads:

The Stockfish method works as follows
* a master thread is parked in a loop that does a blocking I/O read from stdin
* once the data becomes available (if at all), the call returns and the master thread is scheduled by the operating system again. So it parses the result, and if it happens to be a "stop" then it sets a global variable Signals.Stop = true
* Then all the searching thread(s) observe that Signals.Stop variable, and when it's true, they know it's time to stop.

The Stockfish method is arguably better, because:
* it can be written only with portable code (at least in C11 or C++11)
* my implementation is potentially racy in the SMP case. But I don't care, as DiscoCheck is single threaded.
* it's also more elegant.

[lucasart]

The uci script uses go infinite and stop after a timeout, so there are only two possibilities:
- there is a problem with the script itself
- engines don't handle go infinite and/or stop properly

Thanks. That explains everything. DiscoCheck doesn't handle the "stop" command (because I hate writing ugly ifdef's and pateform specific code). However, it handles the "go movetime", so I can modify the script to do that instead.

On the other hand, I don't really care, as I'm really not interested in these tournaments. I was just curious as to what this fingerprint was.

Lucas, any possibility for DiscoCheck supporting infinite analysis and skills level?

I've just implemented the UCI_LimitStrength and UCI_Elo features. It's a bit of a sloppy method, but I used a "guesstimized" formula:

nodes = 2^(8+(ELO-1400)/100)

So doubling the node count adds 100 ELO, and 2^8=256 nodes corresponds to 1400 ELO. When I do statistics based on self-play, it seems that the increment of doubling the node count is around 180 ELO on average, but that is computer vs. computer and self-play. Against human it must be half less I guess.

Anywa, not great, but better than nothing.

[Laskos]

The uci script uses go infinite and stop after a timeout, so there are only two possibilities:
- there is a problem with the script itself
- engines don't handle go infinite and/or stop properly

Thanks. That explains everything. DiscoCheck doesn't handle the "stop" command (because I hate writing ugly ifdef's and pateform specific code). However, it handles the "go movetime", so I can modify the script to do that instead.

On the other hand, I don't really care, as I'm really not interested in these tournaments. I was just curious as to what this fingerprint was.

Lucas, any possibility for DiscoCheck supporting infinite analysis and skills level?

I've just implemented the UCI_LimitStrength and UCI_Elo features. It's a bit of a sloppy method, but I used a "guesstimized" formula:

nodes = 2^(8+(ELO-1400)/100)

So doubling the node count adds 100 ELO, and 2^8=256 nodes corresponds to 1400 ELO. When I do statistics based on self-play, it seems that the increment of doubling the node count is around 180 ELO on average, but that is computer vs. computer and self-play. Against human it must be half less I guess.

Anywa, not great, but better than nothing.

For diminishing returns something like 2^(8+((ELO-1400)/100)^1.1) might be more accurate (assuming ELO>1400).

[lucasart]

For diminishing returns something like 2^(8+((ELO-1400)/100)^1.1) might be more accurate (assuming ELO>1400).

I've done some measurements, and the diminishing return effect is not as clear as I thought. It really starts to kick in after a certain depth, probably due to move count and SEE pruning at very low depths. I got a goot fit with the following model:

* nodes = 2^(n+8) for n >= 0
* ELO(self-play) ~= 200*n^0.93

(ELO scale is obviously relative to an arbitrary constant. here I chose ELO=0 when n=0, that is to say for 256 nodes)

Then I would transform that into human ELO based on a pure guess:
* 256 nodes corresponds to 1400 ELO (?)
* computer vs computer (as opposed to computer vs human) combined with the fact that it's self-play, inflates ELO by a factor two

So I get:

ELO(human scale) = 1400 + 100*n^0.93

I'm still waiting for more test results to finish, as it takes longer for large values of n, obviously... I need to see how the diminishing return effect accelerates (or not) at n=11 and n=12. So far, I've measured n=0..10

[hgm]

I have a problem doing the similarity test. I downloaded the Windows package from the CSVN website, but when I run fingerprint.exe, it starts the engine (I see its process in the task manager), but it is not using any CPU, and fingerprint.exe seems to hang.

I also cannot compile the sources that came with it (produces error messages).

Rather than trying to debug a piece of software that I don't know, I can modify WinBoard to run the engine on the positions (it already does that, in match mode with a -loadPositionFile), and abort the 'games' after the first move, writing the result to a file.

But to do that, I must know what format I should save in. Can anyone show me a few lines of what is supposed to be in the output file?

[pijl]

The csvn test software uses the 'st' command for winboard engines. It also requires support of both 'ping' and 'setboard' (probably could have done without the 'ping-pong' but that made programming a bit easier). If you support these commands I can help if you provide me a protocol trace.

[tpetzke]

1r3rk1/4pp1p/2Bpn1p1/4q3/1PR5/3QP1P1/5P1P/2R3K1 w - - bm d3d2; c0 "<iCE 0.4 v1609 x32> score cp 79";
r1bqk2r/pp3pp1/2p2nnp/3p4/1b1P4/2N1PNB1/PPQ2PPP/2R1KB1R b Kkq - bm h6h5; c0 "<iCE 0.4 v1609 x32> score cp -37";
2rq1nk1/ppr1bppp/2n1p3/3pP3/3P1NQ1/1P1NB3/P4PPP/R2R2K1 w - - bm a1c1; c0 "<iCE 0.4 v1609 x32> score cp 0";

Richard told me that this epd format would be acceptable. Probably even without the c0 part. It is just there because because I'm also reusing some existing code.

Thomas...

[hgm]

The csvn test software uses the 'st' command for winboard engines. It also requires support of both 'ping' and 'setboard' (probably could have done without the 'ping-pong' but that made programming a bit easier). If you support these commands I can help if you provide me a protocol trace.

What do you mean by "protocol trace"? Is that a log of the GUI-engine communication? Does fingerprint.exe make such a log?

I didn't work on Spartacus for years, but I don't think it supports 'st' mode. That doesn't seem to explain why fingerprint.exe hangs, though, at worst it should make Spartacus forfeit on time (if fingerprint.exe keeps track of that).

I am feeding Spartacus the epd file now at 60 moves/min, which in practice causes it to search from 0.5 to 1.3 sec per position. The format is not entirely as Thomas shows it, WinBoard now saves it as:

Code: Select all

1r3rk1/4pp1p/2Bpn1p1/4q3/1PR5/3QP1P1/5P1P/2R3K1 w - - 0 1 bm c6d5
r1bqk2r/pp3pp1/2p2nnp/3p4/1b1P4/2N1PNB1/PPQ2PPP/2R1KB1R b Kkq - 0 1 bm h6h5
2rq1nk1/ppr1bppp/2n1p3/3pP3/3P1NQ1/1P1NB3/P4PPP/R2R2K1 w - - 0 1 bm f2f3

I can easily edit the " 0 1" away and add a ';' at the end. (As it is already half-way through the file that would be quicker than modifying the way WinBoard saves them and starting all over again.)