Open letter to chess programmers

[Sven]

Where is a pipe ("|") in the evidence, during the execution spins like a propeller fan!

You reported this:

Already on the second day with the same screen:

Code: Select all

...
Engine search: 1/10000 |

and I took that for granted so I assumed it did not print anything after the "|".

In task manager each version of Capivara appears using a 25% of the Core2Quad.

That would indeed indicate that the problem is not a problem of the tool but of your engine not terminating the search.

Sven

[Sven]

Well I still don't find it. I can't find EPD anywhere on this website in fact. It would really help if they could write that page in english and put the EPD in the download package...

The page is in Dutch *and* English, and this is the current EPD download link

Sven

[lucasart]

It just hangs here, and doesn't produce anything. I see that 8 processes of my engine are running, but the output stream is empty

Code: Select all

$ ./run-simtest-uci ./Engines/discocheck_4.1 8 1 < ./simcsvn1.unix.epd

Did anyone manage to get it to work in Linux with an UCI engine ? How ?

[mar]

The uci script uses go infinite and stop after a timeout, so there are only two possibilities:
- there is a problem with the script itself
- engines don't handle go infinite and/or stop properly

[rvida]

I guess what would be the percentage of coincidence if we execute the fingerprint N times with the same engine.

Can this test give us a (somehow) error margin? Has anyone tried this test?

with ~8,000 positions, an error of ~ 0.5% 1 standard deviation, if I remember well.

I think that the SD is even lower for most engines (excluding Rybka 3 and related engines).

Some engines maintain persistent context between searches (eg. history tables, positional gain tables, etc). This might increase the error margin because every search becomes dependent on the previous one. And with random jitter in search time the error/randomness accumulates. Sending 'ucinewgame' before position setup helps in such cases (if the engine indeed clears all the state information when receiving this command).

just my 2c

Richard

[lucasart]

The uci script uses go infinite and stop after a timeout, so there are only two possibilities:
- there is a problem with the script itself
- engines don't handle go infinite and/or stop properly

Thanks. That explains everything. DiscoCheck doesn't handle the "stop" command (because I hate writing ugly ifdef's and pateform specific code). However, it handles the "go movetime", so I can modify the script to do that instead.

On the other hand, I don't really care, as I'm really not interested in these tournaments. I was just curious as to what this fingerprint was.

[laoliveirajr]

.

It would be possible to find a solution similar to the SIM03 - Similary tester version 03 (available on the download page of the Komodo) to generate fingerprints.

I believe SIM03 run well with almost all engines.

Missing only generate the file with 10,000 positions to run with SIM03 !!!

.

[kinderchocolate]

The uci script uses go infinite and stop after a timeout, so there are only two possibilities:
- there is a problem with the script itself
- engines don't handle go infinite and/or stop properly

Thanks. That explains everything. DiscoCheck doesn't handle the "stop" command (because I hate writing ugly ifdef's and pateform specific code). However, it handles the "go movetime", so I can modify the script to do that instead.

On the other hand, I don't really care, as I'm really not interested in these tournaments. I was just curious as to what this fingerprint was.

Lucas, any possibility for DiscoCheck supporting infinite analysis and skills level?

[lucasart]

The uci script uses go infinite and stop after a timeout, so there are only two possibilities:
- there is a problem with the script itself
- engines don't handle go infinite and/or stop properly

Thanks. That explains everything. DiscoCheck doesn't handle the "stop" command (because I hate writing ugly ifdef's and pateform specific code). However, it handles the "go movetime", so I can modify the script to do that instead.

On the other hand, I don't really care, as I'm really not interested in these tournaments. I was just curious as to what this fingerprint was.

Lucas, any possibility for DiscoCheck supporting infinite analysis and skills level?

OK: the next release will handle the stop command (hence infinite analysis).

As for skill level, you can already limit the strength by using a depth and/or nodes limit. As far as I know there are two ways of limiting ELO:
(i) quick and dirty: each skill level corresponds to a limit in terms of depth and/or nodes (probably a min depth and max nodes is best).
(ii) more proper: start by implementing multi-PV, and choose randomly from the various PVs, with parameters that effectively control the skill level.

Typically what happens is that:
(i) plays very strong positionally, dominates the newbie player, and makes a stupid mistake every now and then (due to low depth) that looses some games. not a very realistic way of simulating a weak player.
(ii) is a more balanced blend, playing moves that are both positionally inferior and tactically (but some blunder margin can avoid really stupid tactics).

But whether you do (i) or (ii), you will never make an engine simulate a weak player in a credible way. That's not how weak players play, and the computer will play random moves that don't follow any plan. A weak player will play with a plan: a stupid one, and sometimes sticking to the plan to stubbornly disregarding important things, but still with some plan.

Best way is to play against real humans on a chess server

Long story short, I may implement (i) or perhaps (ii) but I don't know when. Also there's the problem of calibrating those, so that the ELO value is somewhat realistic (this is a difficult, perhaps it could be tuned based on user feedback, but I don't see any reliable way of doing it).

[kinderchocolate]

The uci script uses go infinite and stop after a timeout, so there are only two possibilities:
- there is a problem with the script itself
- engines don't handle go infinite and/or stop properly

Thanks. That explains everything. DiscoCheck doesn't handle the "stop" command (because I hate writing ugly ifdef's and pateform specific code). However, it handles the "go movetime", so I can modify the script to do that instead.

On the other hand, I don't really care, as I'm really not interested in these tournaments. I was just curious as to what this fingerprint was.

Lucas, any possibility for DiscoCheck supporting infinite analysis and skills level?

OK: the next release will handle the stop command (hence infinite analysis).

As for skill level, you can already limit the strength by using a depth and/or nodes limit. As far as I know there are two ways of limiting ELO:
(i) quick and dirty: each skill level corresponds to a limit in terms of depth and/or nodes (probably a min depth and max nodes is best).
(ii) more proper: start by implementing multi-PV, and choose randomly from the various PVs, with parameters that effectively control the skill level.

Typically what happens is that:
(i) plays very strong positionally, dominates the newbie player, and makes a stupid mistake every now and then (due to low depth) that looses some games. not a very realistic way of simulating a weak player.
(ii) is a more balanced blend, playing moves that are both positionally inferior and tactically (but some blunder margin can avoid really stupid tactics).

But whether you do (i) or (ii), you will never make an engine simulate a weak player in a credible way. That's not how weak players play, and the computer will play random moves that don't follow any plan. A weak player will play with a plan: a stupid one, and sometimes sticking to the plan to stubbornly disregarding important things, but still with some plan.

Best way is to play against real humans on a chess server

Long story short, I may implement (i) or perhaps (ii) but I don't know when. Also there's the problem of calibrating those, so that the ELO value is somewhat realistic (this is a difficult, perhaps it could be tuned based on user feedback, but I don't see any reliable way of doing it).

Thanks. I think the priority is i, it'd make the engine more useful to me. Regards about calibration, the only thing you can do is paring the engine against humans. Anything else is just guessing but let's worry about it later : -)