How to reduce elo in Crafty 22.1 JA?

[gerold]

Using Crafty 22.1 JA 32 bit in Arena. How to reduce elo ?
Thanks in advance.

Gerold.

[Jim Ablett]

Using Crafty 22.1 JA 32 bit in Arena. How to reduce elo ?
Thanks in advance.

Gerold.

Hello Gerold,

You use the new 'skill' command (add it to 'crafty.rc').
e.g skill 50

Quote from Bob Hyatt's post http://64.68.157.89/forum/viewtopic.php ... 86&t=21264

It is a number between 1 and 100, where 100 is normal, and anything lower represents a degraded level of play...

Let's take skill 50 as an example. First thing this does is reduce all extensions and reductions by 50%, which reduces tactical skill. This includes things like check extensions, null-move and LRM type reductions, etc. It then reduces the evaluation numbers by 50% as well, and adds in a random value that is 50% of the value of a pawn.

I then ran a ton of cluster games to sort of calibrate what skill does. Roughly, skill 70 reduces the playing level by about 200 Elo. Skill 50, reduces it by 400 Elo. By the time you get to 10 or so it is playing real chess, but playing with almost no sense of material or positional values... Note that a random eval still plays something that looks like chess because it is a poor-man's way of maintaining a mobility evaluation term...

Jim.

[bob]

Using Crafty 22.1 JA 32 bit in Arena. How to reduce elo ?
Thanks in advance.

Gerold.

If it is compiled with -DSKILL as an option, all you need to do is to use "skill n" where n is between 1 and 100. 100 is normal level. A rough estimate from cluster testing suggests 70 will drop it by 200 Elo, 50 will drop it by 400 Elo, and as skill goes down so does the Elo, although you might notice it is somewhat non-linear as currently implemented. Skill 1 is _really_ bad and I would probably not go below 10 as it plays pretty weakly down there.

If you try it and find any particular setting you like, feel free to make a suggestion like "patzer = skill 25" or whatever and I can introduce some symbolic names as well so you could add "skill patzer" to your crafty.rc file or send it as a start-up command, or even make it part of the personality feature so it could be declared as the default skill setting (personality does not include skill at the moment but could).

One proviso is that I tried to make this fairly hardware independent. The basic idea is this:

As skill drops, the search tricks are trimmed back. For example, skill 50 reduces all search extensions and reductions by 50%. At that level, checks extend by 1/2 ply rather than 1 ply. Null-move reduces by 1 ply rather than 2. Etc. Also, as skill drops, the positional evaluation is dropped by that same percentage, and a random value between zero and one pawn is added. At skill 50, you get this:

eval = real_score * 50 / 100 + random_score * 50 / 100

at skill 10, you get this:

eval = real_score * 10 / 100 + random_score * 90 / 100

Notice real eval goes down, random component goes up.

Another minor issue:

search extensions/reductions are in units of 1/4, which means that skill 100 uses full reductions and extensions, but anything from 75-99 uses 3/4 of the extension/reduction amount. Skill 50-74 uses 50%. So it is a little jumpy. The scores become more random uniformly from 1-100 skill settings, but not the search degradation. As a result, you will see a more significant change as you drop skill to 99, or 74, or 49, or 24, as those are the points where the extensions take a big hit by 1/4 a unit.

Hope that helps. I will continue to work on this, but it can fix crafty so that anyone can beat it, without making it play completely silly-looking chess. And note that a pure random eval still plays a somewhat reasonable level of chess because of how that turns into a mobility-type eval unexpectedly...

[Gregory Owett]

That is not its strength, but the number of Mo he uses that I would like to reduce.

Indeed, when he plays a series of games, his hash becomes gradually increasingly great, unless I change anything ! Eg. it starts with 128 and there are over 900 after a few parties.

Is this normal?

[gerold]

Thanks Jim and Robert for the reply. That should keep me
busy for a little while.

Gerold.

[Dirt]

Hope that helps. I will continue to work on this, but it can fix crafty so that anyone can beat it, without making it play completely silly-looking chess. And note that a pure random eval still plays a somewhat reasonable level of chess because of how that turns into a mobility-type eval unexpectedly...

Just to make sure it works I just played a game at 1%. Unfortunately for degraded Crafty its speculative queen sacrifice didn't work out! I though the evaluations were interesting, as being down a queen and a bishop for a pawn is apparently worth about 100 centipawns.

I think appending the skill setting (if less than 100%) to the name of the engine when reporting to the GUI might be a nice touch. You wouldn't want people to forget that they aren't playing Crafty at full strength.

[bob]

That is not its strength, but the number of Mo he uses that I would like to reduce.

Indeed, when he plays a series of games, his hash becomes gradually increasingly great, unless I change anything ! Eg. it starts with 128 and there are over 900 after a few parties.

Is this normal?

Not sure what you mean. Crafty does not adjust its hash table size after each game, whatever you set is what it uses, unless you use the "adaptive" command, which was primarily designed for a chess server.

hash table size should remain constant unless the GUI is telling it to make it bigger for unknown reasons.

[bob]

Hope that helps. I will continue to work on this, but it can fix crafty so that anyone can beat it, without making it play completely silly-looking chess. And note that a pure random eval still plays a somewhat reasonable level of chess because of how that turns into a mobility-type eval unexpectedly...

Just to make sure it works I just played a game at 1%. Unfortunately for degraded Crafty its speculative queen sacrifice didn't work out! I though the evaluations were interesting, as being down a queen and a bishop for a pawn is apparently worth about 100 centipawns.

I think appending the skill setting (if less than 100%) to the name of the engine when reporting to the GUI might be a nice touch. You wouldn't want people to forget that they aren't playing Crafty at full strength.

Good idea. And not to forget, I mentioned that "skill 1" was a tad on the far end of the playable extreme. I played a bunch (bunch = tens of thousands) of cluster games against a group of computer opponents to get an idea of how the skill command worked. If Crafty played even with two opponents at skill 100, I kept reducing skill level until it lost 3 of every 4 games, which is about 200 rating points.

But remember that this is against computers. It is more than likely that skill 70 is 200 Elo weaker against computers, but perhaps significantly less weak against humans. Computers tend to exaggerate rating differences between programs or versions, as opposed to ratings against humans where the tactical skill reduction might not be as important.

[gerold]

Hope that helps. I will continue to work on this, but it can fix crafty so that anyone can beat it, without making it play completely silly-looking chess. And note that a pure random eval still plays a somewhat reasonable level of chess because of how that turns into a mobility-type eval unexpectedly...

Just to make sure it works I just played a game at 1%. Unfortunately for degraded Crafty its speculative queen sacrifice didn't work out! I though the evaluations were interesting, as being down a queen and a bishop for a pawn is apparently worth about 100 centipawns.

I think appending the skill setting (if less than 100%) to the name of the engine when reporting to the GUI might be a nice touch. You wouldn't want people to forget that they aren't playing Crafty at full strength.

Good idea. And not to forget, I mentioned that "skill 1" was a tad on the far end of the playable extreme. I played a bunch (bunch = tens of thousands) of cluster games against a group of computer opponents to get an idea of how the skill command worked. If Crafty played even with two opponents at skill 100, I kept reducing skill level until it lost 3 of every 4 games, which is about 200 rating points.

But remember that this is against computers. It is more than likely that skill 70 is 200 Elo weaker against computers, but perhaps significantly less weak against humans. Computers tend to exaggerate rating differences between programs or versions, as opposed to ratings against humans where the tactical skill reduction might not be as important.

Very good. This is interesting feature.I put it at skill 50. Makes
for a good game for players less than 1400 elo.

Thanks Robert,

Gerold.

P.S. Be good trainning for openings using large opening book.

[Dirt]

Very good. This is interesting feature.I put it at skill 50. Makes
for a good game for players less than 1400 elo.

Thanks Robert,

Gerold.

P.S. Be good trainning for openings using large opening book.

Just to see what it takes to get Crafty down to the level of my favorite patsy:

Code: Select all

2 Crafty-22.1 JA (4%)   1220  161  128    15   80%  1000   27%
3 Chad's Chess V015     1000   91   90    41   50%  1035   22%
4 Crafty-22.1 JA (3%)    854  158  190    10   30%  1000   20%
6 Crafty-22.1 JA (2%)    644  218  468     6    8%  1000   17%