Page 1 of 1

question about weak level of engines

Posted: Sun Oct 06, 2019 9:05 pm
by Uri Blass
I prefer the weak levels to have the following properties:
1)They are not deterministic so humans cannot learn a fixed way to beat them.
2)Their playing strength is fixed and not dependent on hardware or time control.

Are there engines with weak levels that have these properties when the levels are very weak but still stronger than the random mover?

The target is to test beginners and give them initial rating based on their performance against the weak levels.

Re: question about weak level of engines

Posted: Sun Oct 06, 2019 9:56 pm
by Branko Radovanovic
Uri Blass wrote:
Sun Oct 06, 2019 9:05 pm
I prefer the weak levels to have the following properties:
1)They are not deterministic so humans cannot learn a fixed way to beat them.
2)Their playing strength is fixed and not dependent on hardware or time control.

Are there engines with weak levels that have these properties when the levels are very weak but still stronger than the random mover?
Absolutely makes sense. Personally, I don't care for the "easy" levels that actually depend on hardware or TC, that doesn't make any sense to me.

Back to your question: shouldn't most engines (weaker ones in particular) that support either UCI_Elo or node-based levels theoretically satisfy the above two conditions? Elo in the 800-1000 range should be easy to get, although I'm not sure about playing strength below that level. Very weak but stronger than random mover is about Elo 400 or thereabouts - never actually tried it.

Also, I'd like to see one more property:
3) Their style of play (blunders in particular) is reasonably similar to a that of a weak human.

This turns out to be much harder to satisfy.

Re: question about weak level of engines

Posted: Mon Oct 07, 2019 12:28 am
by bob
You have to be careful when talking about randomness. I have/had a very good "skill" level mechanism in Crafty. Quite a few years ago someone complained that the skill 0 or 1 setting was still playing at 2000 Elo level. Turns out the "Don Beal Effect" was clearly showing its head there. That is, if you just use random evaluations, positions with high mobility have a better chance of getting a larger score and backing it up compared to positions with little mobility that don't get the larger random numbers often enough to help.

I had to go back and do two things:

(1) make the eval more and more random, as before;

(2) but also turn down search extensions and greatly limit the search depth. I ended up using a "time burner" loop so that the program would not just move instantly.

Once I did both, the complaints went away as the skill 1 version played pretty badly. Would not say it was an 800 or worse, but it was very bad compared to the normal Crafty skill level.

Re: question about weak level of engines

Posted: Mon Oct 07, 2019 3:15 am
by Ovyron
Uri Blass wrote:
Sun Oct 06, 2019 9:05 pm
I prefer the weak levels to have the following properties:
1)They are not deterministic so humans cannot learn a fixed way to beat them.
2)Their playing strength is fixed and not dependent on hardware or time control.
Rebel Decade 3.0 had the properties you mention. To boot, it had a mechanism that would delay the engine's moves, because other chess programs of the time would play instantly and this wasn't natural, so Rebel would come up with a move instantly but still take its time to play it.

Note this might not have been true on Fridays, where it played at the strength of Rebel 10, so only use it the rest of the week.

Re: question about weak level of engines

Posted: Mon Oct 07, 2019 4:31 am
by MikeB
Uri Blass wrote:
Sun Oct 06, 2019 9:05 pm
I prefer the weak levels to have the following properties:
1)They are not deterministic so humans cannot learn a fixed way to beat them.
2)Their playing strength is fixed and not dependent on hardware or time control.

Are there engines with weak levels that have these properties when the levels are very weak but still stronger than the random mover?

The target is to test beginners and give them initial rating based on their performance against the weak levels.
You might want check out my next release of Honey - it will have 1900 level settings - from FIDE 1000 to FIDE 2900 , and one may select any Elo within that range. The ratings were first synced to CCRL and then then converted to FIDE via formula proposed by Kai. It will clearly resemble FIDE more closely than CCRL rated engines at the lower levels. At higher levels, say around FIDE 2700 , FIDE and CCRL are pretty close to each other anyway.

It features a non busy sleep so moves are not played instantly and is hardware independent. It has a UCI setting to adjust engine Knps to your computer. So whether you play on your computer or your Pi, the levels will be close to each other. It if does not play with sufficient variety for your tastes, one can select the variety option and it will still play close to the selected FIDE level as the nps are increased to make up for the decrease in Elo due to variety play. It also has both, Lc0 like scoring percentage evaluation or one can select the standard centipawn evaluation.

Also, a second version , Honey-fd ( fd == fortress detection) will also be made available for download , It was designed to to be used when a position appears to be fortress or potential fortress. Full credit for this code snippet goes to Joe Ellis, the author of Matefinder and Crystal.

Example:


Code: Select all

info depth 34 seldepth 75 multipv 1 score cp 8271 nodes 563000795 nps 23811571 hashfull 503 tbhits 0 time 23644 pv g3g4 h5g4 h4h5 d8h8 e1h4 d7f7 h4g5 e6d7 g5g6 d7c7 h2e2 h8h7 e5e6 f7e7 e2e1 c7b6 g6f5 g4g3 f5f6 g3g2 f4f5 e7e8 f6g5 h7h8 e1g1 h8g8 g5h4 g8f8 h4h3 e8e7 c1d2 e7h7 h3g4 f8h8 f5f6 h7ht
Note - Fortress detections currently costs about 60 to 70 Elo at a very fast micro bullet time control of 15 sec with .25 sec increment and less Elo with longer time controls. But obviously ,if you use it for analysis, that does not matter.

Hoping to release by end of this week, maybe Friday.

Re: question about weak level of engines

Posted: Mon Oct 07, 2019 5:18 am
by Uri Blass
MikeB wrote:
Mon Oct 07, 2019 4:31 am
Uri Blass wrote:
Sun Oct 06, 2019 9:05 pm
I prefer the weak levels to have the following properties:
1)They are not deterministic so humans cannot learn a fixed way to beat them.
2)Their playing strength is fixed and not dependent on hardware or time control.

Are there engines with weak levels that have these properties when the levels are very weak but still stronger than the random mover?

The target is to test beginners and give them initial rating based on their performance against the weak levels.
You might want check out my next release of Honey - it will have 1900 level settings - from FIDE 1000 to FIDE 2900 , and one may select any Elo within that range. The ratings were first synced to CCRL and then then converted to FIDE via formula proposed by Kai. It will clearly resemble FIDE more closely than CCRL rated engines at the lower levels. At higher levels, say around FIDE 2700 , FIDE and CCRL are pretty close to each other anyway.

It features a non busy sleep so moves are not played instantly and is hardware independent. It has a UCI setting to adjust engine Knps to your computer. So whether you play on your computer or your Pi, the levels will be close to each other. It if does not play with sufficient variety for your tastes, one can select the variety option and it will still play close to the selected FIDE level as the nps are increased to make up for the decrease in Elo due to variety play. It also has both, Lc0 like scoring percentage evaluation or one can select the standard centipawn evaluation.

Also, a second version , Honey-fd ( fd == fortress detection) will also be made available for download , It was designed to to be used when a position appears to be fortress or potential fortress. Full credit for this code snippet goes to Joe Ellis, the author of Matefinder and Crystal.

Example:


Code: Select all

info depth 34 seldepth 75 multipv 1 score cp 8271 nodes 563000795 nps 23811571 hashfull 503 tbhits 0 time 23644 pv g3g4 h5g4 h4h5 d8h8 e1h4 d7f7 h4g5 e6d7 g5g6 d7c7 h2e2 h8h7 e5e6 f7e7 e2e1 c7b6 g6f5 g4g3 f5f6 g3g2 f4f5 e7e8 f6g5 h7h8 e1g1 h8g8 g5h4 g8f8 h4h3 e8e7 c1d2 e7h7 h3g4 f8h8 f5f6 h7ht
Note - Fortress detections currently costs about 60 to 70 Elo at a very fast micro bullet time control of 15 sec with .25 sec increment and less Elo with longer time controls. But obviously ,if you use it for analysis, that does not matter.

Hoping to release by end of this week, maybe Friday.
I would like levels below Fide rating 1000 but it is clear that the formula
Elo FIDE = (0.7 x Elo CCRL) + 840 is not correct for the low levels.

Re: question about weak level of engines

Posted: Mon Oct 07, 2019 5:25 am
by MikeB
Uri Blass wrote:
Mon Oct 07, 2019 5:18 am
MikeB wrote:
Mon Oct 07, 2019 4:31 am
Uri Blass wrote:
Sun Oct 06, 2019 9:05 pm
I prefer the weak levels to have the following properties:
1)They are not deterministic so humans cannot learn a fixed way to beat them.
2)Their playing strength is fixed and not dependent on hardware or time control.

Are there engines with weak levels that have these properties when the levels are very weak but still stronger than the random mover?

The target is to test beginners and give them initial rating based on their performance against the weak levels.
You might want check out my next release of Honey - it will have 1900 level settings - from FIDE 1000 to FIDE 2900 , and one may select any Elo within that range. The ratings were first synced to CCRL and then then converted to FIDE via formula proposed by Kai. It will clearly resemble FIDE more closely than CCRL rated engines at the lower levels. At higher levels, say around FIDE 2700 , FIDE and CCRL are pretty close to each other anyway.

It features a non busy sleep so moves are not played instantly and is hardware independent. It has a UCI setting to adjust engine Knps to your computer. So whether you play on your computer or your Pi, the levels will be close to each other. It if does not play with sufficient variety for your tastes, one can select the variety option and it will still play close to the selected FIDE level as the nps are increased to make up for the decrease in Elo due to variety play. It also has both, Lc0 like scoring percentage evaluation or one can select the standard centipawn evaluation.

Also, a second version , Honey-fd ( fd == fortress detection) will also be made available for download , It was designed to to be used when a position appears to be fortress or potential fortress. Full credit for this code snippet goes to Joe Ellis, the author of Matefinder and Crystal.

Example:


Code: Select all

info depth 34 seldepth 75 multipv 1 score cp 8271 nodes 563000795 nps 23811571 hashfull 503 tbhits 0 time 23644 pv g3g4 h5g4 h4h5 d8h8 e1h4 d7f7 h4g5 e6d7 g5g6 d7c7 h2e2 h8h7 e5e6 f7e7 e2e1 c7b6 g6f5 g4g3 f5f6 g3g2 f4f5 e7e8 f6g5 h7h8 e1g1 h8g8 g5h4 g8f8 h4h3 e8e7 c1d2 e7h7 h3g4 f8h8 f5f6 h7ht
Note - Fortress detections currently costs about 60 to 70 Elo at a very fast micro bullet time control of 15 sec with .25 sec increment and less Elo with longer time controls. But obviously ,if you use it for analysis, that does not matter.

Hoping to release by end of this week, maybe Friday.
I would like levels below Fide rating 1000 but it is clear that the formula
Elo FIDE = (0.7 x Elo CCRL) + 840 is not correct for the low levels.
You are correct - but it would be incremental and it is pretty weak, If you need weaker, turn adaptive play or variety or both on.
About 1200 and up is correct, not 100% perfect , but reasonable.

Re: question about weak level of engines

Posted: Mon Oct 07, 2019 6:04 am
by lkaufman
Uri Blass wrote:
Sun Oct 06, 2019 9:05 pm
I prefer the weak levels to have the following properties:
1)They are not deterministic so humans cannot learn a fixed way to beat them.
2)Their playing strength is fixed and not dependent on hardware or time control.

Are there engines with weak levels that have these properties when the levels are very weak but still stronger than the random mover?

The target is to test beginners and give them initial rating based on their performance against the weak levels.
The skill levels of Komodo have both of the properties you mention, assuming you set "Threads" to 1 or to any constant value.

Re: question about weak level of engines

Posted: Mon Oct 07, 2019 7:51 am
by hgm
N.E.G. is much stronger than a random mover (it should beat one by nearly 99% or so), and highly randomizes its moves. That is, when there is something to randomize: if it can (re)capture in only one way, it will of course do it, and not consider moves that throw away material just to randomize. But that leaves plenty of opportunity to randomize.

People told me it makes a very entertaining sparring partner for children that just learned the game.

Re: question about weak level of engines

Posted: Fri Oct 11, 2019 9:13 pm
by Uri Blass
I found that n.e.g could beat some weak level of a free application in my phone.

This is against level 2 when I played manually the moves in the phone.

This is the game against level 3 when Neg lost



The email of the developer of the application is
Support@aifactory.co.uk

address
23 The avenue,Hatch End Middlesex Ha5 4EN United Kingdom

They claim level 1 is 750 elo level 2 is 920 elo level 3 is 1122 elo level 4 is 1235 elo level 5 is 1352 elo level 6 is 1435 elo...
level 12 is 2100 elo.

Note that my guess is the real elo is smaller at least for the low levels.

I decided to play a short game against level 12 and did not think much about my moves and the thing forced a draw by a sacrifice.