Stockfish Level Settings on LiChess

[BrendanJNorman]

Excuse the language, but this seems like a half-assed implementation. At the lower levels luck is going to be the main factor of how often it blunders and how bad the blunder is, so that by pure luck level 5 could play weaker than level 4, just completely inconsistent.

But the reality is, MOST OF THE TIME, the higher level plays stronger. This is the same as normal players.

Typically a 1700 player will defeat a 1600 player, yet sometimes there are upsets.

Stockfish naturally comes with a curve of strength, Depth 1 is weaker than Depth 2 which is weaker than Depth 3, which is weaker than Depth 4, which is weaker than Depth 5. It is a shame that they don't take advantage of this and instead set it to 5 and throw dice to decide how it plays.

It's their own interpretation. Their own experiment. I applaud them for it.

And from my own experience, the "blunders" at level 7 say, are of much higher level than the blunders at levels 1-3 as an example.

It's not all hanging a queen randomly.

A student could lose to level 1 and beat level 5, there's no progression if you're unlucky. Blegh.

Have you tried it?

I have close to ten students now regularly (daily) playing against the levels trying to climb (after black and white wins) to higher levels.

Sometimes they give a higher level a try for fun. And I can tell you, nobody who struggles to beat level 2 (as an example) can beat level 5.

And nobody who can beat level 5 has any trouble crushing levels 1-2.

Seems reasonable to me.


Anyway man, good to see you back here. I downloaded chi-chi today and its a fun toy. Well done.

[Ovyron]

Typically a 1700 player will defeat a 1600 player, yet sometimes there are upsets.

Yeah, but when due to luck a level plays at 1700 level and the NEXT one at 1600, then what kind of progression are we seeing? It's also possible human psyche is very involving in this (student is told she beat level 4 and faces level 5, so the increase in number makes it feel stronger, because she's told it is, even if by luck it plays worse.)

You're happy with "most of the time" where it would have been EASIER to to make it work "all the time."

Sometimes they give a higher level a try for fun.

If they knew how it worked they'd skip the first 4 levels and play level 5 until it plays worse by luck. Chances are good it will.

I don't need to "try it", I can read stockfish's source and see how it works, here's 3 ways to do it:

Fixed depth - the engine tries its best and is only limited by its search horizon, as the horizon increases the student can no longer beat it easily, but we know the blunders can be exploited - students would find "a depth they can't beat."

UCI levels - the engine throws dice to see where it blunders and how badly. This has already been designed by smart people that know what they're doing, delivering a progression in strength the higher the elo, while still having the possibility of upsets due to luck, but keeping it reasonable.

Lichess Levels - I know, what if we mix the concepts above and bring the worst of both worlds?? At least for levels 5 and lower, what happens next seems fine, though again, the difficulty spike seems too steep.

So have YOU tried the other two methods? It's not that it's unreasonable, it's that we already have 2 methods that are superior available.

Anyway man, good to see you back here. I downloaded chi-chi today and its a fun toy. Well done.

Thanks Brendan, glad to hear you still have students, over here the dream was cancelled since COVID hit in 2020. The next Chi-Chi is in the works, I don't know what to expect from NNUE powered material imbalance evaluation, but I think a super strong entity that thinks a knight or a bishop is more valuable than a rook could be achieved now.

[Odd Gunnar Malin]

Fixed depth - the engine tries its best and is only limited by its search horizon, as the horizon increases the student can no longer beat it easily, but we know the blunders can be exploited - students would find "a depth they can't beat."

UCI levels - the engine throws dice to see where it blunders and how badly. This has already been designed by smart people that know what they're doing, delivering a progression in strength the higher the elo, while still having the possibility of upsets due to luck, but keeping it reasonable.

If you play against an engine with fixed depth wouldn't this automatic make you play in a different way than you would do against human. A human see a lot more in simplified or endgame position than in complicated middlegame. A 2 move search in a complicated position could knock out a 1800 player while 2 move in the endgame is less than a 1000 player do.
How accurate is the Elo setting of Stockfish, I see it start on 1350 (ca. Skill 0), do a player below 1000 even have a chance.

(I have never tried to play against Stockfish)

[Ovyron]

A 2 move search in a complicated position could knock out a 1800 player while 2 move in the endgame is less than a 1000 player do.

2 moves is 4ply and 4ply is very dumb all around, so no 1800 player is going to get knocked by that. It'd be better to use 4ply instead of 5ply with randomness, because then it's all about the randomness.

The only reason you'd use this is because you don't want the engine to play the same moves every time and make in an exercise on memorizing what it plays and "dig" and "grind" until you find a winning combination. Randomness is supposed to help with that but Chessmaster already proved that false with its personalities and its opening book.

Heh, Chessmaster was already doing this perfectly 2 decades ago, and much better than any Stockfish depth or UCI setting or Lichess levels ever dream to do, because what people don't get is that it makes to sense to grab the strongest thing out there and cripple it down, Chessmaster had well thought out chess playing personalities and outstanding progression and didn't need to be 3400 elo at full strength. After trying that mode out in Chessmaster 9000 any other future attempt seemed lacking in comparison, so it's a problem that has been solved for a long while.

How accurate is the Elo setting of Stockfish,

As accurate as can be if you correlate the elo pools and stretch the ratings, it'll depend on the hardware but stockfish will use the same dice and same blunder magnitude depending on the level, so you can split it in parts and have the same progression by using a different scale.

I see it start on 1350 (ca. Skill 0), do a player below 1000 even have a chance.

That's why I suggested fixed depth, we start at depth 1, which a monkey cab beat, then we increase it until we find "the depth that can't be beaten", and ONLY then we apply the UCI ELO / randomness to it and mix it with depth, this point will depend on the student as stronger students can beat fixed depth without the randomness. Then they can move up to a higher depth without randomness.

(I have never tried to play against Stockfish)

I bet you can beat depths 1 to 3 easily on your first try.

[Uri Blass]

A 2 move search in a complicated position could knock out a 1800 player while 2 move in the endgame is less than a 1000 player do.

2 moves is 4ply and 4ply is very dumb all around, so no 1800 player is going to get knocked by that. It'd be better to use 4ply instead of 5ply with randomness, because then it's all about the randomness.

The only reason you'd use this is because you don't want the engine to play the same moves every time and make in an exercise on memorizing what it plays and "dig" and "grind" until you find a winning combination. Randomness is supposed to help with that but Chessmaster already proved that false with its personalities and its opening book.

Heh, Chessmaster was already doing this perfectly 2 decades ago, and much better than any Stockfish depth or UCI setting or Lichess levels ever dream to do, because what people don't get is that it makes to sense to grab the strongest thing out there and cripple it down, Chessmaster had well thought out chess playing personalities and outstanding progression and didn't need to be 3400 elo at full strength. After trying that mode out in Chessmaster 9000 any other future attempt seemed lacking in comparison, so it's a problem that has been solved for a long while.

How accurate is the Elo setting of Stockfish,

As accurate as can be if you correlate the elo pools and stretch the ratings, it'll depend on the hardware but stockfish will use the same dice and same blunder magnitude depending on the level, so you can split it in parts and have the same progression by using a different scale.

I see it start on 1350 (ca. Skill 0), do a player below 1000 even have a chance.

That's why I suggested fixed depth, we start at depth 1, which a monkey cab beat, then we increase it until we find "the depth that can't be beaten", and ONLY then we apply the UCI ELO / randomness to it and mix it with depth, this point will depend on the student as stronger students can beat fixed depth without the randomness. Then they can move up to a higher depth without randomness.

(I have never tried to play against Stockfish)

I bet you can beat depths 1 to 3 easily on your first try.

You assume that it is easy to beat depth 1 but I think that it is dependent on the evaluation function.
I remember reading in the past that lc0 plays very strong at 1 nodes per move(not even depth 1).

[Fritz 0]

A 2 move search in a complicated position could knock out a 1800 player while 2 move in the endgame is less than a 1000 player do.

2 moves is 4ply and 4ply is very dumb all around, so no 1800 player is going to get knocked by that. It'd be better to use 4ply instead of 5ply with randomness, because then it's all about the randomness.

The only reason you'd use this is because you don't want the engine to play the same moves every time and make in an exercise on memorizing what it plays and "dig" and "grind" until you find a winning combination. Randomness is supposed to help with that but Chessmaster already proved that false with its personalities and its opening book.

Heh, Chessmaster was already doing this perfectly 2 decades ago, and much better than any Stockfish depth or UCI setting or Lichess levels ever dream to do, because what people don't get is that it makes to sense to grab the strongest thing out there and cripple it down, Chessmaster had well thought out chess playing personalities and outstanding progression and didn't need to be 3400 elo at full strength. After trying that mode out in Chessmaster 9000 any other future attempt seemed lacking in comparison, so it's a problem that has been solved for a long while.

How accurate is the Elo setting of Stockfish,

As accurate as can be if you correlate the elo pools and stretch the ratings, it'll depend on the hardware but stockfish will use the same dice and same blunder magnitude depending on the level, so you can split it in parts and have the same progression by using a different scale.

I see it start on 1350 (ca. Skill 0), do a player below 1000 even have a chance.

That's why I suggested fixed depth, we start at depth 1, which a monkey cab beat, then we increase it until we find "the depth that can't be beaten", and ONLY then we apply the UCI ELO / randomness to it and mix it with depth, this point will depend on the student as stronger students can beat fixed depth without the randomness. Then they can move up to a higher depth without randomness.

(I have never tried to play against Stockfish)

I bet you can beat depths 1 to 3 easily on your first try.

Don't forget extensions (checks, captures and threats). So it's not exactly just 4 ply. For example, Komodo 14 level 16 is 4 ply search with randomness, and it's certainly no weaker than 1800 at 15+10 time control.

[Ovyron]

You assume that it is easy to beat depth 1

No, I know because I've done it. I can beat depth 1 every day of the week, and twice on fridays if I play it twice.

Actually, depth 1 has become weaker over the years, and you can find the stockfish patches that made it weaker. Patches that made stockfish lose against itself at hyperbullet, but made it stronger at longer time controls, made lower depth weaker.

You probably can find some stockfish that is the toughest at depth 1, it may have been the first NNUE versions or so.

For all effects and purposes I'm taking about depth 1 in one thread, before people claim that on a 64core depth 1 plays like depth 6 on one thread or whatever.

[Fritz 0]

A 2 move search in a complicated position could knock out a 1800 player while 2 move in the endgame is less than a 1000 player do.

2 moves is 4ply and 4ply is very dumb all around, so no 1800 player is going to get knocked by that. It'd be better to use 4ply instead of 5ply with randomness, because then it's all about the randomness.

The only reason you'd use this is because you don't want the engine to play the same moves every time and make in an exercise on memorizing what it plays and "dig" and "grind" until you find a winning combination. Randomness is supposed to help with that but Chessmaster already proved that false with its personalities and its opening book.

Heh, Chessmaster was already doing this perfectly 2 decades ago, and much better than any Stockfish depth or UCI setting or Lichess levels ever dream to do, because what people don't get is that it makes to sense to grab the strongest thing out there and cripple it down, Chessmaster had well thought out chess playing personalities and outstanding progression and didn't need to be 3400 elo at full strength. After trying that mode out in Chessmaster 9000 any other future attempt seemed lacking in comparison, so it's a problem that has been solved for a long while.

How accurate is the Elo setting of Stockfish,

As accurate as can be if you correlate the elo pools and stretch the ratings, it'll depend on the hardware but stockfish will use the same dice and same blunder magnitude depending on the level, so you can split it in parts and have the same progression by using a different scale.

I see it start on 1350 (ca. Skill 0), do a player below 1000 even have a chance.

That's why I suggested fixed depth, we start at depth 1, which a monkey cab beat, then we increase it until we find "the depth that can't be beaten", and ONLY then we apply the UCI ELO / randomness to it and mix it with depth, this point will depend on the student as stronger students can beat fixed depth without the randomness. Then they can move up to a higher depth without randomness.

(I have never tried to play against Stockfish)

I bet you can beat depths 1 to 3 easily on your first try.

Don't forget extensions (checks, captures and threats). So it's not exactly just 4 ply. For example, Komodo 14 level 16 is 4 ply search with randomness, and it's certainly no weaker than 1800 at 15+10 time control.

To support this claim, I ran a 10.000 games match between Komodo 14.1 level 16 and Dragon 3 1800 Elo. Komodo won by 29 Elo points.