What are Stockfish and Komodo estimated rating for these 5 depths ?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: Harvey Williamson, Dann Corbit, hgm

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
Chessqueen
Posts: 1116
Joined: Wed Sep 05, 2018 12:16 am
Full name: Nancy M Pichardo

What are Stockfish and Komodo estimated rating for these 5 depths ?

Post by Chessqueen » Fri Jul 24, 2020 4:06 am

1st Stockfish at depth 1 Please provide estimated rating = Komodo =
1st Stockfish at depth 2 Please provide estimated rating = Komodo =
1st Stockfish at depth 3 Please provide estimated rating = Komodo =
1st Stockfish at depth 4 Please provide estimated rating = Komodo =
1st Stockfish at depth 5 Please provide estimated rating = Komodo =


We all know why Stockfish is soo strong, because it is the Program that reaches the highest depth withing a certain time control, but has anybody done a 100 match games between Komodo vs Stockfish with depth set to 22 or 24 ?

lkaufman
Posts: 4413
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: What are Stockfish and Komodo estimated rating for these 5 depths ?

Post by lkaufman » Fri Jul 24, 2020 4:38 am

Chessqueen wrote:
Fri Jul 24, 2020 4:06 am
1st Stockfish at depth 1 Please provide estimated rating = Komodo =
1st Stockfish at depth 2 Please provide estimated rating = Komodo =
1st Stockfish at depth 3 Please provide estimated rating = Komodo =
1st Stockfish at depth 4 Please provide estimated rating = Komodo =
1st Stockfish at depth 5 Please provide estimated rating = Komodo =


We all know why Stockfish is soo strong, because it is the Program that reaches the highest depth withing a certain time control, but has anybody done a 100 match games between Komodo vs Stockfish with depth set to 22 or 24 ?
At one ply it's a close match, with a small edge for Komodo (may depend on which version of each is used), but at depths 2 thru 5 it is a gross mismatch in favor of Komodo, reaching hundreds of elo around depth 5 or so. At higher depths everything depends on whether you are talking about one thread or many threads. On one thread SF passes Komodo somewhere in the early teens but doesn't get a lot more depth in general, while on many threads SF gets a lot more depth in the same time but may be weaker at a given depth, depending on how many threads. So it's complicated.
Komodo rules!

User avatar
Ozymandias
Posts: 1243
Joined: Sun Oct 25, 2009 12:30 am

Re: What are Stockfish and Komodo estimated rating for these 5 depths ?

Post by Ozymandias » Fri Jul 24, 2020 7:17 am

lkaufman wrote:
Fri Jul 24, 2020 4:38 am
On one thread SF passes Komodo somewhere in the early teens
As a general rule of thumb, you need an extra ply for Komodo to equal SF at those depths. This, of course, comes at a time price (about 50%).

Chessqueen
Posts: 1116
Joined: Wed Sep 05, 2018 12:16 am
Full name: Nancy M Pichardo

Re: What are Stockfish and Komodo estimated rating for these 5 depths ?

Post by Chessqueen » Fri Jul 24, 2020 1:20 pm

lkaufman wrote:
Fri Jul 24, 2020 4:38 am
Chessqueen wrote:
Fri Jul 24, 2020 4:06 am
1st Stockfish at depth 1 Please provide estimated rating = Komodo =
1st Stockfish at depth 2 Please provide estimated rating = Komodo =
1st Stockfish at depth 3 Please provide estimated rating = Komodo =
1st Stockfish at depth 4 Please provide estimated rating = Komodo =
1st Stockfish at depth 5 Please provide estimated rating = Komodo =


We all know why Stockfish is soo strong, because it is the Program that reaches the highest depth withing a certain time control, but has anybody done a 100 match games between Komodo vs Stockfish with depth set to 22 or 24 ?
At one ply it's a close match, with a small edge for Komodo (may depend on which version of each is used), but at depths 2 thru 5 it is a gross mismatch in favor of Komodo, reaching hundreds of elo around depth 5 or so. At higher depths everything depends on whether you are talking about one thread or many threads. On one thread SF passes Komodo somewhere in the early teens but doesn't get a lot more depth in general, while on many threads SF gets a lot more depth in the same time but may be weaker at a given depth, depending on how many threads. So it's complicated.
My main question still remain what is the estimated rating for Depth 1. even if it is ridiculously low, and what depth you believe that Komodo will be competitive versus you or anybody close to your Elo? Last question regardless of what system it is used wether your 32 cores or a 4 cores like mine since once it reaches certain depth it will play the exact move. At what depth do you believe that GM Lenderman could be competitive with Komodo with all pieces NO Odds. ? We should start playing versus Komodo with different depths to find out or assign rating estimates to Komodo.

lkaufman
Posts: 4413
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: What are Stockfish and Komodo estimated rating for these 5 depths ?

Post by lkaufman » Fri Jul 24, 2020 3:29 pm

Chessqueen wrote:
Fri Jul 24, 2020 1:20 pm
lkaufman wrote:
Fri Jul 24, 2020 4:38 am
Chessqueen wrote:
Fri Jul 24, 2020 4:06 am
1st Stockfish at depth 1 Please provide estimated rating = Komodo =
1st Stockfish at depth 2 Please provide estimated rating = Komodo =
1st Stockfish at depth 3 Please provide estimated rating = Komodo =
1st Stockfish at depth 4 Please provide estimated rating = Komodo =
1st Stockfish at depth 5 Please provide estimated rating = Komodo =


We all know why Stockfish is soo strong, because it is the Program that reaches the highest depth withing a certain time control, but has anybody done a 100 match games between Komodo vs Stockfish with depth set to 22 or 24 ?
At one ply it's a close match, with a small edge for Komodo (may depend on which version of each is used), but at depths 2 thru 5 it is a gross mismatch in favor of Komodo, reaching hundreds of elo around depth 5 or so. At higher depths everything depends on whether you are talking about one thread or many threads. On one thread SF passes Komodo somewhere in the early teens but doesn't get a lot more depth in general, while on many threads SF gets a lot more depth in the same time but may be weaker at a given depth, depending on how many threads. So it's complicated.
My main question still remain what is the estimated rating for Depth 1. even if it is ridiculously low, and what depth you believe that Komodo will be competitive versus you or anybody close to your Elo? Last question regardless of what system it is used wether your 32 cores or a 4 cores like mine since once it reaches certain depth it will play the exact move. At what depth do you believe that GM Lenderman could be competitive with Komodo with all pieces NO Odds. ? We should start playing versus Komodo with different depths to find out or assign rating estimates to Komodo.
1. For Depth 1 ply, the choice of engine doesn't matter much, I usually quote 1200 as a rough guide for Komodo 1 ply, but I could be off 200 elo either way, this is not carefully measured. But for players with ratings below 1000, we have to weaken it further to give them a chance, so I think it's at least in the right ballpark..
2. You are wrong to say that number of cores/threads doesn't affect the play at fixed depth. It most certainly does on modern engines that use "lazy smp". The more threads used, the higher the quality at the same depth.
3. To talk about a rating for a given depth, you have to specify the time limit for the human. Are you talking about blitz, rapid, or 2 hours plus 30" per move? It makes a huge difference. We do have some data on this, especially for blitz. In fact, GM Lenderman played some 3' + 2" games with different Skill levels of Komodo on one thread, and the closest match was Skill 21. The top Skill levels are mostly defined by fixed depth, with some modest randomness thrown in, which becomes much more significant at lower levels. Skill 21 is basically an 8 ply search, so you can say that 8 ply Komodo is around 2650 FIDE BLITZ level. Nakamura in Blitz was pretty well matched with what is now Skill 23, a ten ply search, so 10 ply is around 2850 FIDE BLITZ level. I would guess that Lenderman playing 15' + 10" Rapid is around the same strength as Nakamura playing 3' + 2" blitz, so perhaps in 15' + 10" Rapid FIDE 2650 is a good match with 10 ply Komodo. We have no data regarding standard (2 hours + 30") games, but by extrapolation I can guess that Komodo 12 ply (Skill 24) would be a good match for a 2650 player. All of this assumes using just one thread.
Komodo rules!

Chessqueen
Posts: 1116
Joined: Wed Sep 05, 2018 12:16 am
Full name: Nancy M Pichardo

Re: What are Stockfish and Komodo estimated rating for these 5 depths ?

Post by Chessqueen » Fri Jul 24, 2020 4:04 pm

Chessqueen wrote:
Fri Jul 24, 2020 1:20 pm
lkaufman wrote:
Fri Jul 24, 2020 4:38 am
Chessqueen wrote:
Fri Jul 24, 2020 4:06 am
1st Stockfish at depth 1 Please provide estimated rating = Komodo =
1st Stockfish at depth 2 Please provide estimated rating = Komodo =
1st Stockfish at depth 3 Please provide estimated rating = Komodo =
1st Stockfish at depth 4 Please provide estimated rating = Komodo =
1st Stockfish at depth 5 Please provide estimated rating = Komodo =


We all know why Stockfish is soo strong, because it is the Program that reaches the highest depth withing a certain time control, but has anybody done a 100 match games between Komodo vs Stockfish with depth set to 22 or 24 ?
At one ply it's a close match, with a small edge for Komodo (may depend on which version of each is used), but at depths 2 thru 5 it is a gross mismatch in favor of Komodo, reaching hundreds of elo around depth 5 or so. At higher depths everything depends on whether you are talking about one thread or many threads. On one thread SF passes Komodo somewhere in the early teens but doesn't get a lot more depth in general, while on many threads SF gets a lot more depth in the same time but may be weaker at a given depth, depending on how many threads. So it's complicated.
My main question still remain what is the estimated rating for Depth 1. even if it is ridiculously low, and what depth you believe that Komodo will be competitive versus you or anybody close to your Elo? Last question regardless of what system it is used wether your 32 cores or a 4 cores like mine since once it reaches certain depth it will play the exact move. At what depth do you believe that GM Lenderman could be competitive with Komodo with all pieces NO Odds. ? We should start playing versus Komodo with different depths to find out or assign rating estimates to Komodo.
I lost at Depth4 then I replayed it by taking the Rook in move 28 with Check instead of taking the Bishop :roll:


User avatar
Ovyron
Posts: 4405
Joined: Tue Jul 03, 2007 2:30 am

Re: What are Stockfish and Komodo estimated rating for these 5 depths ?

Post by Ovyron » Fri Jul 24, 2020 6:06 pm

Note Stockfish takes a while to add stuff to its search for many iterations, so a real depth 1 is reached when the depth counter has already gone up. This makes it have one of the weakest depth 1s around (I'd like someone to report an even weaker depth 1 from a different engine.)

The problem with such low ratings is that results are very random, but some 800 elo would sound about right, e.g. any human that has just learned the rules of chess should be able to beat depth 1 consistently after a few games.

lkaufman
Posts: 4413
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: What are Stockfish and Komodo estimated rating for these 5 depths ?

Post by lkaufman » Fri Jul 24, 2020 6:50 pm

Ovyron wrote:
Fri Jul 24, 2020 6:06 pm
Note Stockfish takes a while to add stuff to its search for many iterations, so a real depth 1 is reached when the depth counter has already gone up. This makes it have one of the weakest depth 1s around (I'd like someone to report an even weaker depth 1 from a different engine.)

The problem with such low ratings is that results are very random, but some 800 elo would sound about right, e.g. any human that has just learned the rules of chess should be able to beat depth 1 consistently after a few games.
That is far from true. The lower levels of Komodo had to be further crippled beyond their strength at one ply in order for thousands of chess.com players to have a chance against them. A player rated 800 is typically an 8 year old kid who gives away pieces with almost every move, far worse than a one ply search, which doesn't make "obvious" blunders (obvious to a novice). Stockfish isn't much worse than Komodo at 1 ply; it is much worse at 2 thru 6 ply. All engines see pretty much the same stuff at one ply, with details differing regarding whether they see quiet checks in reply. I would bet that a one ply search would score more than 50% against a randomly selected group of humans who replied "yes" when asked if they knew how to play chess.
Komodo rules!

User avatar
Ovyron
Posts: 4405
Joined: Tue Jul 03, 2007 2:30 am

Re: What are Stockfish and Komodo estimated rating for these 5 depths ?

Post by Ovyron » Fri Jul 24, 2020 7:02 pm

I see. I find really strange that the lowest FIDE rating that can be achieved is 1000 and starting people get higher than that already. My students stop dropping pieces after a few games and have no problem getting to a higher level than Stockfish's depth 1, it's surprising 800 elo players at chess.com keep dropping pieces and never improve beyond depth 1's strength. Maybe mentoring is necessary and what I observe isn't typical (because nobody is there to explain to them why they lost.)

lkaufman
Posts: 4413
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: What are Stockfish and Komodo estimated rating for these 5 depths ?

Post by lkaufman » Fri Jul 24, 2020 8:07 pm

Ovyron wrote:
Fri Jul 24, 2020 7:02 pm
I see. I find really strange that the lowest FIDE rating that can be achieved is 1000 and starting people get higher than that already. My students stop dropping pieces after a few games and have no problem getting to a higher level than Stockfish's depth 1, it's surprising 800 elo players at chess.com keep dropping pieces and never improve beyond depth 1's strength. Maybe mentoring is necessary and what I observe isn't typical (because nobody is there to explain to them why they lost.)
Are your students adults, teenagers, or kids? I know few adults with ratings like 800 (these would have to be USCF or online ratings since FIDE doesn't go that low); when I was actively teaching lots of elementary school kids a decade ago an 800 rating was usually typical of maybe the third best kid in the school, someone who needed about queen and knight odds from me for a fair game. What is the lowest FIDE-rated student you have who has actually beaten Stockfish at depth 1, and what age?
Komodo rules!

Post Reply