Perfect chess engine elo ( 32 men TB) can be within 200 of Stocfish in Tcec LTC conditions

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

User avatar
Ozymandias
Posts: 1532
Joined: Sun Oct 25, 2009 2:30 am

Re: Perfect chess engine elo ( 32 men TB) can be within 200 of Stocfish in Tcec LTC conditions

Post by Ozymandias »

Ideas like leaving out opening books, or forcing some lines to be played, render results useless in terms of Elo strength.

Elo measures the difference between two players from the starting position. Yes, there have been rated games in the past, even in thematic tournaments (like correspondence) but it was a minority of games. Computer chess is used to start from a set of openings and it was mostly ok, as long as the number of positions was high. This is not the case at TCEC. Talking about Elo figures makes little sense in these conditions.
Alayan
Posts: 550
Joined: Tue Nov 19, 2019 8:48 pm
Full name: Alayan Feh

Re: Perfect chess engine elo ( 32 men TB) can be within 200 of Stocfish in Tcec LTC conditions

Post by Alayan »

Ozymandias wrote: Fri Nov 06, 2020 12:04 pm Ideas like leaving out opening books, or forcing some lines to be played, render results useless in terms of Elo strength.

Elo measures the difference between two players from the starting position. Yes, there have been rated games in the past, even in thematic tournaments (like correspondence) but it was a minority of games. Computer chess is used to start from a set of openings and it was mostly ok, as long as the number of positions was high. This is not the case at TCEC. Talking about Elo figures makes little sense in these conditions.
There is no reason it has to be from the starting position. What is needed is well-defined conditions.

Engine X on hardware Y with time control T and a set of forced openings B.

When you try to change conditions and carry over strength estimates, it can quickly become useless and misleading.
Madeleine Birchfield
Posts: 512
Joined: Tue Sep 29, 2020 4:29 pm
Location: Dublin, Ireland
Full name: Madeleine Birchfield

Re: Perfect chess engine elo ( 32 men TB) can be within 200 of Stocfish in Tcec LTC conditions

Post by Madeleine Birchfield »

hgm wrote: Fri Nov 06, 2020 11:49 am EGT often play moronically stupid in drawn positions. An engine playing only from a 32-men EGTB would of course never lose, but it would also virtually never win. In all its games it would very quickly built up a nearly-losing disadvatage. And only then would it start to fight for a draw, which it would reach through very clever moves. Such an engine would probably not even be at the top of the current CCRL list. It would never lose, but the fact that it would also never win against 2500-rated opponents would severely depress its rating.

EGT are not the answer to everything. You need some form of opponent modeling to select good move in drawn positions. Merely trying to make the game last long in the hope the opponent will err won't do it. It would just lead to pointless shuffling of its own pieces, trying to minimize Pawn moves to keep them for resetting the ply counter each time it approaches 100. Just like minimizing the number of non-losing replies also is silly: that would just lead to quickly trading everything, because it can be recaptured in only few ways, and if the opponent forgets to recapture, you have won. But of course there is zero chance that he will forget to recapture. You somehow have to be able to seek out positions where the opponent is likely to make an error.
I think the top rated 8x8 English draughts engines have this problem; it is impossible for the engine to lose, but it might accidentally convert a win into a draw from time to time.
JohnWoe
Posts: 491
Joined: Sat Mar 02, 2013 11:31 pm

Re: Perfect chess engine elo ( 32 men TB) can be within 200 of Stocfish in Tcec LTC conditions

Post by JohnWoe »

With 32 men EGTB you still need some heuristics so it would play the hardest moves. Keep lots of pieces on the board. Prolong the game maximum. Little bit of noise here and there Etc.

Instead of something random that gives easy time. If all moves draw except 1. g4 then 1. a4 would give an easy draw for SF NNUE no matter follow up.

In my opinion Stockfish NNUE dev. With a massive book (a few TiBs). Wouldn't lose many games against 32 men EGTB. Propably 45%-50% score due to some bugs? and too much pruning? Maybe it's 100% draws already who knows.
User avatar
Ozymandias
Posts: 1532
Joined: Sun Oct 25, 2009 2:30 am

Re: Perfect chess engine elo ( 32 men TB) can be within 200 of Stocfish in Tcec LTC conditions

Post by Ozymandias »

Alayan wrote: Fri Nov 06, 2020 12:25 pmThere is no reason it has to be from the starting position.
There isn't? What was the purpose of the Elo rating system as it was created?
Raphexon
Posts: 476
Joined: Sun Mar 17, 2019 12:00 pm
Full name: Henk Drost

Re: Perfect chess engine elo ( 32 men TB) can be within 200 of Stocfish in Tcec LTC conditions

Post by Raphexon »

Ozymandias wrote: Fri Nov 06, 2020 3:00 pm
Alayan wrote: Fri Nov 06, 2020 12:25 pmThere is no reason it has to be from the starting position.
There isn't? What was the purpose of the Elo rating system as it was created?
To better predict the strength of (human) chess players than previous rating systems.
BrendanJNorman
Posts: 2526
Joined: Mon Feb 08, 2016 12:43 am
Full name: Brendan J Norman

Re: Perfect chess engine elo ( 32 men TB) can be within 200 of Stocfish in Tcec LTC conditions

Post by BrendanJNorman »

duncan wrote: Fri Nov 06, 2020 10:20 am
BrendanJNorman wrote: Fri Nov 06, 2020 9:58 am Talking about "perfect chess" or 32 piece TBs is like talking about aliens or similar.

We have next to no data to analyze any of these things.

Given that Stockfish plays far from perfect chess (very strong is still far from perfect, just as 7 piece tablebases are exponentially far away from 32 piece...imo), my feeling is that it would lose 100-0 to 32 piece tablebases or a "perfect chess" entity.
Do you have an elo estimate for 32 piece. I think Kai would you just give it another 400 elo.
Maybe I'm ignorant on the technology, but to me, 32 piece tablebases=perfect play and perfect play= Elo unlimited.

It should be for example: 1.Nf3 d5 -+ (mate in 42)

Stockfish is say 3680 now (or whatever) and still far from perfect chess...far from perfect.

If Stockfish NNUE is even 3700, probably perfect chess is 5000 or more.

Totally theoretical of course. Even Capablanca thought chess was "solved" and that was in the 1930s. If Stockfish can still lose games, and it can, it is FAR from perfect...so perfect chess would crush SF 100-0 IMO.

Bit of rambling...12.18am and a couple nice German beers downed. :)
Leo
Posts: 1078
Joined: Fri Sep 16, 2016 6:55 pm
Location: USA/Minnesota
Full name: Leo Anger

Re: Perfect chess engine elo ( 32 men TB) can be within 200 of Stocfish in Tcec LTC conditions

Post by Leo »

Nay Lin Tun wrote: Fri Nov 06, 2020 12:30 am If there were no opening book, Stockfish will be minimally able to draw another Stockfish or Leela in 95% of games. If Stockfish is playing against 32 men TB, the extreme condition would be 50 draw 50 losses , which is within 200 elo.


On the other hand, current Stockfish with contempt may be slightly higher rated than 32 men TB in Round Robin rating pool. Prettty crazy right? Actually 32 men TB doesn't store statistics of winning chances or knowledge of maximising chances to win which will subsequently lose some elo. For example, in KRB vs KR, 7 men TB will randomly throw away one rook as it knows that all moves are draw. ( KB vs KR is still draw). Carry behaviour of how this 7 men TB play into 32 men TB, we can see TB will be randomly choose the first move 1. a3 or f3 or h3 or e4 (assuming that except 1.g4, all other 19 moves are likely to be draw). Subsequently, 32 men TB will choose next move like. 1. a3 e5 2. a4 ( It still knows as a draw but it throw away practical winning chances)
Quote: "The size of 8-man tablebases will be 100 times larger than the size of 7-man tablebases. To fully compute them, one will need about 10 PB (10,000 TB) of disk space and 50 TB of RAM. Only the top 10 supercomputers can solve the 8-man problem in 2014. The first 1000-move mate is unlikely to be found until 2020 when a part of a TOP100 supercomputer may be allowed to be used for solving this task."

I cant comprehend a 32 man table base and the storage needed for it.
Advanced Micro Devices fan.
User avatar
Ozymandias
Posts: 1532
Joined: Sun Oct 25, 2009 2:30 am

Re: Perfect chess engine elo ( 32 men TB) can be within 200 of Stocfish in Tcec LTC conditions

Post by Ozymandias »

Raphexon wrote: Fri Nov 06, 2020 4:18 pm
Ozymandias wrote: Fri Nov 06, 2020 3:00 pm
Alayan wrote: Fri Nov 06, 2020 12:25 pmThere is no reason it has to be from the starting position.
There isn't? What was the purpose of the Elo rating system as it was created?
To better predict the strength of (human) chess players than previous rating systems.
Exactly, and what position did those players start their games from (99% of the time)?
Dann Corbit
Posts: 12537
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: Perfect chess engine elo ( 32 men TB) can be within 200 of Stocfish in Tcec LTC conditions

Post by Dann Corbit »

The draw prophecies are going on the assumption that chess is a draw. It is likely true but we do not know that.
Another possibility is that perfect chess is a win for white.
Another possibility is that perfect chess is a win for black.
Play the strongest engine today on the strongest hardware today against the strongest program of ten years ago on the strongest hardware of ten years ago.
I don't think we will see a majority of draws.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.