How does an engine with 0 ELO play ?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

Milos
Posts: 4190
Joined: Wed Nov 25, 2009 1:47 am

Re: How does an engine with 0 ELO play ?

Post by Milos »

Kotlov wrote:
Milos wrote:RM can never be 0 Elo, because engine playing perfect suicide chess is infinitely weaker than RM in normal chess.
If taken as 0 Elo=RM then "suicide chess players" will be less than 0.
IDK why your not understanding that?
Why take arbitrary value for 0 Elo as RM when you can define an absolute 0Elo player?
Robert Pope
Posts: 558
Joined: Sat Mar 25, 2006 8:27 pm

Re: How does an engine with 0 ELO play ?

Post by Robert Pope »

Milos wrote:
Kotlov wrote:
Milos wrote:RM can never be 0 Elo, because engine playing perfect suicide chess is infinitely weaker than RM in normal chess.
If taken as 0 Elo=RM then "suicide chess players" will be less than 0.
IDK why your not understanding that?
Why take arbitrary value for 0 Elo as RM when you can define an absolute 0Elo player?
Because you can't define an absolute 0 Elo player. But using a random mover is an objective benchmark that can be defined and measured against.
You could also define 0 Elo as absolute.
It is actually quite simple. Absolute 0 Elo player would have 0 point from any match against anyone.
1. Where is this absolute 0 Elo player, so we can match him against other players and find their rating? Answer: He doesn't exist, so we can't. Just like there is no absolute perfect player in existence.

2. If this 0 Elo player actually existed, he would be useless as a benchmark, since every player in the world would by definition score 100% against him.

3. What happens when two 0 Elo players play against each other? They can't both lose, so it is impossible to exist with your definition above.
User avatar
Kotlov
Posts: 266
Joined: Fri Jul 10, 2015 9:23 pm
Location: Russia

Re: How does an engine with 0 ELO play ?

Post by Kotlov »

Milos wrote:Why take arbitrary value for 0 Elo as RM when you can define an absolute 0Elo player?
Because we can not define an absolute(strong or suicide) player.

Elo is a just dB (Decibel) for measuring relative strength chess players.

But we have to choose some well-known (and always the same) level what would be able to compare. It's called the Reference Zero.
Eugene Kotlov
Hedgehog 2.1 64-bit coming soon...
Milos
Posts: 4190
Joined: Wed Nov 25, 2009 1:47 am

Re: How does an engine with 0 ELO play ?

Post by Milos »

Kotlov wrote:
Milos wrote:Why take arbitrary value for 0 Elo as RM when you can define an absolute 0Elo player?
Because we can not define an absolute(strong or suicide) player.

Elo is a just dB (Decibel) for measuring relative strength chess players.

But we have to choose some well-known (and always the same) level what would be able to compare. It's called the Reference Zero.
Ok so you want an arbitrary level that is very weak and easy achievable.
Well, why still RM? It is not deterministic, it depends on the implementation of PRNG and if you play 2 random movers against each other, one will almost always have a non-0Elo score.
I can think of a dozen better implementations than a random mover.
Why not sort all possible moves in a given position alphabetically according to algebraic notation and pick the first one always?
That way you get engine that is equally weak as random mover but is fully deterministic and if played it against itself with switching sides you'd always get equal score.
User avatar
Kotlov
Posts: 266
Joined: Fri Jul 10, 2015 9:23 pm
Location: Russia

Re: How does an engine with 0 ELO play ?

Post by Kotlov »

Milos wrote:and if you play 2 random movers against each other, one will almost always have a non-0Elo score.
I will have difference between them = 0 Elo as well as between two Stockfish 9. You really undestand what rating Elo means?
Eugene Kotlov
Hedgehog 2.1 64-bit coming soon...
Milos
Posts: 4190
Joined: Wed Nov 25, 2009 1:47 am

Re: How does an engine with 0 ELO play ?

Post by Milos »

Kotlov wrote:
Milos wrote:and if you play 2 random movers against each other, one will almost always have a non-0Elo score.
I will have difference between them = 0 Elo as well as between two Stockfish 9. You really undestand what rating Elo means?
You are the one who doesn't understand it.
If you play 2 identical engines that play random moves the score will almost never be equal no matter how many games you play. It is like throwing a coin N times and expecting to have equal number of heads and tails. That will almost never happen.
SF 9 is also not fully deterministic (especially if you play on many cores), so if you play N games of SF9 vs SF9, chance for the score to exactly be N/2:N/2 is extremely slim.
Robert Pope
Posts: 558
Joined: Sat Mar 25, 2006 8:27 pm

Re: How does an engine with 0 ELO play ?

Post by Robert Pope »

Milos wrote:
Kotlov wrote:
Milos wrote:and if you play 2 random movers against each other, one will almost always have a non-0Elo score.
I will have difference between them = 0 Elo as well as between two Stockfish 9. You really undestand what rating Elo means?
You are the one who doesn't understand it.
If you play 2 identical engines that play random moves the score will almost never be equal no matter how many games you play. It is like throwing a coin N times and expecting to have equal number of heads and tails. That will almost never happen.
But, the more times you flip a fair coin, the percent of times it lands heads will converge to 50% in the limit, which is equivalent to having the same elo.

If you play a tournament of 100 random movers, of course some will have better performance than others. That's what the error bars around an elo score are about. And if you play enough games, you can get that error arbitrarily small.

The point of anchoring to a random mover is to give an objective, well-defined baseline against which to judge other players. And in order for it to be useful, yes, there are a number of issues to deal with, particularly having a wide variety of intermediate engines in the huge gap between a random mover and 1000 FIDE.

But at the end of the day, the only point is to have a fixed reference. As an analogy, it really doesn't have any practical importance if we define the temperature 0 degrees as the point at which water freezing, or at the point that electrons stop vibrating, as long as other people know how it is defined and can use it meaningfully.

We can just as easily anchor elo to Stockfish 1.0 = 2000 or Scorpio 8.0 = 2500 or any similar measure, as long as it is defined and available to benchmark against. A random mover has the added advantage that almost any programmer can trivially create one.

A big downside of a random mover baseline is that they don't improve with increased hardware, while almost all other engines do. That means elo at different time controls and/or hardware would be harder to compare. If you're going to bag on the idea, that's where you should aim, not the obvious fact that two equally strong engines won't always score exactly 50% against each other.
Last edited by Robert Pope on Tue Apr 17, 2018 7:35 pm, edited 1 time in total.
Milos
Posts: 4190
Joined: Wed Nov 25, 2009 1:47 am

Re: How does an engine with 0 ELO play ?

Post by Milos »

Robert Pope wrote:But, the more times you flip a fair coin, the percent of times it lands heads will converge to 50% in the limit, which is equivalent to having the same elo.
Only true if your PRNG is ideal which is never the case.
Since the number of games you play is never infinite you will almost never have exactly equal score. And probability to have exactly equal score is reducing the more games you play.
On the other hand using fully deterministic engine as your anchor can reduce error margins by sqrt(2) the same effect as if you played only half of the games.
So I still don't understand why use a fully non-deterministic engine as your anchor when you can have equally weak engine that is fully deterministic?
Robert Pope
Posts: 558
Joined: Sat Mar 25, 2006 8:27 pm

Re: How does an engine with 0 ELO play ?

Post by Robert Pope »

Milos wrote:
Robert Pope wrote:But, the more times you flip a fair coin, the percent of times it lands heads will converge to 50% in the limit, which is equivalent to having the same elo.
Only true if your PRNG is ideal which is never the case.
Since the number of games you play is never infinite you will almost never have exactly equal score. And probability to have exactly equal score is reducing the more games you play.
On the other hand using fully deterministic engine as your anchor can reduce error margins by sqrt(2) the same effect as if you played only half of the games.
So I still don't understand why use a fully non-deterministic engine as your anchor when you can have equally weak engine that is fully deterministic?
That PRNG argument is a total red herring. The impact of a poor PRNG vs a perfect PRNG has literally no impact on how well a random mover will score.

A deterministic engine would be fine to anchor against, as I already posted, any engine is fine. But then what is the basis for giving your deterministic engine an elo of 0, and not my slightly different deterministic engine an elo of 0?

Even worse, given two different deterministic engines, engine A may win 100% against B from the opening position, 30% from a random 2-move book, and 45% with a strong opening book. So what is their relative elo? A random mover doesn't suffer from the accidental biases that a deterministic mover would have.

I feel like I'm banging my head against the wall here, so I am going to try to stop responding.
Milos
Posts: 4190
Joined: Wed Nov 25, 2009 1:47 am

Re: How does an engine with 0 ELO play ?

Post by Milos »

Robert Pope wrote:
Milos wrote:
Robert Pope wrote:But, the more times you flip a fair coin, the percent of times it lands heads will converge to 50% in the limit, which is equivalent to having the same elo.
Only true if your PRNG is ideal which is never the case.
Since the number of games you play is never infinite you will almost never have exactly equal score. And probability to have exactly equal score is reducing the more games you play.
On the other hand using fully deterministic engine as your anchor can reduce error margins by sqrt(2) the same effect as if you played only half of the games.
So I still don't understand why use a fully non-deterministic engine as your anchor when you can have equally weak engine that is fully deterministic?
That PRNG argument is a total red herring. The impact of a poor PRNG vs a perfect PRNG has literally no impact on how well a random mover will score.
True for random mover not true for coin. I quoted exactly your sentence regarding the coin. Maybe you should be more carefully when formulating statements.
Even worse, given two different deterministic engines, engine A may win 100% against B from the opening position, 30% from a random 2-move book, and 45% with a strong opening book. So what is their relative elo? A random mover doesn't suffer from the accidental biases that a deterministic mover would have.
That is real red herring. Could you actually support your belief that fully deterministic engine I proposed few posts before would suffer from accidental biases any more than a pure random mover?