Heyho, I already mentioned it in the programmers section, for comparing engines
over computer chess history or different architectures I propose a metric like:
Elo / (Transistorcount*Frequency)
Can be applied backwards with electro-mechanical relays, vacuum-tubes,
transistors, ICs and microchips.
Can be applied sidewards with CPU, VPU, ASIC, FPGA, GPU, TPU.
Can be applied with or w/o memory (e.g. SRAM or DRAM, no core rope memory?).
Dunno how to be applied forward for quantum-gates, memristors and alike.
--
Srdja
Comparing Chess Engines over History or Architectures - Elo / (Transistorcount*Frequency)
Moderator: Ras
-
- Posts: 3040
- Joined: Wed Mar 10, 2010 10:18 pm
- Location: Hamburg, Germany
- Full name: Srdja Matovic
-
- Posts: 12144
- Joined: Thu Mar 09, 2006 12:57 am
- Location: Birmingham UK
- Full name: Graham Laight
Re: Comparing Chess Engines over History or Architectures - Elo / (Transistorcount*Frequency)
Hmmmmm....
If I call transistors*frequency "computer power" (and accepting that extra transistors are not actually directly equivalent to a higher clock frequency - though I think there's probably a good correlation between GeekBench score and elo score - so for me "GeekBench score" would be better than transistors*frequency), then I would have thought that the thing you really want to know is...
Elo for specified program at given computer power.
For a given program, elo for given computer power is likely to be logarithmic if you're lucky, and have a hard upper limit if you're not lucky.
So with computer power on the x axis and elo on the y axis, it's likely to have this shape (ignore the actual numbers on the y axis, or multiply them by 1000):
Big breakthroughs that meaningfully move the curve upwards are becoming increasingly rare: the last one was the application of ANNs to the task, following Google's breakthrough in Go with AlphaGo, and people working out how to apply that to chess.
These major breakthroughs (sometimes called "paradigm shifts") have always been more interesting to me than techniques which yield marginal gains. Unfortunately for me, most members of this forum prefer to discuss the marginal gains.
If I call transistors*frequency "computer power" (and accepting that extra transistors are not actually directly equivalent to a higher clock frequency - though I think there's probably a good correlation between GeekBench score and elo score - so for me "GeekBench score" would be better than transistors*frequency), then I would have thought that the thing you really want to know is...
Elo for specified program at given computer power.
For a given program, elo for given computer power is likely to be logarithmic if you're lucky, and have a hard upper limit if you're not lucky.
So with computer power on the x axis and elo on the y axis, it's likely to have this shape (ignore the actual numbers on the y axis, or multiply them by 1000):
Big breakthroughs that meaningfully move the curve upwards are becoming increasingly rare: the last one was the application of ANNs to the task, following Google's breakthrough in Go with AlphaGo, and people working out how to apply that to chess.
These major breakthroughs (sometimes called "paradigm shifts") have always been more interesting to me than techniques which yield marginal gains. Unfortunately for me, most members of this forum prefer to discuss the marginal gains.
Want to attract exceptional people? Be exceptional.
-
- Posts: 3040
- Joined: Wed Mar 10, 2010 10:18 pm
- Location: Hamburg, Germany
- Full name: Srdja Matovic
Re: Comparing Chess Engines over History or Architectures - Elo / (Transistorcount*Frequency)
Hmm, does GeekBench run on the 6502, IBM DeepBlue, Hydra, Google TPU, etc.?towforce wrote: ↑Sat Nov 26, 2022 3:13 pm Hmmmmm....
If I call transistors*frequency "computer power" (and accepting that extra transistors are not actually directly equivalent to a higher clock frequency - though I think there's probably a good correlation between GeekBench score and elo score - so for me "GeekBench score" would be better than transistors*frequency), then I would have thought that the thing you really want to know is...
The funny thing is, Lc0 is derived from A0 is derived from AlphaGo, MCTS-PUCT with CNNs came from the Go world to chess, SF AB NNUE came to chess from the Shogi world...I am not that much into NNs with NNUE, but my take for the next Elo jump would be NNOM++ - Move Ordering Neural Networks:towforce wrote: ↑Sat Nov 26, 2022 3:13 pm Elo for specified program at given computer power.
For a given program, elo for given computer power is likely to be logarithmic if you're lucky, and have a hard upper limit if you're not lucky.
So with computer power on the x axis and elo on the y axis, it's likely to have this shape (ignore the actual numbers on the y axis, or multiply them by 1000):
Big breakthroughs that meaningfully move the curve upwards are becoming increasingly rare: the last one was the application of ANNs to the task, following Google's breakthrough in Go with AlphaGo, and people working out how to apply that to chess.
These major breakthroughs (sometimes called "paradigm shifts") have always been more interesting to me than techniques which yield marginal gains. Unfortunately for me, most members of this forum prefer to discuss the marginal gains.
https://talkchess.com/forum3/viewtopic.php?f=7&t=80364
--
Srdja
-
- Posts: 3040
- Joined: Wed Mar 10, 2010 10:18 pm
- Location: Hamburg, Germany
- Full name: Srdja Matovic
Re: Comparing Chess Engines over History or Architectures - Elo / (Transistorcount*Frequency)
Hmm, keep in mind, there is also a space-time tradeoff and knowledge-search tradeoff going on, imagine 32-men EGTB, imagine TB neural networks....towforce wrote: ↑Sat Nov 26, 2022 3:13 pm [...]
Elo for specified program at given computer power.
For a given program, elo for given computer power is likely to be logarithmic if you're lucky, and have a hard upper limit if you're not lucky.
So with computer power on the x axis and elo on the y axis, it's likely to have this shape (ignore the actual numbers on the y axis, or multiply them by 1000):
[...]
--
Srdja
-
- Posts: 12144
- Joined: Thu Mar 09, 2006 12:57 am
- Location: Birmingham UK
- Full name: Graham Laight
Re: Comparing Chess Engines over History or Architectures - Elo / (Transistorcount*Frequency)
One qubit = one transistor, and the "frequency" is the number of times it can be put into a superposition, entangled with other qubits, collapsed from a superposition to a single state, and be measured in a second (allowing for decoherence, error correction etc).
I don't actually think you'll ever have to worry about that, though. There once was a computer with the following specification:
* 400 flops
* weighed over 27,000 kg
* big enough to park a school bus inside
* consumed 160 kw of power
* unreliable: it only worked half the time (literally!)
* programming it entailed rewiring it, which took days!
* price: $6 million in 2022 dollars
What a crap computer by today's standards! Absolute rubbish!
This was ENIAC in 1946. At the time, it was actually remarkably good - far better than anything that had gone before - and was really good for big calculations. Consequently, more computers were built, and they rapidly improved.
However, todays quantum computers are nowhere near as good as today's standard computers: not even cheap computers! So while they're a good idea, I don't think they're going to get the investment to develop like 1940s/1950s computers did.
Want to attract exceptional people? Be exceptional.
-
- Posts: 3040
- Joined: Wed Mar 10, 2010 10:18 pm
- Location: Hamburg, Germany
- Full name: Srdja Matovic
Re: Comparing Chess Engines over History or Architectures - Elo / (Transistorcount*Frequency)
Well, at some point Moore's Law is dead, we already have GPUs with 450W and CPUs with 300+W. 3nm, 2nm, 2+nm (I know, marketing, but reflects transistorcount/mm2) in pipe, but in the long-term we need alternatives, quantum, memristors, photonics, dunno.
--
Srdja
PS:
-
- Posts: 12144
- Joined: Thu Mar 09, 2006 12:57 am
- Location: Birmingham UK
- Full name: Graham Laight
Re: Comparing Chess Engines over History or Architectures - Elo / (Transistorcount*Frequency)
Not forgetting the topic of this thread - better software!
I know I'm in a minority here, but I am sincere in my beliefs that:
1. It's possible to play outstanding chess with a small and fast algorithm
2. While I obviously cannot prove this, I genuinely believe that the available evidence makes this more likely than not
PS:
Want to attract exceptional people? Be exceptional.
-
- Posts: 539
- Joined: Tue Jul 03, 2018 10:19 am
- Full name: Folkert van Heusden
Re: Comparing Chess Engines over History or Architectures - Elo / (Transistorcount*Frequency)
like early versions of rofchade
-
- Posts: 3040
- Joined: Wed Mar 10, 2010 10:18 pm
- Location: Hamburg, Germany
- Full name: Srdja Matovic
Re: Comparing Chess Engines over History or Architectures - Elo / (Transistorcount*Frequency)
I knowtowforce wrote: ↑Sat Nov 26, 2022 7:05 pm Not forgetting the topic of this thread - better software!
I know I'm in a minority here, but I am sincere in my beliefs that:
1. It's possible to play outstanding chess with a small and fast algorithm
2. While I obviously cannot prove this, I genuinely believe that the available evidence makes this more likely than not
"God's Algorithm for Chess"
https://talkchess.com/forum3/viewtopic. ... 92#p932844
--
Srdja
-
- Posts: 12144
- Joined: Thu Mar 09, 2006 12:57 am
- Location: Birmingham UK
- Full name: Graham Laight
Re: Comparing Chess Engines over History or Architectures - Elo / (Transistorcount*Frequency)
It sounds as though the early versions of Rofchade achieved a very good evaluation at very low cost (in "computer power"). However, when it switched to NNUE, two things happened:
1. "computer power" required for evaluation increased
2. The overall strength increased
So: it was a very good hand-crafted evaluation (HCE), but like all the other HCEs it missed some important things. We also know that super-fast deep searching also misses important things. It is very likely that all today's NNs also miss important things.
Based on what we know about NNs, it's extremely likely that an algorithm could be both smaller (hence faster) and more accurate. Given that "death by draw" has come to the top level of correspondence chess, a top quality small/fast chess algorithm would quite likely be able to get the best available result from most positions even on a cheap computer.
Anyway - back to the topic: if early versions of Rofchade get good results at low cost, then they would get a high logarithmic curve on my chart (earlier in this thread) of elo against computer power.
Last edited by towforce on Sun Nov 27, 2022 2:59 pm, edited 2 times in total.
Want to attract exceptional people? Be exceptional.