AndrewGrant wrote: ↑Wed Jun 26, 2024 11:55 am
I don't know how things would go at TCEC. The CPUs there are weaker than what you can readily get as a consumer now. While the GPU at TCEC is the same as at CCC. So it is hard to say what amount of impact that has. Speed is worth a lot of elo, for 1-thread bullet games. Less so as the time control grows, and the thread count grows. But it is also clear to me that Lc0 does not "scale better" than Alpha-Beta engines.
For me, Lc0 definitly gains more strength with each doubling of computing-power (no matter, if doubling thinking-time or 2x fast hardware), compared to other engines. But not because Lc0 scales better, I agree here. But just because Lc0 has so much lower nps than Stockfish, Torch etc. And it is a fact in computerchess, that doubling the computing-power or doubling the thinking-time gives the less Elo-gain the higher the number of calculated nodes per played move on the board (depends on nps and thinking-time) already were before the doubling. IMHO, if we could accelerate both Lc0 and Stockfish by a really huge factor (10000x or so (perhaps, when the light-based CPUs are coming in the future...)), Lc0 should beat Stockfish and all other nnue-neuralnet engines.
I read that lc0 needs to memroize all the tree in order to play and it means that it cannot play better at 3000 minutes per move relative to 300 minutes per move.
I do not know why people believe lc0 scales better than stockfish and I saw no proof for it.
I believe that usually the stonger engine earns more elo from doubling the speed except maybe very fast time control that is faster than bullet and if you claim something different for stockfish and lc0 you need to prove it.
You can make a tournament when stockfish play at 1+0.3 time control and opponents play at x+0.3x time control when x>1 to make the result 50% when later you repeat the same tournament when stockfish use 10+3 and opponents use 10x+3x to see who is the winner.
the faster the time control the higher the Elo different between Stockfish to the others with 1 core. If I compare 66+6 with 6-pieces with the results 8+3 with 5-pieces on AMD-5950 hardware with 4.2Ghz (same group of 40 engines, each one against every others). A good example is Stockfish from end of the last year if I compare with the older last Stockfish version without NN. The different against 40 opponents with booth time controls is 93 Elo only (speaking from both Stockfish versions - 93 Elo -) after around 800 games. Same for the other engines after Stockfish ... the differents are much lower to Stockfish as in all the rating systems with the "very fast time-controls" the rating list people used.
For me it make sense what Andrew had written. I am very sure that Lc0 scales clearly better (can see that also with my strong 3700er grafic card) as CPU engines with longer time controls if I compare that with very fast time controls. For me since a while not longer a question and it seems very clear. No more proofs necessary for myself.
Note: Light to see if I compare 40 in 8 with 40 in 20 minutes (older experiments with 1 core). But with 66+6 the differents are really much smaller.
Different Stockfish last version without NN to Stockfish end of the last year with NN = 93 Elo with 66+6, 6-pieces vs. 40 opponents.
Different Stockfish last version without NN to Stockfish end of the last year with NN = 224 Elo with 8+3, 5 pieces vs. the same 40 opponents.
Different for the same test for Lc0 0.30.0 vs. the same 41 engines / time controls is only 46 Elo and not 131 Elo (Stockfish test results).
Best
Frank
The only problem in my testing is ... (think so) ...
Maybe Stockfish lost to many power because I had to many engines in the field the Elo differents to Stockfish are to hight (more as 250 Elo). But Lc0 had the same group of opponents!! Or maybe the field of 40-CPU opponents used to many Stockfish code??! (testing without Stockfish derivative engines). Often I think the playing style of all of the TOP-50 engines are in many cases indeed very comparable. Concerned the younger very strong engines, can play in the group of TOP-50 engines. The style of Lc0 is just great and very own, in the mid-game and the late mid-game and of course the earlier endgame. So I like the say that the style of Lc0 is for me just great, but also the style from many very own CPU-engines. But this is more or less a very personaly opinion because I like more the mid-games and looking here more as in the for me boring endgames.
I think the Torch idea is not bad and interesting if a group of programmer with so many knowledge start such a project. On the other hand we have Stockfish with a lot of more programmers with knowledge and the project is since many years very interesting and great for all of us. Same what the programmers of Lc0 do for us.
For me is much more interesting to have Kallisto, Berserk or Ethereal as Torch. On the other hand interesting to see what Torch can do, so I am looking for the Torch results too.
I am more a fan from the work of programmers where are working alone, because we have Stockfish and Lc0. I think that is enough! So, all the eng-eng games and rating list systems are more interesting if the programmers are working alone. We can compare the ideas, the playing styles the engines produced with all the own ideas the programmers have.
We should not feel like the Torch authors owe us something, such as a cheap or free version of the engine.
Just as Bruce Moreland never gave us Ferret, that was his right, so also nobody owes us a copy of Torch.
Eventually, I just made a game in my mind of "an engine I cannot prove does not exist for me" so the private engines stopped bothering me.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
Dann Corbit wrote: ↑Thu Jun 27, 2024 1:31 pm
We should not feel like the Torch authors owe us something, such as a cheap or free version of the engine.
Just as Bruce Moreland never gave us Ferret, that was his right, so also nobody owes us a copy of Torch.
Eventually, I just made a game in my mind of "an engine I cannot prove does not exist for me" so the private engines stopped bothering me.
Fallacious. Torch is not controlled by the Torch authors. Somehow (possibly without holding up in a legal sense, we don't know), a US monopoly capitalist outfit (objective maximise for shareholders) managed to extract source code out of the public domain, extract student programmers (who up to then were publishing in the public domain) for some temporary period, and have now basically removed what was common knowledge out of the common domain, to everybody's detriment. Conceptually, it's highly unlikely that there's anything in Torch that isn't actually in Stockfish, and (guessing) the probable only reason the source code isn't publicly available is that nobody cared or dared to put if back into the commons. Basically you've been robbed. Analogies to Ferret/Bruce are nonsensical. More analogous would be the dispossession of Scottish crofters by aristocratic English landlords.
chrisw wrote: ↑Thu Jun 27, 2024 2:53 pmSomehow (possibly without holding up in a legal sense, we don't know), a US monopoly capitalist outfit (objective maximise for shareholders) managed to extract source code out of the public domain
Unless the Torch authors wrote the code themselves, which is perfectly legal, or used some code portions by people who like their open source projects being turned into closed source products (BSD / MIT / Apache licence).
and (guessing) the probable only reason the source code isn't publicly available is that nobody cared or dared to put if back into the commons.
The reason is most likely business. Unlimited game analysis is behind is subscription paywall on chess.com, and the obvious question is why anyone would pay for something he can do for free at home using Stockfish. Torch was meant to be the answer, as in "because chess.com has the strongest engine that you don't get anywhere else". Didn't quite work out so far, but improving the usefulness of the analysis for human players is still on the table, with a similar argument for the exclusive analysis.
chrisw wrote: ↑Thu Jun 27, 2024 2:53 pm
Conceptually, it's highly unlikely that there's anything in Torch that isn't actually in Stockfish, and (guessing) the probable only reason the source code isn't publicly available
The thinking of a man without his own capacity to invent.
We experimented a lot in Koi, so did Andy in eth & Jay in berserk etc. I'm not sure why anyone would expect this to have changed when we worked on Torch. Personally I got a lot more freedom to experiment (more hw etc.) which led to some quite interesting ideas. It is indeed unfortunate that the tricks we found won't be public knowledge untill someone else finds them. It's also possible they wouldn't' work in sf. I am very happy to have had the opportunity to work on Torch, it was an amazing group of people.
I understand there is no way for an outside observer to confirm anything I've just said. Similarly, if we ignore the improvements made to openbench, our work has almost no practical value to the engine developer community.
Personally I feel little responsibility to contribute to the community at this stage. I am content with the tiny contributions I've made in the past. I do wish Torch was open source, but it's because like most engine devs, I'd love to see my work prove useful to other people (I really do think it's an amazing engine). I don't have a problem with the fact we worked on a closed source engine in and of itself.
Not really a response to anyone in particular, but more of a dump of my feelings on the matter. Have a nice day folks
I am grateful for open source software - but that's a choice: companies should be allowed to keep their software private unless there's an overwhelming public interest in taking it off them (e.g. a national emergency).
Human chess is partly about tactics and strategy, but mostly about memory
