End of the Torch Experiement

[AndrewGrant]

We already did a youtube signature disable (back in April?) because he was spamming youtube vids into his sig back then. I'm assuming he must have found a way round the disable and then inserted a current political propaganda video, via sig, into the yesterday post. He knows it's not allowed and I, for one, do not intend to play a moderation war with him of escalating measure/countermeasure. Hence the ban.

based

[Kanizsa]

Torch has beaten Leela in multiple Bullet and Blitz events in a row; Never managed to win one of the Rapid events; But has beaten Leela in a h2h classical event.

"We all know that the alpha-beta search approach tends to outperform neural architectures, so it’s not surprising that Torch consistently defeated Leela in Bullet and Blitz competitions. However, I’m surprised by your mention of Torch beating Leela in a ‘classical’ event. For me, and likely for many others, ‘classical’ refers to the traditional time control, such as that used in the TCEC (Top Chess Engine Championship). I regard you as a precise and knowledgeable person in this field, and I’d appreciate your opinion on a couple of questions.

Firstly, could you clarify what you mean by ‘classical chess’?"

As for the second question: what do you think the result of TCEC 26 would have been in an alternate final between Leela and Torch based on Noomen’s book?

[AndrewGrant]

Andrew, when will the free version of Torch be available?

As a stand-alone UCI engine, there are no plans for this. Torch is available in your browser, on chess.com, however.

[AndrewGrant]

"We all know that the alpha-beta search approach tends to outperform neural architectures, so it’s not surprising that Torch consistently defeated Leela in Bullet and Blitz competitions. However, I’m surprised by your mention of Torch beating Leela in a ‘classical’ event. For me, and likely for many others, ‘classical’ refers to the traditional time control, such as that used in the TCEC (Top Chess Engine Championship). I regard you as a precise and knowledgeable person in this field, and I’d appreciate your opinion on a couple of questions.

Firstly, could you clarify what you mean by ‘classical chess’?"

As for the second question: what do you think the result of TCEC 26 would have been in an alternate final between Leela and Torch based on Noomen’s book?

The event was here, https://www.chess.com/computer-chess-ch ... nd-torch-1 , in which Torch won with a score of 51 to 49. The time control was 30 minutes + 5 seconds. Lichess considers this as "classical" for example. Although many would call this the upper-end of Rapid.

I don't know how things would go at TCEC. The CPUs there are weaker than what you can readily get as a consumer now. While the GPU at TCEC is the same as at CCC. So it is hard to say what amount of impact that has. Speed is worth a lot of elo, for 1-thread bullet games. Less so as the time control grows, and the thread count grows. But it is also clear to me that Lc0 does not "scale better" than Alpha-Beta engines.

My guess would be the same guess I've made for every recent Rapid or Classical event between Leela and Torch: Coin Flip.

[pohl4711]

I don't know how things would go at TCEC. The CPUs there are weaker than what you can readily get as a consumer now. While the GPU at TCEC is the same as at CCC. So it is hard to say what amount of impact that has. Speed is worth a lot of elo, for 1-thread bullet games. Less so as the time control grows, and the thread count grows. But it is also clear to me that Lc0 does not "scale better" than Alpha-Beta engines.

For me, Lc0 definitly gains more strength with each doubling of computing-power (no matter, if doubling thinking-time or 2x fast hardware), compared to other engines. But not because Lc0 scales better, I agree here. But just because Lc0 has so much lower nps than Stockfish, Torch etc. And it is a fact in computerchess, that doubling the computing-power or doubling the thinking-time gives the less Elo-gain the higher the number of calculated nodes per played move on the board (depends on nps and thinking-time) already were before the doubling. IMHO, if we could accelerate both Lc0 and Stockfish by a really huge factor (10000x or so (perhaps, when the light-based CPUs are coming in the future...)), Lc0 should beat Stockfish and all other nnue-neuralnet engines.

[Werewolf]

I don't know how things would go at TCEC. The CPUs there are weaker than what you can readily get as a consumer now. While the GPU at TCEC is the same as at CCC. So it is hard to say what amount of impact that has. Speed is worth a lot of elo, for 1-thread bullet games. Less so as the time control grows, and the thread count grows. But it is also clear to me that Lc0 does not "scale better" than Alpha-Beta engines.

For me, Lc0 definitly gains more strength with each doubling of computing-power (no matter, if doubling thinking-time or 2x fast hardware), compared to other engines. But not because Lc0 scales better, I agree here. But just because Lc0 has so much lower nps than Stockfish, Torch etc. And it is a fact in computerchess, that doubling the computing-power or doubling the thinking-time gives the less Elo-gain the higher the number of calculated nodes per played move on the board (depends on nps and thinking-time) already were before the doubling. IMHO, if we could accelerate both Lc0 and Stockfish by a really huge factor (10000x or so (perhaps, when the light-based CPUs are coming in the future...)), Lc0 should beat Stockfish and all other nnue-neuralnet engines.

I may be misunderstanding your point above, apologies if I am.
But if Lc0 can get close to the strength of SF/Torch using far fewer nps, then its greater improvement per doubling of compute actually is better scaling, surely? What else can we call it?
I would have thought your points above only became valid if there was a clear difference in strength between the top 3.
Your final point about light-based CPUs also supports superior scaling (I agree here).

[Uri Blass]

I don't know how things would go at TCEC. The CPUs there are weaker than what you can readily get as a consumer now. While the GPU at TCEC is the same as at CCC. So it is hard to say what amount of impact that has. Speed is worth a lot of elo, for 1-thread bullet games. Less so as the time control grows, and the thread count grows. But it is also clear to me that Lc0 does not "scale better" than Alpha-Beta engines.

For me, Lc0 definitly gains more strength with each doubling of computing-power (no matter, if doubling thinking-time or 2x fast hardware), compared to other engines. But not because Lc0 scales better, I agree here. But just because Lc0 has so much lower nps than Stockfish, Torch etc. And it is a fact in computerchess, that doubling the computing-power or doubling the thinking-time gives the less Elo-gain the higher the number of calculated nodes per played move on the board (depends on nps and thinking-time) already were before the doubling. IMHO, if we could accelerate both Lc0 and Stockfish by a really huge factor (10000x or so (perhaps, when the light-based CPUs are coming in the future...)), Lc0 should beat Stockfish and all other nnue-neuralnet engines.

I read that lc0 needs to memroize all the tree in order to play and it means that it cannot play better at 3000 minutes per move relative to 300 minutes per move.

I do not know why people believe lc0 scales better than stockfish and I saw no proof for it.
I believe that usually the stonger engine earns more elo from doubling the speed except maybe very fast time control that is faster than bullet and if you claim something different for stockfish and lc0 you need to prove it.

You can make a tournament when stockfish play at 1+0.3 time control and opponents play at x+0.3x time control when x>1 to make the result 50% when later you repeat the same tournament when stockfish use 10+3 and opponents use 10x+3x to see who is the winner.

[Frank Quisinsky]

Hi Uri,

the faster the time control the higher the Elo different between Stockfish to the others with 1 core. If I compare 66+6 with 6-pieces with the results 8+3 with 5-pieces on AMD-5950 hardware with 4.2Ghz (same group of 40 engines, each one against every others). A good example is Stockfish from end of the last year if I compare with the older last Stockfish version without NN. The different against 40 opponents with booth time controls is 93 Elo only (speaking from both Stockfish versions - 93 Elo -) after around 800 games. Same for the other engines after Stockfish ... the differents are much lower to Stockfish as in all the rating systems with the "very fast time-controls" the rating list people used.

For me it make sense what Andrew had written. I am very sure that Lc0 scales clearly better (can see that also with my strong 3700er grafic card) as CPU engines with longer time controls if I compare that with very fast time controls. For me since a while not longer a question and it seems very clear. No more proofs necessary for myself.

Note: Light to see if I compare 40 in 8 with 40 in 20 minutes (older experiments with 1 core). But with 66+6 the differents are really much smaller.

Best
Frank

[Frank Quisinsky]

Ah sorry (800 games) ...

Different Stockfish last version without NN to Stockfish end of the last year with NN = 93 Elo with 66+6, 6-pieces vs. 40 opponents.
Different Stockfish last version without NN to Stockfish end of the last year with NN = 224 Elo with 8+3, 5 pieces vs. the same 40 opponents.

Different for the same test for Lc0 0.30.0 vs. the same 41 engines / time controls is only 46 Elo and not 131 Elo (Stockfish test results).

Best
Frank

The only problem in my testing is ... (think so) ...
Maybe Stockfish lost to many power because I had to many engines in the field the Elo differents to Stockfish are to hight (more as 250 Elo). But Lc0 had the same group of opponents!! Or maybe the field of 40-CPU opponents used to many Stockfish code??! (testing without Stockfish derivative engines). Often I think the playing style of all of the TOP-50 engines are in many cases indeed very comparable. Concerned the younger very strong engines, can play in the group of TOP-50 engines. The style of Lc0 is just great and very own, in the mid-game and the late mid-game and of course the earlier endgame. So I like the say that the style of Lc0 is for me just great, but also the style from many very own CPU-engines. But this is more or less a very personaly opinion because I like more the mid-games and looking here more as in the for me boring endgames.

[Frank Quisinsky]

Hi there,

I think the Torch idea is not bad and interesting if a group of programmer with so many knowledge start such a project. On the other hand we have Stockfish with a lot of more programmers with knowledge and the project is since many years very interesting and great for all of us. Same what the programmers of Lc0 do for us.

For me is much more interesting to have Kallisto, Berserk or Ethereal as Torch. On the other hand interesting to see what Torch can do, so I am looking for the Torch results too.

I am more a fan from the work of programmers where are working alone, because we have Stockfish and Lc0. I think that is enough! So, all the eng-eng games and rating list systems are more interesting if the programmers are working alone. We can compare the ideas, the playing styles the engines produced with all the own ideas the programmers have.

Best
Frank