Mystery engine at CCC

[Modern Times]

These days with open source everyone has got far too used to getting something free of charge. There will always be a mix of free and commercial. And there is nothng wrong with people being paid for their efforts and/or making money. It is the world we live in.

[Steve Maughan]

I welcome Torch as I'm all for competition!

@AndrewGrant A few questions for the authors:

1. When was the first line of Torch's code written?
2. What language is Torch written in (I assume C++, but Zig is another possibility)
3. Did you use another engine's code or data structures as a base?
4. How do you work as a team? Have you divided the work up so someone is responsible for the NN, while some else is responsible for the search etc?
5. From the game logs, it seems Torch doesn't use tablebases. Why was this decision taken? Is it because it'll be a WASM engine and won't have access to tablebases in the browser?

Anyway, congrats on the new engine. I'm looking forward to testing!

— Steve

[AndrewGrant]

@AndrewGrant A few questions for the authors:

1. When was the first line of Torch's code written?
2. What language is Torch written in (I assume C++, but Zig is another possibility)
3. Did you use another engine's code or data structures as a base?
4. How do you work as a team? Have you divided the work up so someone is responsible for the NN, while some else is responsible for the search etc?
5. From the game logs, it seems Torch doesn't use tablebases. Why was this decision taken? Is it because it'll be a WASM engine and won't have access to tablebases in the browser?

1. First test was submitted on March 10th, repository created on February 17th.
2. C++ as per usual for engines, but with a focus on staying in the C-isms to keep it fast
3. Project is entirely from scratch. As an example, the move format does not match Koi, Ethereal, or even Berserk, IIRC
4. I won't answer that
5. We just don't have Tablebases yet committed. I've mentioned at CCC that we do have the code, its just not worth much elo, and so we've not rushed to push it out. We'll have them eventually.

Thanks for your kind words.

[Steve Maughan]

1. First test was submitted on March 10th, repository created on February 17th.

Five months to create such an engine! That is truly incredible. I wouldn't have thought it possible in less than a year, even for a large company like Microsoft or Google. What was the first estimate of strength after the March 10th test?

- Steve

[AndrewGrant]

1. First test was submitted on March 10th, repository created on February 17th.

Five months to create such an engine! That is truly incredible. I wouldn't have thought it possible in less than a year, even for a large company like Microsoft or Google. What was the first estimate of strength after the March 10th test?

- Steve

I don't know I'm afraid. "First test" meant, first proper test where you compare the current version vs one with a change. I never collected any data about the rate of progression. Although we could get that retroactively, which would be a cool project. Although a time waste for now I'm afraid.

[Rebel]

1. First test was submitted on March 10th, repository created on February 17th.

Five months to create such an engine! That is truly incredible. I wouldn't have thought it possible in less than a year, even for a large company like Microsoft or Google. What was the first estimate of strength after the March 10th test?

- Steve

The question for the Torch team to answer is if there is a relationship with the buying of Komodo Dragon by chess.com

Statement from Larry on chess.com on March 17 - Five years ago, chess.com purchased a stake in KomodoChess along with rights to use our software. Now the merger is complete; chess.com has bought out KomodoChess entirely. Chess.com will now run the website, pay for further development of Komodo, and for the most part take over the responsibilities of both myself and Mark Lefler. Mark and I will remain as paid consultants thru 2025, but chess.com will make the decisions. Dietrich Kappe remains onboard as the NN trainer. It is not yet clear who will be the main programmer or programmers, but development will continue, although perhaps in different directions. The goal of becoming the world's number one engine will remain as a prime goal, but it is expected that this will take some major work and hence I wouldn't expect any quick results.

[dkappe]

The question for the Torch team to answer is if there is a relationship with the buying of Komodo Dragon by chess.com

There’s certainly a temporal relationship as one happened before the other.

[Robert Flesher]

I'll respond to a few of the comments:

Will Torch only be tested on Chess.com, or is there a chance that the engine will play in the TCEC tournament as well?

For now, this is the case. It is an undertaking to be able to provide binaries to other groups ( TCEC, CCRL, CEGT, FastGM, SPCC, ...). Because we need to protect ourselves from being reverse engineered, and also from testers sharing the binaries with the entire world. But I'm certain that we want to share when we can, and the reason is very simple. If someone googles "Chess Engine Rating List", I want them to see Torch.

Openbench already existed before Torch, chesscom might improve it which is nice but not because they want to help the community in general but because of their own interests.

I disagree with your assessment here. You say "not because they want to help the community". Well that is exactly why OpenBench's changes are still open source. OpenBench is, for all intents and purposes, me providing my code to the world for free. It would be great if OpenBench had more people contributing, but that is not the case. It was an explicit decision to continue sharing my changes with the world. The work we are doing for Torch, via OpenBench as the central location for all of it, helps the community.

In the top twenty engines according to CCRL ( Adding in Torch, and subtracting Fire ), 50% of them are using OpenBench. Torch, Berserk, Ethereal, RubiChess, Igel, Kovisto, Seer, Clover, Minic, Viridithas. And many many engines below that top-20 cutoff are too.

As you already said before, you could close source Openbench but that is something that you wouldn't even have a reason to do before now.

Indeed. I could stop pushing changes to OpenBench starting right now. It would benefit Torch because it would not let the Fishtest folk see any ideas. It would also hinder new developers from joining the space, which helps us ( If they were to eventually help Stockfish ), and hurts us ( If they were to eventually join us with Torch ). OpenBench, ( for the sake of Torch ), is being developed on Chess.com's dime. That is an incredible contribution to the entire engine development community. I don't know how I can make this point any more clear and pronounced.

Now you also made it clear that you guys are not willing to help the open source chess engine community anymore. If you guys come up with a good and original idea, you will not try to implement it in Stockfish or any other open source engine like you would before, now only in Torch, only for chesscom.

The two statements here are logically placed together in, but don't make sense together. Its possible to help the community, AND maintain any of our secret sauce. I, and others, still talk in the OpenBench discord, or the Stockfish discord. I'll still bounce ideas around with Vizvez. But when we come up with real novel ideas like we have thus far, we don't want to share that edge. Otherwise there is no edge. I want to beat Stockfish by being different than Stockfish.

If Torch ideas do not reach SF framework, it would be a good stress test to see how SF team holds, but there are many people in SF team to just give up, IMHO, so I think they will more less keep the current slow pace of progress. Elo race becomes more difficult!

Free market economics dictates that Torch is good for Stockfish development. What is the draw in endlessly improving Stockfish, when there is no viable competition? Leela has not been anyway close for a good while now... but now Torch comes in? Gets strong very quickly? Takes some back to back game-pairs off of Stockfish at CCC? Okay now there is a fire under the feet of Stockfish developers and fans.

The number 1 engine is the one that wins the TCEC tournament. Everyone agrees: This is the best engine!

I agree with this sentiment to an extent. TCEC indeed has the best time controls, but for that they lose out on sample size. So the results are a little bit flimsy if engines are close in strength. TCEC and CCC both have the same GPUs, but TCEC has a significantly weaker set of CPUs.

If Chess.com comes out with their own engine, then I don't want to trust Chess.com tournaments anymore.

For this statement to be true for you, you would have to assume that I am doing something unethical as the tournament director. But I don't recall anyone complaining in the time I was running things with Ethereal as my engine. As someone who wrote a framework that 50% of the top 20 engines uses, as someone who has elo gaining contributions into both Stockfish and Dragon, and as a TD who works closely with the Lc0 team to make sure that everything is running as they desire -- I find any such assumption or accusation about me running things unethically to be quite offensive and unfounded.

Lc0 becomes really strong with strong GPU

Indeed. It would be better to have Lc0 with 4090s, but those are hard to get on servers. Which is why both TCEC and CCC are using 2xA100s last I checked. So Leela is pretty fair between the two venues, the only nuance is the CPUs have a little bit of an impact. I believe Leela wants low core count but fast CPUs. Just like CPU engines do.

As a very long time chess player, chess engine tester, but also a beta tester. I think Torch is very welcomed, as is all other new creations that try to become something new and head for the summit of the mountain. Cheers, to the new, to the brave, who seek to challenge the top. This, is good, very good for chess!

[lkaufman]

1. First test was submitted on March 10th, repository created on February 17th.

Five months to create such an engine! That is truly incredible. I wouldn't have thought it possible in less than a year, even for a large company like Microsoft or Google. What was the first estimate of strength after the March 10th test?

- Steve

The question for the Torch team to answer is if there is a relationship with the buying of Komodo Dragon by chess.com

Statement from Larry on chess.com on March 17 - Five years ago, chess.com purchased a stake in KomodoChess along with rights to use our software. Now the merger is complete; chess.com has bought out KomodoChess entirely. Chess.com will now run the website, pay for further development of Komodo, and for the most part take over the responsibilities of both myself and Mark Lefler. Mark and I will remain as paid consultants thru 2025, but chess.com will make the decisions. Dietrich Kappe remains onboard as the NN trainer. It is not yet clear who will be the main programmer or programmers, but development will continue, although perhaps in different directions. The goal of becoming the world's number one engine will remain as a prime goal, but it is expected that this will take some major work and hence I wouldn't expect any quick results.

The KomodoChess buyout was agreed just before the start of this year, but not announced until March. I didn't know whether chess.com would keep the Komodo Dragon codebase and try to improve it, or start from scratch just using ideas from it; either way Mark and I were onboard as advisors for three years. I don't think chess.com knew which way they would go when they agreed to the buyout either. As it turned out, they chose the second option, also giving it a new name as it is not a derivative. I look on Torch as chess.com's successor to Dragon, although we may still release one final Dragon if there is a good reason to do so.

[Modern Times]

I do find it really strange that they bought Komodo, a beautiful engine 2nd only to Stockfish, then a few months later essentially kill it off and fund a brand new engine from scratch. It makes no sense to me.