Mystery engine at CCC

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

peter
Posts: 3410
Joined: Sat Feb 16, 2008 7:38 am
Full name: Peter Martan

Re: Mystery engine at CCC

Post by peter »

jshriver wrote: Sun Jul 16, 2023 11:19 pm
peter wrote: Fri Jul 14, 2023 9:48 pm
Komodochess.com site doesn't allow log in anymore, does it?
I truly hope in memory of Don Dailey it keeps growing. I'just checked I can login to download links for previous purchases.

Image

Noticed they did remove Komodo 8 which Mr. Dailey was kind enough to give me a free copy. Then again the older copes are still free to download anyway.
Thanks for telling, I still can't log in. Even tried to let me send a new password in the meantime more often than once:
An email has been sent to you with a password to get into your account.
, but didn't get any email several days after latest try, of course I checked spamfilter regularly too.
Just now I wrote to the old adress of gmail- help, wait and see, regards
Peter.
AndrewGrant
Posts: 1960
Joined: Tue Apr 19, 2016 6:08 am
Location: U.S.A
Full name: Andrew Grant

Re: Mystery engine at CCC

Post by AndrewGrant »

I'll respond to a few of the comments:
Will Torch only be tested on Chess.com, or is there a chance that the engine will play in the TCEC tournament as well?
For now, this is the case. It is an undertaking to be able to provide binaries to other groups ( TCEC, CCRL, CEGT, FastGM, SPCC, ...). Because we need to protect ourselves from being reverse engineered, and also from testers sharing the binaries with the entire world. But I'm certain that we want to share when we can, and the reason is very simple. If someone googles "Chess Engine Rating List", I want them to see Torch.
Openbench already existed before Torch, chesscom might improve it which is nice but not because they want to help the community in general but because of their own interests.
I disagree with your assessment here. You say "not because they want to help the community". Well that is exactly why OpenBench's changes are still open source. OpenBench is, for all intents and purposes, me providing my code to the world for free. It would be great if OpenBench had more people contributing, but that is not the case. It was an explicit decision to continue sharing my changes with the world. The work we are doing for Torch, via OpenBench as the central location for all of it, helps the community.

In the top twenty engines according to CCRL ( Adding in Torch, and subtracting Fire ), 50% of them are using OpenBench. Torch, Berserk, Ethereal, RubiChess, Igel, Kovisto, Seer, Clover, Minic, Viridithas. And many many engines below that top-20 cutoff are too.
As you already said before, you could close source Openbench but that is something that you wouldn't even have a reason to do before now.
Indeed. I could stop pushing changes to OpenBench starting right now. It would benefit Torch because it would not let the Fishtest folk see any ideas. It would also hinder new developers from joining the space, which helps us ( If they were to eventually help Stockfish ), and hurts us ( If they were to eventually join us with Torch ). OpenBench, ( for the sake of Torch ), is being developed on Chess.com's dime. That is an incredible contribution to the entire engine development community. I don't know how I can make this point any more clear and pronounced.
Now you also made it clear that you guys are not willing to help the open source chess engine community anymore. If you guys come up with a good and original idea, you will not try to implement it in Stockfish or any other open source engine like you would before, now only in Torch, only for chesscom.
The two statements here are logically placed together in, but don't make sense together. Its possible to help the community, AND maintain any of our secret sauce. I, and others, still talk in the OpenBench discord, or the Stockfish discord. I'll still bounce ideas around with Vizvez. But when we come up with real novel ideas like we have thus far, we don't want to share that edge. Otherwise there is no edge. I want to beat Stockfish by being different than Stockfish.
If Torch ideas do not reach SF framework, it would be a good stress test to see how SF team holds, but there are many people in SF team to just give up, IMHO, so I think they will more less keep the current slow pace of progress. Elo race becomes more difficult!
Free market economics dictates that Torch is good for Stockfish development. What is the draw in endlessly improving Stockfish, when there is no viable competition? Leela has not been anyway close for a good while now... but now Torch comes in? Gets strong very quickly? Takes some back to back game-pairs off of Stockfish at CCC? Okay now there is a fire under the feet of Stockfish developers and fans.
The number 1 engine is the one that wins the TCEC tournament. Everyone agrees: This is the best engine!
I agree with this sentiment to an extent. TCEC indeed has the best time controls, but for that they lose out on sample size. So the results are a little bit flimsy if engines are close in strength. TCEC and CCC both have the same GPUs, but TCEC has a significantly weaker set of CPUs.
If Chess.com comes out with their own engine, then I don't want to trust Chess.com tournaments anymore.
For this statement to be true for you, you would have to assume that I am doing something unethical as the tournament director. But I don't recall anyone complaining in the time I was running things with Ethereal as my engine. As someone who wrote a framework that 50% of the top 20 engines uses, as someone who has elo gaining contributions into both Stockfish and Dragon, and as a TD who works closely with the Lc0 team to make sure that everything is running as they desire -- I find any such assumption or accusation about me running things unethically to be quite offensive and unfounded.
Lc0 becomes really strong with strong GPU
Indeed. It would be better to have Lc0 with 4090s, but those are hard to get on servers. Which is why both TCEC and CCC are using 2xA100s last I checked. So Leela is pretty fair between the two venues, the only nuance is the CPUs have a little bit of an impact. I believe Leela wants low core count but fast CPUs. Just like CPU engines do.
Eduard
Posts: 1439
Joined: Sat Oct 27, 2018 12:58 am
Location: Germany
Full name: N.N.

Re: Mystery engine at CCC

Post by Eduard »

If the Stockfish community were instead of Chess.com and they were running tournaments like Chess.com does, I wouldn't like that either. From the moment someone says "I'm building a new engine and I want to be number 1", then they should refrain from tournaments as public competitions.

By the way, I haven't read anywhere that the Stockfish community speaks of Stockfish as the number 1 engine. Nowhere does it say "we are number 1". It just says "Stockfish is one of the best engines in the world". At Chess.com, however, the motto is "we want to be No. 1". A little humility might do you good, thanks!

Lc0 is still there, Lc0 plays very well on very strong hardware! That's why Lc0 has already won TCEC and also the cup competition. For testers like CCRL or others, it is not possible to apply such hardware for Lc0 as at TCEC. That's why I think that TCEC would be a worthy tournament to determine which engine is the best, at TCEC Lc0 has better chances than in ranked tournaments where testing is done with significantly weaker GPUs.
ImNotStockfish
Posts: 56
Joined: Tue Sep 14, 2021 12:29 am
Full name: .

Re: Mystery engine at CCC

Post by ImNotStockfish »

AndrewGrant wrote: Mon Jul 17, 2023 9:17 am
Now you also made it clear that you guys are not willing to help the open source chess engine community anymore. If you guys come up with a good and original idea, you will not try to implement it in Stockfish or any other open source engine like you would before, now only in Torch, only for chesscom.
The two statements here are logically placed together in, but don't make sense together. Its possible to help the community, AND maintain any of our secret sauce. I, and others, still talk in the OpenBench discord, or the Stockfish discord. I'll still bounce ideas around with Vizvez. But when we come up with real novel ideas like we have thus far, we don't want to share that edge. Otherwise there is no edge. I want to beat Stockfish by being different than Stockfish.
Your engines were already different than Stockfish, most of the original ideas that work in your engines don't work in Stockfish (and vice versa), but at least you used to tried them.
AndrewGrant wrote: Mon Jul 17, 2023 9:17 am What is the draw in endlessly improving Stockfish, when there is no viable competition?
What is the problem with this? Isn't the point to try to make the strongest chess engine? Even if Stockfish was the only engine in the world, what would be the problem with endlessly trying to improve it? If another engine appeared that suddenly catches up by using novel ideas, those ideas could be tried with Stockfish and if they worked then the whole world could enjoy an even better engine!
AndrewGrant wrote: Mon Jul 17, 2023 9:17 am Leela has not been anyway close for a good while now...
Leela is getting pretty close to Stockfish at TCEC hw and TCs and even winning some (arguably very sss) events there.
ImNotStockfish
Posts: 56
Joined: Tue Sep 14, 2021 12:29 am
Full name: .

Re: Mystery engine at CCC

Post by ImNotStockfish »

Eduard wrote: Mon Jul 17, 2023 10:32 am From the moment someone says "I'm building a new engine and I want to be number 1", then they should refrain from tournaments as public competitions.
You still haven't explained why they should refrain.
Eduard wrote: Mon Jul 17, 2023 10:32 am At Chess.com, however, the motto is "we want to be No. 1". A little humility might do you good, thanks!
Chesscom is just like most for-profit companies, they are not going to be humble.
Eduard wrote: Mon Jul 17, 2023 10:32 am That's why I think that TCEC would be a worthy tournament to determine which engine is the best, at TCEC Lc0 has better chances than in ranked tournaments where testing is done with significantly weaker GPUs.
Again, CCC and TCEC both have the same GPUs. Why is TCEC the only worthy tournament?
AndrewGrant
Posts: 1960
Joined: Tue Apr 19, 2016 6:08 am
Location: U.S.A
Full name: Andrew Grant

Re: Mystery engine at CCC

Post by AndrewGrant »

Eduard wrote: Mon Jul 17, 2023 10:32 am By the way, I haven't read anywhere that the Stockfish community speaks of Stockfish as the number 1 engine. Nowhere does it say "we are number 1". It just says "Stockfish is one of the best engines in the world". At Chess.com, however, the motto is "we want to be No. 1". A little humility might do you good, thanks!
Everyone knows who the King is :) The King need not declare it.
"We want to be No. 1" is not lacking humility. "We will become No. 1", now that would need some humility.

And I'm unconvinced by Leela. She maybe gets close to Stockfish at TCEC? Well the CPUS there are simply bad bad bad. Hell, even CCC lacks a machine with VNNI support. Any perceived gains from Leela moving to 4090s, are mirrored in Stockfish getting a better CPU as well.

That being said, Leela is a different animal. Breakthroughs happen in that world more often than the alpha-beta world. So this all could change at the drop of a hat. But its not the case today in my estimation.
Eduard
Posts: 1439
Joined: Sat Oct 27, 2018 12:58 am
Location: Germany
Full name: N.N.

Re: Mystery engine at CCC

Post by Eduard »

ImNotStockfish wrote: Mon Jul 17, 2023 11:21 am
Eduard wrote: Mon Jul 17, 2023 10:32 am From the moment someone says "I'm building a new engine and I want to be number 1", then they should refrain from tournaments as public competitions.
You still haven't explained why they should refrain.
Eduard wrote: Mon Jul 17, 2023 10:32 am At Chess.com, however, the motto is "we want to be No. 1". A little humility might do you good, thanks!
Chesscom is just like most for-profit companies, they are not going to be humble.
Eduard wrote: Mon Jul 17, 2023 10:32 am That's why I think that TCEC would be a worthy tournament to determine which engine is the best, at TCEC Lc0 has better chances than in ranked tournaments where testing is done with significantly weaker GPUs.
Again, CCC and TCEC both have the same GPUs. Why is TCEC the only worthy tournament?
Everyone should decide for themselves whether they like the Chess.com style or not. I do not like him! Where commercial interests are behind it, I will stay away from the same people's tournaments. The chess.com tournaments mean nothing to me anymore. That's my opinion and I'm allowed to say it.
CornfedForever
Posts: 648
Joined: Mon Jun 20, 2022 4:08 am
Full name: Brian D. Smith

Re: Mystery engine at CCC

Post by CornfedForever »

Eduard wrote: Mon Jul 17, 2023 12:25 pm
Everyone should decide for themselves whether they like the Chess.com style or not. I do not like him! Where commercial interests are behind it, I will stay away from the same people's tournaments. The chess.com tournaments mean nothing to me anymore. That's my opinion and I'm allowed to say it.
You are of course allowed to say it. But everyone familiar with your self-interest inclinations knows 'why' you are saying what you have been saying.
marsell
Posts: 107
Joined: Tue Feb 07, 2012 11:14 am

Re: Mystery engine at CCC

Post by marsell »

Finally someone is saying what I have been thinking for a long time and I can agree with it 100%.
schahmatist
Posts: 37
Joined: Sat Mar 12, 2022 3:52 pm
Full name: Ben Krik

Re: Mystery engine at CCC

Post by schahmatist »

Sorry for naive question, but does it mean that Torch UCI engine (whatever its strongest version) won't be available for free?

I completely understand , that you, guys, put lots of efforts in it and would like to make deserved money. Still would be awfully nice to be able to test Torch against other engines without having to pay for it.