How many NN Engines are there ?

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

carldaman
Posts: 2287
Joined: Sat Jun 02, 2012 2:13 am

Re: How many NN Engines are there ?

Post by carldaman »

Madeleine Birchfield wrote: Thu Dec 17, 2020 9:21 pm
Chessqueen wrote: Wed Dec 16, 2020 10:34 pm So this rating was taken before the NN addition which Houdini is listed on top of ethereal, if Houdini 6 is a clone CCRL and TCEC should stop testing them :?: http://ccrl.chessdom.com/ccrl/4040/
Houdini hasn't been a part of TCEC since Season 17, around the time that Houdini 5 and 6 were discovered to be Stockfish clones.

CCRL is an unreliable ratings list; they commonly remove other engines and put them on their 'CCRL Kill List' but don't bother to remove illegal GPL-violating Stockfish clones such as Houdini and Fire, put engines tested on 1 CPU, 2 CPUs, 4 CPUS, and 8 CPUs on the same list, test engines on different hardware strengths (i.e. Ethereal 12.75 in the 40/2 list is tested only on 8 CPU, and not on 1 CPU like other engines), and test engines by putting them against different opponents of different strengths, making the ratings list a reflection of the strength of the opponents in testing, rather than the strength of the engine itself.
Much too easy to criticize from one's armchair. I for one am glad they're involved in testing as many engines as they are. If you have all these ideas on how to improve things, you could offer up some friendly suggestions. :|

Chessbase sold Houdini for many years, how come they're not on the hook for selling it? Where were all the 'decompiler' warriors years ago when those Fire and Houdini versions first came out? It might have helped to find out sooner, before hours and hours of computer and electricity usage were invested into testing them.
AndrewGrant
Posts: 1960
Joined: Tue Apr 19, 2016 6:08 am
Location: U.S.A
Full name: Andrew Grant

Re: How many NN Engines are there ?

Post by AndrewGrant »

carldaman wrote: Fri Dec 18, 2020 9:16 am Much too easy to criticize from one's armchair. I for one am glad they're involved in testing as many engines as they are. If you have all these ideas on how to improve things, you could offer up some friendly suggestions. :|
I do believe that is exactly what was done.
carldaman wrote: Fri Dec 18, 2020 9:16 am Chessbase sold Houdini for many years, how come they're not on the hook for selling it?
If you scroll up a bit you'll see exactly this as well.
User avatar
Gabor Szots
Posts: 1466
Joined: Sat Jul 21, 2018 7:43 am
Location: Budapest, Hungary
Full name: Gabor Szots

Re: How many NN Engines are there ?

Post by Gabor Szots »

Madeleine Birchfield wrote: Thu Dec 17, 2020 9:21 pmi.e. Ethereal 12.75 in the 40/2 list is tested only on 8 CPU, and not on 1 CPU
Ethereal 12.75 64-bit is going to appear on this week's list.

You know, we have a lot of engines to test so sometimes a particular engine has to wait until it gets CPU time.

PS. I am waiting eagerly for your first constructive post. BTW, I have still not found out under which name you had posted before you found this Madeleine Birchfield thing.
Gabor Szots
CCRL testing group
carldaman
Posts: 2287
Joined: Sat Jun 02, 2012 2:13 am

Re: How many NN Engines are there ?

Post by carldaman »

AndrewGrant wrote: Fri Dec 18, 2020 10:02 am
carldaman wrote: Fri Dec 18, 2020 9:16 am Much too easy to criticize from one's armchair. I for one am glad they're involved in testing as many engines as they are. If you have all these ideas on how to improve things, you could offer up some friendly suggestions. :|
I do believe that is exactly what was done.
carldaman wrote: Fri Dec 18, 2020 9:16 am Chessbase sold Houdini for many years, how come they're not on the hook for selling it?
If you scroll up a bit you'll see exactly this as well.
It looks like the Houdini/Chessbase discussion is just getting started. I don't recall much talk of them selling
a GPL-ed SF derivative years ago (Houdini 5/6 in particular). I know it's a few months since people have figured
more definitive things about the origins of Houdini, but without too much getting into how CB has profited from it, which probably represents a much bigger problem.

A big company like CB can't be bothered to waste resources digging into how legit some of their products are, while
continuing to profit from them, but we ask the same all too easily of testers who may not have the time, skill or resources to investigate such matters very deeply. Someone should have brought this up soon after those Fire and Houdini versions came out, but it took years.

:|
Madeleine Birchfield
Posts: 512
Joined: Tue Sep 29, 2020 4:29 pm
Location: Dublin, Ireland
Full name: Madeleine Birchfield

Re: How many NN Engines are there ?

Post by Madeleine Birchfield »

Gabor Szots wrote: Fri Dec 18, 2020 10:55 am Ethereal 12.75 64-bit is going to appear on this week's list.

You know, we have a lot of engines to test so sometimes a particular engine has to wait until it gets CPU time.
If so, then that is good news for CCRL indeed. I personally think the engines should be tested on 1 CPU first, before being tested on multiple CPUs.
Madeleine Birchfield
Posts: 512
Joined: Tue Sep 29, 2020 4:29 pm
Location: Dublin, Ireland
Full name: Madeleine Birchfield

Re: How many NN Engines are there ?

Post by Madeleine Birchfield »

Gabor Szots wrote: Fri Dec 18, 2020 10:55 am BTW, I have still not found out under which name you had posted before you found this Madeleine Birchfield thing.
I find your insistence that I am not Madeleine Birchfield quite insulting to be honest. I've been a lurker here for most of my time here; I never had an account here until this September. I used to post much more frequently in other places such as the Leela, Stockfish, and TCEC discords, as well as in other non-computer chess related forums such as boards.ie, republican.ie, alternatehistory.com, and laatste.info (the World Draughts Forum) under my name, or the Irish Gaelic version of my name, all before coming to Talkchess. You could go to those sites and find my posts there. I'm mostly on Talkchess because this is where most of the discussion about NNUE is taking place; previously it was in the Stockfish discord and in the TCEC discord.

Before NNUE came on the scene, I did not care about most computer chess engines, as I was only interested in strong engines that used neural networks (i.e. Leela and Allie). At the time, I could talk about in the Leela and Allie discords, rather than here. I only found Talkchess in the first place because I was doing research on Allie and found a few posts here by gonzochess mentioning his engine and the fact that he hopes to ultimately implement minimax search in Allie. However, most people here at the time were absolutely uninterested in talking about neural networks and applying them to their engine, so I had no interest in Talkchess for a very long time other than periodically checking for updates to Allie. Because of that, I was content to simply lurk in the background, spending most of my time talking about Leela and TCEC on the Leela discord, watching kingscrusher and Matthew Sadler's Youtube videos and commenting on them, and lurking in the fishcooking and lczero google groups. And if the shogi engine developers never ported a neural network to Stockfish, then it perhaps would have stayed that way.

However, the Stockfish community started experimenting with NNUE over the summer, and because of that I became extremely interested in Stockfish and the work that they have been doing, and I joined the Stockfish discord as a result. There were a lot of people posting on Talkchess about NNUE, such as Henk Drost, Dietrich Kappe, Norman Schmidt, Joerg Oster and so on, but the same people were on the Stockfish discord, and they were closer to the action talking with actual Stockfish developers such as Joost van der Vondele and Gary Linscott, so much of my attention drifted over to the Stockfish discord, fishcooking, and the Stockfish github. It was there in the Stockfish discord where I first found Vivien Clauzon, Volodymyr Shcherbyna, and Connor McMonigle, and became interested in their chess engines Minic, Igel, and Seer as they were early adopters of NNUE, and from there it brought me into discussions with the other engine developers in the TCEC discord, such as Terje Kirstihagen, the Weiss author who was trying to implement NNUE without success, and Andrew Grant, the Ethereal author who was extremely upset about NNUE at the time. All this time, I was keeping an eye on the discussions on Talkchess but I continued to lurk in the background.

It wasn't until TCEC announced new NNUE guidelines and Andrew Grant started a few drama-filled threads on Talkchess that I felt the need to say something on Talkchess itself regarding the TCEC NNUE guidelines. Hence why my first post was about the TCEC NNUE guidelines and how at the time I saw a double standard between the treatment of Allie/Leela and the treatment of engines using NNUE. This drama not only occurred in Talkchess, but also the Leela, Stockfish, and TCEC discords which I was a part of. Meanwhile, at the time I was shifting my focus away from Stockfish and Leela towards the other engine developers, seeing which of them would adopt NNUE, and as a result, after creating an account here, my activity on the Leela and Stockfish discords slowly declined, while my activity here increased. It was a huge culture shock to discover that even with the hype around NNUE, many people on Talkchess were not interested in the newest algorithms in computer chess, instead talking about hardware, chess analysis, handicap games, and old chess engines from the 1980s.

Many of my views about computer chess that you see here posted on TCEC were largely developed outside of Talkchess on the Stockfish, Leela, and TCEC discords, fishcooking, and the lczero google groups. For example, the fact that Houdini was a Stockfish clone for example I first found out about it through a discussion on fishcooking, and which I later confirmed on the TCEC discord why Houdini was no longer participating in TCEC Season 17.
Guenther
Posts: 4718
Joined: Wed Oct 01, 2008 6:33 am
Location: Regensburg, Germany
Full name: Guenther Simon

Re: How many NN Engines are there ?

Post by Guenther »

Madeleine Birchfield wrote: Fri Dec 18, 2020 12:55 pm
Gabor Szots wrote: Fri Dec 18, 2020 10:55 am BTW, I have still not found out under which name you had posted before you found this Madeleine Birchfield thing.
...
I used to post much more frequently in other places such as the Leela, Stockfish, and TCEC discords, as well as in other non-computer chess related forums such as boards.ie, republican.ie, alternatehistory.com,
...
We know ... since the beginning ... and also what to expect

https://www.alternatehistory.com/wiki/d ... birchfield
https://rwbc-chess.de

[Trolls n'existent pas...]
Madeleine Birchfield
Posts: 512
Joined: Tue Sep 29, 2020 4:29 pm
Location: Dublin, Ireland
Full name: Madeleine Birchfield

Re: How many NN Engines are there ?

Post by Madeleine Birchfield »

Guenther wrote: Fri Dec 18, 2020 1:02 pm We know ... since the beginning ... and also what to expect

https://www.alternatehistory.com/wiki/d ... birchfield
My behaviour on the main forums was fine and on topic. It was my behaviour when discussing about politics and current events in their chat subforum (their CTF equivalent) that got me banned from alternatehistory.com.

I suppose what I didn't realise at the time was that it wasn't necessarily the politics that got me banned, but the fact that I hold certain extremely strong views and am very stubborn about them, all the while thinking that my views are the only correct view, when I don't know enough about the field to have an informed view. While over there it was over politics (especially US politics), here it seems to have been neural networks and more recently NNUE technology.

I once held extremely strong views that neural networks were the answer to everything in computer chess and it was extremely easy to implement them, and because of that, everybody should implement them in their engines and completely abandon their handcrafted evaluation functions. this particular view dates back to the AlphaZero-Stockfish games, and especially the extreme reaction to AlphaZero by the computer chess community solidified this view for me. Recent discussions in the past few weeks with Connor McMonigle, Andrew Grant, Kieren Pearson, and so on have made me realise just how hard it is to write a good trainer and train a neural network and how much a good handcrafted evaluation function aids in training. The whole clone business and the recent TCEC NNUE guidelines have caused me to view Stockfish's NNUE code with suspicion, with the idea that engines would be better off if they used their own NNUE adaptation instead of Stockfish's. TCEC Season 20 has been somewhat humbling as well in both regards, realising that not every engine that adopts NNUE or writes their own NNUE code does well (Marvin comes to mind).
Chessqueen
Posts: 5685
Joined: Wed Sep 05, 2018 2:16 am
Location: Moving
Full name: Jorge Picado

Re: How many NN Engines are there ?

Post by Chessqueen »

AndrewGrant wrote: Fri Dec 18, 2020 10:02 am
carldaman wrote: Fri Dec 18, 2020 9:16 am Much too easy to criticize from one's armchair. I for one am glad they're involved in testing as many engines as they are. If you have all these ideas on how to improve things, you could offer up some friendly suggestions. :|
I do believe that is exactly what was done.
carldaman wrote: Fri Dec 18, 2020 9:16 am Chessbase sold Houdini for many years, how come they're not on the hook for selling it?
If you scroll up a bit you'll see exactly this as well.
I would like to donate $20.00 to your project for developing such a great program chess engine, but I have a question is the website to donate a secure website that only you will receive it? Another question that I have I downloaded Ethereal, but I still can NOT get it to work, Arena can NOT find the executable file anywhere. I Need help, and thanks against for developing such strong chess engine which is a la par with Komodo 13.3 or close to Komodo 14.

NOTE: I notice that in this division it is very hard for any engine with Black to win, the most that they can do is draw with black, since they are so close in ratings https://tcec-chess.com/live.html
Chessqueen
Posts: 5685
Joined: Wed Sep 05, 2018 2:16 am
Location: Moving
Full name: Jorge Picado

Re: How many NN Engines are there ?

Post by Chessqueen »

Chessqueen wrote: Fri Dec 18, 2020 3:11 pm
AndrewGrant wrote: Fri Dec 18, 2020 10:02 am
carldaman wrote: Fri Dec 18, 2020 9:16 am Much too easy to criticize from one's armchair. I for one am glad they're involved in testing as many engines as they are. If you have all these ideas on how to improve things, you could offer up some friendly suggestions. :|
I do believe that is exactly what was done.
carldaman wrote: Fri Dec 18, 2020 9:16 am Chessbase sold Houdini for many years, how come they're not on the hook for selling it?
If you scroll up a bit you'll see exactly this as well.
I would like to donate $20.00 to your project for developing such a great program chess engine, but I have a question is the website to donate a secure website that only you will receive it? Another question that I have I downloaded Ethereal, but I still can NOT get it to work, Arena can NOT find the executable file anywhere. I Need help, and thanks against for developing such strong chess engine which is a la par with Komodo 13.3 or close to Komodo 14.

NOTE: I notice that in this division it is very hard for any engine with Black to win, the most that they can do is draw with black, since they are so close in ratings, and Defenchess 2.3_dev2 is suffering for NOT using NN or SF NNUE like the others https://tcec-chess.com/live.html