Dangerous turn

[Daniel Shawul]

And Texel-tuned engines are often optimized to game outcome (whatever engine played the game), not the score itself.

This doesn't matter much. Infact training against game outcome (Z) is better than score (Q) because Z can potentially incorporate
long-term effects. A0 used Z entirely but you can also mix in say 10% of Q to Z for faster learning.

Yes that was my point. No need for SF score in tuning data for Texel tuning method.
In fact quiet-labeled.edp and violent.edp from Zurichess doesn't have SF score inside ... just game outcome...
And those games were played with Zurichess ... not Stockfish (http://talkchess.com/forum3/viewtopic.p ... dp#p686204)
So I don't get your statement

They have been tuning their evals from datasets scored by stockfish such as Zurichess's set.

Did you read the link you posted ?

4. Next, from each quiet position a game using Stockfish 080916 was played.
The results were stored in quiet-labeled.epd.

In any case, whether you use the actual stockfish search score or the outcome from the game played by it hardly makes a difference
to the tuning process. A Z vs Q training result in the same training eventually.

[xr_a_y]

And Texel-tuned engines are often optimized to game outcome (whatever engine played the game), not the score itself.

This doesn't matter much. Infact training against game outcome (Z) is better than score (Q) because Z can potentially incorporate
long-term effects. A0 used Z entirely but you can also mix in say 10% of Q to Z for faster learning.

Yes that was my point. No need for SF score in tuning data for Texel tuning method.
In fact quiet-labeled.edp and violent.edp from Zurichess doesn't have SF score inside ... just game outcome...
And those games were played with Zurichess ... not Stockfish (http://talkchess.com/forum3/viewtopic.p ... dp#p686204)
So I don't get your statement

They have been tuning their evals from datasets scored by stockfish such as Zurichess's set.

Did you read the link you posted ?

4. Next, from each quiet position a game using Stockfish 080916 was played.
The results were stored in quiet-labeled.epd.

In any case, whether you use the actual stockfish search score or the outcome from the game played by it hardly makes a difference
to the tuning process. A Z vs Q training result in the same training eventually.

Oops ! missed that sorry. You are right.

To go on, on the same subject, today most people are using lichess data, I think it is "real" game outcome but I'm not sure.

[Daniel Shawul]

Raw data from lichess or other human game sources are not usually quality.
Programmers usually use either the score embedded in lichess games (again Stockfish is the mentor) or label them
themselves with the strongest engine. Here is a python script from Arasan for example, default scorer engine is stockfish-8
https://github.com/jdart1/arasan-chess/ ... sitions.py

When dkappe trained his BadGyal net he used Stockfish to generate the policy data. Only person that used that approach as far as I know.

The best quality games you can find are computer games and even that does not produce good nets or evaluations e.g. Giraffe and Leela-ccrl nets

[Alayan]

Anybody is free to make whatever fork or copy they want, but TCEC needs to deal with what's allowed to compete in their tournament. So many ways of copying other engines/architectures are completely fine to make your hobby engine but are not fine to compete at TCEC and shouldn't be fine to be presented in rating lists as if original engines. Fat Fritz is the 3rd engine on CCRL blitz, while Stoof should be 3rd and Fat Fritz marked as derivative.

The argument that you're free to fork GPL code and that no single chess idea should be exclusive to an engine and so on has no bearing on what rules tournaments should adopt to ensure the tournament isn't flooded with derivatives, swamping actually original engines. It makes the competition pointless, just start with a SF-Leela SuFi and be done, the lower leagues are a farce.

The originality status of an engine can evolve with time - Weiss started as a clone of VICE but so much was changed, added or completely rewritten that very little is left of the original engine, and neither search or eval are copies.

TCEC is too weak to take a stance against copy-pasting the architecture for competitive purposes. They don't want to enforce net architecture incompatibility, after all they did allow Allie which copy-pasted Leela's net architecture.

So they find themselves in a rough spot. Allie was just one engine, now they're facing a flood of NNUEs. If 10 engines copy-paste the same NNUE architecture and copy SF nets because "numbers are not the property of anybody", the whole competition is a farce. 80% of the engines effective originality is gone.

So TCEC came up with this "net must be trained on the engine's own eval" rule as a last ditch to at least give the impression that there will be originality among the NNUE engines. Realistically, some authors will care, some won't, and it's unenforceable.

[Daniel Shawul]

So many ways of copying other engines/architectures are completely fine to make your hobby engine but are not fine to compete at TCEC and shouldn't be fine to be presented in rating lists as if original engines.

To be fair to TCEC their rule says the NNUE infrastructure is considered a library.

Guidelines for use of NNUE at TCEC:
1) NNUE code can be used and considered as if it was a library (even if it is not literally one).
2) Custom modifications to the basic NNUE code are strongly encouraged, it should be considered rather like a starting point.
3) All NNUE training data should be generated by the unique engine's own search and/or eval code.

Do the NNUE guidelines apply outside NNUE technology?
No, of course not.

It sounds to me like they want to encourage variety and experimentation with NNUE but do not want to insist eveybody reinvents the wheel (the NN inference code). I don't agree with their third rule i.e. don't use SF data. But I don't belive you need to invent a new NNUE architecture to be original. Btw what exactly is this "NNUE archtecture" anwyay ? It is a highly-optimized 3-layer dense network. I hear the incremental update gains only about 11% speedup , so their is really not much originality left if you drop that for example.

So they find themselves in a rough spot. Allie was just one engine, now they're facing a flood of NNUEs. If 10 engines copy-paste the same NNUE architecture and copy SF nets because "numbers are not the property of anybody", the whole competition is a farce. 80% of the engines effective originality is gone.

The cudnn backend of Allie shared with lc0 can be considered a library too... Ankan actually gave explicit permission to Allie author AFAIK.
Someone (i think twoforce) summarized it very well. If you have the need to be _really_ original you have got to write the linux kernel, the cuda driver etc which doesn't make any sense. I recently wanted to update to cudnn-v8 but was stomped to find out its size has more than tripled and now stands at 1.2 GB. I can't fathom the amount of optimization and human hours spent on producing this library. I am just a mere user of this library and I don't need to reinvent the wheel when the focus of the competiion is AI-chess ?

[Alayan]

It's funny to see you defend this, because Scorpio is going to be crushed by the NNUE flood and low-effort eval copies.

So many ways of copying other engines/architectures are completely fine to make your hobby engine but are not fine to compete at TCEC and shouldn't be fine to be presented in rating lists as if original engines.

To be fair to TCEC their rule says the NNUE infrastructure is considered a library.

Guidelines for use of NNUE at TCEC:
1) NNUE code can be used and considered as if it was a library (even if it is not literally one).
2) Custom modifications to the basic NNUE code are strongly encouraged, it should be considered rather like a starting point.
3) All NNUE training data should be generated by the unique engine's own search and/or eval code.

Do the NNUE guidelines apply outside NNUE technology?
No, of course not.

It sounds to me like they want to encourage variety and experimentation with NNUE but do not want to insist eveybody reinvents the wheel (the NN inference code). I don't agree with their third rule i.e. don't use SF data. But I don't belive you need to invent a new NNUE architecture to be original. Btw what exactly is this "NNUE archtecture" anwyay ? It is a highly-optimized 3-layer dense network. I hear the incremental update gains only about 11% speedup , so their is really not much originality left if you drop that for example.

If they wanted to encourage variety and experimentation, they should require architectural innovations. Copying the net architecture with king-pieces input (which also makes it easy to copy SF nets) has no originality.

It's not more original in any way than copying the full evaluation code of Stockfish classical, resetting all parameters to 0, and tuning them independently.

So they find themselves in a rough spot. Allie was just one engine, now they're facing a flood of NNUEs. If 10 engines copy-paste the same NNUE architecture and copy SF nets because "numbers are not the property of anybody", the whole competition is a farce. 80% of the engines effective originality is gone.

The cudnn backend of Allie shared with lc0 can be considered a library too... Ankan actually gave explicit permission to Allie author AFAIK.
Someone (i think twoforce) summarized it very well. If you have the need to be _really_ original you have got to write the linux kernel, the cuda driver etc which doesn't make any sense. I recently wanted to update to cudnn-v8 but was stomped to find out its size has more than tripled and now stands at 1.2 GB. I can't fathom the amount of optimization and human hours spent on producing this library. I am just a mere user of this library and I don't need to reinvent the wheel when the focus of the competiion is AI-chess ?

Allie has all rights to exist. Violates no license, no moral rules, nothing. Should it compete in tournaments ? I think the lack of originality on eval and the lack of originality in search (because it uses the exact same policy system to guide its search, making its search more similar to Leela's than 2 AB's engine searches are to each other...), arguably not.

The linux kernel and cuda driver argument is completely stupid. They aren't part of the chess program and don't contribute to chess move selection at all.

The issue with what Allie uses from Leela isn't being able to use GPU efficiently, it's that the net architecture is copy-pasted making the engine behavior much more similar.

[AndrewGrant]

I've sent 12.75 to TCEC for future events (I'm sure they'll remove Ethereal soon via the 'not developed' rule because the leadership dislikes me for fighting for whats right).

I shared 12.75-SFNNUE with them as well. TCEC has two options: Open the field with NNUEs and allow anyone to do it, thus allowing all engines to easily retrofit NNUE and gain similar elo. Or to bar NNUE entries that are not from Stockfish. You can't have it both ways without some hypocricy. Unless of course they want to ban the engines that have trained on Stockfish data.

I've left the TCEC Discord, and asked that they not contact me again. They are hands down the most unprofessional tournament for any game or event I have ever seen. They do not make rules to fit the needs of the competition. They make rules to fit the needs of the engines they want in the competition.

I was planning to obfuscate the Stockfish Network and send that into TCEC, absolutely demolish Leela and Allie, which would cause the directors to take some steps. I don't think I care enough to do it. I don't need to prove the shameless favoring of Leela and Allie -- its apparent to everyone.

I hope TCEC crashes and burns and something better replaces it. CCC has the same issues with a dozen forks of the same program, some programs using Stockfish Networks some not. BUT, CCC does not claim to be this great bastion of competitive play, the highest quality tournament in the land. They are just a venue that plays games, so thats alright.

[MikeB]

I've sent 12.75 to TCEC for future events (I'm sure they'll remove Ethereal soon via the 'not developed' rule because the leadership dislikes me for fighting for whats right).

I shared 12.75-SFNNUE with them as well. TCEC has two options: Open the field with NNUEs and allow anyone to do it, thus allowing all engines to easily retrofit NNUE and gain similar elo. Or to bar NNUE entries that are not from Stockfish. You can't have it both ways without some hypocricy. Unless of course they want to ban the engines that have trained on Stockfish data.

I've left the TCEC Discord, and asked that they not contact me again. They are hands down the most unprofessional tournament for any game or event I have ever seen. They do not make rules to fit the needs of the competition. They make rules to fit the needs of the engines they want in the competition.

I was planning to obfuscate the Stockfish Network and send that into TCEC, absolutely demolish Leela and Allie, which would cause the directors to take some steps. I don't think I care enough to do it. I don't need to prove the shameless favoring of Leela and Allie -- its apparent to everyone.

I hope TCEC crashes and burns and something better replaces it. CCC has the same issues with a dozen forks of the same program, some programs using Stockfish Networks some not. BUT, CCC does not claim to be this great bastion of competitive play, the highest quality tournament in the land. They are just a venue that plays games, so thats alright.

TCEC has been nothing more than a glorified basement tourney for years— not too different than anyone can run. The right way to do it , it to get the "authentic" authors together , decide what the rules and which engines are in . It would not hurt to have some "retired chess programers" in as well to provide some guidance. But the horse has probably already left the barn.

[carldaman]

Just developed should not equal 'not developed'. Besides, someone could change their mind about further development.

You're right about TCEC being unreasonable, though. That's their trademark.

[Daniel Shawul]

It's funny to see you defend this, because Scorpio is going to be crushed by the NNUE flood and low-effort eval copies.

Unlike you my opinions are not affected by whether my engine is not in DivP or not

So many ways of copying other engines/architectures are completely fine to make your hobby engine but are not fine to compete at TCEC and shouldn't be fine to be presented in rating lists as if original engines.

To be fair to TCEC their rule says the NNUE infrastructure is considered a library.

Guidelines for use of NNUE at TCEC:
1) NNUE code can be used and considered as if it was a library (even if it is not literally one).
2) Custom modifications to the basic NNUE code are strongly encouraged, it should be considered rather like a starting point.
3) All NNUE training data should be generated by the unique engine's own search and/or eval code.

Do the NNUE guidelines apply outside NNUE technology?
No, of course not.

It sounds to me like they want to encourage variety and experimentation with NNUE but do not want to insist eveybody reinvents the wheel (the NN inference code). I don't agree with their third rule i.e. don't use SF data. But I don't belive you need to invent a new NNUE architecture to be original. Btw what exactly is this "NNUE archtecture" anwyay ? It is a highly-optimized 3-layer dense network. I hear the incremental update gains only about 11% speedup , so their is really not much originality left if you drop that for example.

If they wanted to encourage variety and experimentation, they should require architectural innovations. Copying the net architecture with king-pieces input (which also makes it easy to copy SF nets) has no originality.

There aren't many neural network architectures to begin with anyway. Even NNUE is a regular three-layer dense net, and uses a "similar trick" of splitting the inputs into two to save computation as Giraffe does. So do you consider NNUE a clone of Giraffe net then ?
For deep nets, you can count the type of architectures on your fingers. A neural network is so robust that the architecture plays a limited role eventually e.g. using Squeeze-excitation vs not.

It's not more original in any way than copying the full evaluation code of Stockfish classical, resetting all parameters to 0, and tuning them independently.

So they find themselves in a rough spot. Allie was just one engine, now they're facing a flood of NNUEs. If 10 engines copy-paste the same NNUE architecture and copy SF nets because "numbers are not the property of anybody", the whole competition is a farce. 80% of the engines effective originality is gone.

The cudnn backend of Allie shared with lc0 can be considered a library too... Ankan actually gave explicit permission to Allie author AFAIK.
Someone (i think twoforce) summarized it very well. If you have the need to be _really_ original you have got to write the linux kernel, the cuda driver etc which doesn't make any sense. I recently wanted to update to cudnn-v8 but was stomped to find out its size has more than tripled and now stands at 1.2 GB. I can't fathom the amount of optimization and human hours spent on producing this library. I am just a mere user of this library and I don't need to reinvent the wheel when the focus of the competiion is AI-chess ?

Allie has all rights to exist. Violates no license, no moral rules, nothing. Should it compete in tournaments ? I think the lack of originality on eval and the lack of originality in search (because it uses the exact same policy system to guide its search, making its search more similar to Leela's than 2 AB's engine searches are to each other...), arguably not.

The linux kernel and cuda driver argument is completely stupid. They aren't part of the chess program and don't contribute to chess move selection at all.

The issue with what Allie uses from Leela isn't being able to use GPU efficiently, it's that the net architecture is copy-pasted making the engine behavior much more similar.

News flash... almost everybody (chess,Go, Hex engines etc) that reimplements AlphaZero uses ResNet (one architecture). DeepMind used to use CNNs before before ResNets are invented. Latest net architecture that does well on imagenet competition is a good candidate for trying in chess. Neither leela nor A0 owns that architecture so this is hardly a criteria to disqualify Allie. If you mean the "cudnn" inference code that allie uses (which is not a net architecture), then I really think Leela shouldn't have written that code in the first place. For every neural architecture you want to write you have to modify your code accordingly. I get the same performance using libraries (TensorRT + cudnn) that will automatically optimize any net for you on the GPU. You could even use TensorFlow only which you can compile with cudnn/tensorRT/opencl support and never worry about anything. Even NNUE can be done with tensorflow, but Tensorflow has a significant overhead (i think 20ms per call) that will disqualify tiny nets like NNUE which are run on CPU. If your NN is bulky enough, this overhead doesn't matter.