Leela data worth +20 elo

[Koivisto]

Hi Vernon,

I'm only posting here to confirm that although Andrew, Connor and some other engine authors don't approve of certain engines in the CCRL lists, they've not tried to directly pressure us into removing them.

In fact, I'm on good terms with them, just as I am with you.

Graham.

This is highly misleading at best. Maybe we (Koi team) aren't included in "they" I guess. We have repeadetly asked you to remove Koivisto from the CCRL list, for as long as FF 2 appears on it. Imo that is very clearly pressuring you. You are giving FF 2 credibility. Giving scams credibility is bad.

[Frank Quisinsky]

Hi Kim,

I think CEGT or CCRL testers will get the software from Chessbase (not sure).
In both systems is Fat Fritz II, or Houdini listed.

I am thinking to test NNSf (Neural Network Stockfish file) is OK, to test clones absolutely not OK.

I have 7 NNSf engines in my list.

- Orion 0.8 NNSf
- Gogobello 3.0 NNSf
- Caissa 0.4 NNSf
- Rodent 1.0 NNSf
- Coiled 1.1 NNSf
- Fire 8 MC.3 NNSf (controversial). In my humble opinion not based on SF.
- Andscacs 0.1 NNSf

With "run-4" the first NNRe (based on Rebel Neural Network).
- TogaIV 1.1 NNRe

In the future much more of such engines will be available (easy to see).
I hope that the programmers develops an own network.
The Caissa programmer is working on it.

In the thread here I am reading Nemorino based on Neural Network from Stockfish.
Only the version 6.00, not newer version I have in my list.

In the available rating systems all is very confused.

Have a look on CEGT ...
Nemorino 6.00 NN inside but all the others with Stockfish NN file missed.
Clones are inside, private available programs are inside.

Other example:
Different dev versions tested, different others not tested.

If you are looking on other rating-systems ... much more confused as in CEGT / CCRL.

Will give the information:
Not easy at all ... people have different opinions about it.

Programmers should be accept the rules of the rating-systems because all the work around it do people in free time.
But I can understand your opinion!

No "king solution" I think!

Best
Frank

[jdart]

So the leela data is huge and of very high quality. You need lots of positions, 10 billion plus, to train these big nets. More like 20 billion if you want to be safe.

Arasan's latest net was trained on about 3.5B positions, plus another 300M or so for reinforcement learning. So you don't need 10B. That's not to say more data would not be helpful.

[dkappe]

So the leela data is huge and of very high quality. You need lots of positions, 10 billion plus, to train these big nets. More like 20 billion if you want to be safe.

Arasan's latest net was trained on about 3.5B positions, plus another 300M or so for reinforcement learning. So you don't need 10B. That's not to say more data would not be helpful.

I think Arasan uses the old nnue architecture, no? There you can get a good result with just 600m positions. With the new bucket scheme, it’s almost like training 8 nets at the same time. There you can’t do with less than 10b.

[Sopel]

The CCRL's most recent policy has been to exclude valid engines because their NNUE was not created from its own engines games.
Interestingly enough, even though SF and Leela share data they are eagerly tested by CCRL.

what hyprocrisy
my guess: a new convoluted and ignorant policy will soon be communicated which attempts to justify their decisions in this regard..
after which connor mcmonicle with chime in to defend the illogical with some obtuse tech jargon mumbo jumbo.

Eagerly awaiting the time CCRL tests Fat Fritz 2 but not Stockfish.

[Vernon Crawford]

This policy is not officially stated anywhere, though it does seem to be the policy CCRL is employing (as a consequence of testers not being interested in testing such engines). Hopefully that clarifies what I meant.

Your foot in your mouth?...no clarification necessary, thanks anyway.

PS - It's clear they will never issue an 'official' policy, much safer to avoid criticism by remaining a moving target...
Sad thing is to see these testing groups buckle under pressure from you and Andrew and others with hateful narrow-minded agendas so easily.
That's a real tragedy.

That the concept of an unofficial policy eludes you should probably not come as a surprise considering you have comprehension issues by your own earlier admission.

"hateful narrow-minded agenda"
That's some laughable hyperbole. I'm genuinely curious as to why you care so deeply about this, Vernon. You don't even seem particularly interested in computer chess based on your post history... odd.

I "don't even seem particularly interested" ? Your ignorance astounds. I own and operate a computer chess site. Check the URL in my signature.

[Sopel]

So the leela data is huge and of very high quality. You need lots of positions, 10 billion plus, to train these big nets. More like 20 billion if you want to be safe.

Arasan's latest net was trained on about 3.5B positions, plus another 300M or so for reinforcement learning. So you don't need 10B. That's not to say more data would not be helpful.

For the current stockfish arch 4B positions seems to be the minimum to stop observing overfitting, and the rest is worth maybe 2-3 elo. You can try digging up some data on this from here https://docs.google.com/document/d/1gTl ... dlhbjev7ve if you have time. dkappe's figure is about right, though on the safe side.

edit. dug it out

very low time control so elo figures are ~2x of fishtest STC

[Vernon Crawford]

To suggest my personal leanings have any impact on CCRL is generous.

"CEGT did pull the Fire versions running Stockfish nets after I pointed it out"

sad to see that they would cave in just from the threat of not receiving your questionable 40 dollar engine...pathetic.

Sorry but, what part of the phrase "after I pointed it out" did you read as "threatened someone if they did not do what I wanted"?

Heres a rough transcript for you:

Andrew: yo, by the way that Fire version you just tested is using a Stockfish net
CEGT: No shot, really? You sure? I know you've long been of the position that Stockfish = Fire.
Andrew: Yeah don't take my word for it, heres the official release, where Norman states its a net from Sergio <link>
CEGT: Thanks.

Happy to provide any more clarity where needed.

Using an eval NNUE based on SF games is not the same as being = Stockfish. How quickly you can jump to that is astounding.
Fact is:
You've crapped on many engines for years
You've embarrassed yourself with ridiculous and egotistical posts about how other than SF, Ethereal and Seer are the only 2 truly 'original' engines
After significant spamming to this end, and effectively influencing enough people, you decided people now must pay for your incredibly 'original' work.
All of which makes perfect sense considering your agenda of self-promotion at all expense, followed by a greedy self-righteous monetary capitalization...while all the time using hundreds of lines (full pages) of code directly taken from SF can be seen in Ethereal (as evidenced by links for comparison in engine origins). Let me know if you're unaware of these, I'll link to them.
All in all it's absolutely pathetic behavior.

Maybe you can get Trump to help you further...the 2 of you seem perfectly compatible.

[AndrewGrant]

Using an eval NNUE based on SF games is not the same as being = Stockfish. How quickly you can jump to that is astounding.
Fact is:
You've crapped on many engines for years
You've embarrassed yourself with ridiculous and egotistical posts about how other than SF, Ethereal and Seer are the only 2 truly 'original' engines
After significant spamming to this end, and effectively influencing enough people, you decided people now must pay for your incredibly 'original' work.
All of which makes perfect sense considering your agenda of self-promotion at all expense, followed by a greedy self-righteous monetary capitalization...while all the time use hundreds of lines (full pages) of code directly taken from SF can be seen in Ethereal (as evidenced by links for comparison in engine origins).
All in all it's absolutely pathetic behavior.

If your intent is to reopen the Fire originality argument, then that should be done or carried on in the Engine Origins thread.

As for your claim about Ethereal, you sound a lot like Wilsonz, who seems to have disappeared. Feel free to start a thread in Engine origins about Ethereal if you feel that I have wronged another engine. If I'm bored, I'll throw a response or two.

And to be clear, I have only complained about Fire and Houdini. Houdini is quite literally Stockfish, which is a position that I don't think anyone refutes. So the claim that I am rampant against other engines... not quite founded. Two engines, one universally agreed to be a clone.

As for the greedy comment... You ever heard of OpenBench? Lol. You don't know what you are talking about, and you are disrepecting yourself by coming in here hot without enough information.

And uh, what does any of this have to do with Leela data?

[AndrewGrant]

Using an eval NNUE based on SF games is not the same as being = Stockfish. How quickly you can jump to that is astounding.

And for some EXTRA bonus clarity:

1. The Fire version we are talking about is not simply using a Network trained ON Stockfish games.
2. The Fire version we are talking about is using a Network that WAS at one point FOR Stockfish.
3. My claim that Fire = Stockfish is separate from any conversation about NNUE origins.

Hope that clears things up for you. I noticed the "SF games is not"... statement after sending the original response.
Perhaps now you understand the argument better.