Hard-Talkchess-2020 set, final release

[AlexChess]

Can someone recommend a good / or the "best" net foe lc0 for testsuites?
Please with a download link.
Thx in advance.

I'm using 42850.pb.gz and is not bad (3400+ ELO) even on the slow mac M1

PS: If you are able, could you compile for Mac M1 Fire NN? I'd like to use Hard-test 2022 with it.

[criko]

Do you have a link for the net?

[peter]

Can someone recommend a good / or the "best" net foe lc0 for testsuites?
Please with a download link.
Thx in advance.

Hi!

Depends much on the hardware and the compile, you're using. For my Nvidia 3070ti this one

https://storage.lczero.org/files/networ ... 0000.pb.gz

was best rather long as for short TC- testing and the

lc0-v0.29.0-rc0 way back then, even bigger nets being somewhat slower.

Then I changed together with the dag- versions of the engine to the one TCEC used then, nr. 784038 from here

https://training.lczero.org/networks/?show_all=1

Now and then I have a look at this one selfplay- ranking here

http://zz4032.rf.gd/lc0/charts.html?i=1

And you probably read the latest thread about that at CSS anyhow too

https://forum.computerschach.de/cgi-bin ... #pid157669

Best regards

[AlexChess]

Do you have a link for the net?

https://pixeldrain.com/u/DhNzKNJg

[criko]

Thx for your answers!!

https://storage.lczero.org/files/networ ... 0000.pb.gz

I will try this one from Peter.

[criko]

Sorry I forgot.

Any suggestions for the lc0.config?

RTX 3070

[peter]

Sorry I forgot.

Any suggestions for the lc0.config?

RTX 3070

I don't bother anymore with trying to find out best setting for own installation, default always was best or almost best at most of my trials anyhow for my GPU, which is very near to yours, at least lately it seemed to fit quite well.
Yet I changed compile latest to the one of CSS- thread and it seemed to score a little bit better (within error bar of course and with about same node- count) with default configuration than the last but one dag-tt I had before did (have a look at the ranking- list I gave in same CSS- thread).

From here

https://ci.appveyor.com/project/Etcaqab ... s/44614483

you have to choose the environment you prefer (probably Cuda as well as I did)

https://ci.appveyor.com/project/Etcaqab ... s/44614483

and then with artifacts

https://ci.appveyor.com/project/Etcaqab ... /artifacts

download

build/lc0.exe

Notice you have to replace a binary already being in old installation- folder including all necessary files that don't come with new download of lc0.exe again, as for .config, I don't use it manually edited but change settings for different purposes (e.g. NNCacheSize) in certain GUIs and store different configurations in UCI- file made by GUI. if you have old .config in your installation- folder rather delete it before new installation in GUI.

Edit: hope the ap-mish-net still works with latest compile, didn't try so, downloading new net with new compile and replacing both in doubled installation- folder.
Be sure to keep a copy of the old folder with the old binary, just in case if there's a mismatch of net and binary yet maybe

[criko]

Hi Peter,

THX a lot!!!

[Plutie]

Sorry I forgot.

Any suggestions for the lc0.config?

RTX 3070

I don't bother anymore with trying to find out best setting for own installation, default always was best or almost best at most of my trials anyhow for my GPU, which is very near to yours, at least lately it seemed to fit quite well.
Yet I changed compile latest to the one of CSS- thread and it seemed to score a little bit better (within error bar of course and with about same node- count) with default configuration than the last but one dag-tt I had before did (have a look at the ranking- list I gave in same CSS- thread).

From here

https://ci.appveyor.com/project/Etcaqab ... s/44614483

you have to choose the environment you prefer (probably Cuda as well as I did)

https://ci.appveyor.com/project/Etcaqab ... s/44614483

and then with artifacts

https://ci.appveyor.com/project/Etcaqab ... /artifacts

download

build/lc0.exe

Notice you have to replace a binary already being in old installation- folder including all necessary files that don't come with new download of lc0.exe again, as for .config, I don't use it manually edited but change settings for different purposes (e.g. NNCacheSize) in certain GUIs and store different configurations in UCI- file made by GUI. if you have old .config in your installation- folder rather delete it before new installation in GUI.

Edit: hope the ap-mish-net still works with latest compile, didn't try so, downloading new net with new compile and replacing both in doubled installation- folder.
Be sure to keep a copy of the old folder with the old binary, just in case if there's a mismatch of net and binary yet maybe

the dag-size branch currently has some big issues, causing huge playing strength drops. I'd recommend you use the dag-bord-lf-se-2 branch instead.

best network is probably 784822 right now. (actually, the WIP transformer networks are probably better, but those are a pain to get working properly at the moment)

regarding settings, you can test these: cpuct 1.53, factor 3.15, base 35560, fpu 0.35, pst 1.22. might be worth testing with default settings as well. another thing worth changing would be minibatch size, which should be 192 per GPU used. the rest aren't worth changing for the sake of this test.

[peter]

the dag-size branch currently has some big issues, causing huge playing strength drops. I'd recommend you use the dag-bord-lf-se-2 branch instead.

best network is probably 784822 right now. (actually, the WIP transformer networks are probably better, but those are a pain to get working properly at the moment)

regarding settings, you can test these: cpuct 1.53, factor 3.15, base 35560, fpu 0.35, pst 1.22. might be worth testing with default settings as well. another thing worth changing would be minibatch size, which should be 192 per GPU used. the rest aren't worth changing for the sake of this test.

Thanks for your tips!
I was just refering to my own short experiences with mainly positional testing (which was criko's question too) as for those nets and compiles given by links by me, those worked well with defualt settings of engine- versions so far, will give your parameter- setting a try, but problem as for that*s the same as for engine- net- combination only:
You have to cross- test always combinations of certain compile (a) with default setting (1) with certain net (x) against same compile with same setting combined with net y, then compile b with default setting and net x, to be sure about which engines works best in which setting with which net.
a1x against a1y, then a1x against b1x, then a1x against a2x, doing this kind of single parameter checking doing by eng-eng-matches only, you won't get statistically relevant results before next net or engine- version will come out

That's why I compare with for me and my own interest relevant positional test suites too, like I did e.g. here

https://forum.computerschach.de/cgi-bin ... #pid157676

Of course those results are quite different ones compared to eng-eng-game-playing, but I get them quickly and they are reproducible and statistically significant within certain also well known error bars.
As for eng-eng- game playing you have prinicpally same problem of test- postions to start from too, different openings, different results, not to talk about selection- problem of hardware- time and number and variety of opponents.

But that all's just my two cents regards