Something goes wrong with lc0 since yesterday?

[yanquis1972]

reassuring +230 elo leap on the last testserver net. i was beginning to worry this one was going to need a reset. elevator up, as it should be for a bit more...

meanwhile, mainserver is still making real gains & playing some impressive chess. endgames have gone from a huge issue to a plus (against my current testmule SF5, anyway) in a week or so.

[CMCanavessi]

reassuring +230 elo leap on the last testserver net. i was beginning to worry this one was going to need a reset. elevator up, as it should be for a bit more...

Kinda expected, as just now the almost-random games from the 1st couple of networks are starting to get out of the training window (500k games). In theory, we should see a couple of big elo gains in the coming networks. Let's pray

meanwhile, mainserver is still making real gains & playing some impressive chess. endgames have gone from a huge issue to a plus (against my current testmule SF5, anyway) in a week or so.

Still not at the level of 390, in my tests. Doing a gauntlet with 455 now to see how close it is, and if it's indeed stronger or not. (check the recent match between 390 and 455 in the matches page at lczero.org, 390 in self-play is still +25 elo)

[Ron Langeveld]

Can you repeat your tests with 10010 and 9010? What hardware are you running them on?

I don't have time to run tests on demand, sorry.
These test are run on 780Ti. This is about 3,5x slower than 1080Ti in nps.
The STS test for ID10010 is finished. Score is 56,3%
Just below (6x64) ID4009 that scored 57,7%

[yanquis1972]

455 lost its first game against SF5, & it was a doozy: https://lichess.org/study/YFUzMu1u/Qu1PN5da

[Laskos]

reassuring +230 elo leap on the last testserver net. i was beginning to worry this one was going to need a reset. elevator up, as it should be for a bit more...

Kinda expected, as just now the almost-random games from the 1st couple of networks are starting to get out of the training window (500k games). In theory, we should see a couple of big elo gains in the coming networks. Let's pray

meanwhile, mainserver is still making real gains & playing some impressive chess. endgames have gone from a huge issue to a plus (against my current testmule SF5, anyway) in a week or so.

Still not at the level of 390, in my tests. Doing a gauntlet with 455 now to see how close it is, and if it's indeed stronger or not. (check the recent match between 390 and 455 in the matches page at lczero.org, 390 in self-play is still +25 elo)

And the gains of test server nets ID10010 and ID10012 are real Elo gains against AB engines, not just artifacts of self-play. By now ID10012 is about 2200 CCRL 40/4' Elo level. Shown Elo in self-games is about 2700, so there is some rating inflation. As for the main server nets, ID452 still performed a bit worse than ID395 against AB engines, but within error margins. So, no progress yet compared to older ID390 and ID395 nets.

[Laskos]

reassuring +230 elo leap on the last testserver net. i was beginning to worry this one was going to need a reset. elevator up, as it should be for a bit more...

Kinda expected, as just now the almost-random games from the 1st couple of networks are starting to get out of the training window (500k games). In theory, we should see a couple of big elo gains in the coming networks. Let's pray

meanwhile, mainserver is still making real gains & playing some impressive chess. endgames have gone from a huge issue to a plus (against my current testmule SF5, anyway) in a week or so.

Still not at the level of 390, in my tests. Doing a gauntlet with 455 now to see how close it is, and if it's indeed stronger or not. (check the recent match between 390 and 455 in the matches page at lczero.org, 390 in self-play is still +25 elo)

And the gains of test server nets ID10010 and ID10012 are real Elo gains against AB engines, not just artifacts of self-play. By now ID10012 is about 2200 CCRL 40/4' Elo level. Shown Elo in self-games is about 2700, so there is some rating inflation. As for the main server nets, ID452 still performed a bit worse than ID395 against AB engines, but within error margins. So, no progress yet compared to older ID390 and ID395 nets.

Strange result (regression) of ID10014 against an AB engine compared to ID10012, 50 Elo points regression, almost within error margins, but their self-games show a 100 Elo points progress. Let's see.

[Ron Langeveld]

Strange result (regression) of ID10014 against an AB engine compared to ID10012, 50 Elo points regression, almost within error margins, but their self-games show a 100 Elo points progress. Let's see.

STS test for ID10014 also regressed. 58,3% where ID 10013 scored 62,7% and ID10012 60,9%

[yanquis1972]

continues to add 50+ elo per net, so either something's terribly off or it's some sort of cohesion process. atm i'm optimistic & expecting a massive jump on any given net.

[Laskos]

continues to add 50+ elo per net, so either something's terribly off or it's some sort of cohesion process. atm i'm optimistic & expecting a massive jump on any given net.

ID10015 progressed to the level of ID10012, so there is a progress now, but I don't know why that regression appeared for ID10014. And in their ratings, ID10015 should be 170 Elo points above ID10012, while against AB engines, they are similar in strength.

[Laskos]

continues to add 50+ elo per net, so either something's terribly off or it's some sort of cohesion process. atm i'm optimistic & expecting a massive jump on any given net.

ID10015 progressed to the level of ID10012, so there is a progress now, but I don't know why that regression appeared for ID10014. And in their ratings, ID10015 should be 170 Elo points above ID10012, while against AB engines, they are similar in strength.

Now a sudden positive jump of about 160 Elo points from ID10015 to ID10017. Pretty erratic behavior against AB engines. So, CCRL 40/4' ratings on GTX 1060 are following:
ID10015 about 2200
ID10017 about 2350