Time handicap tournament. LC0 and SF11

[mmt]

I'll be running various time handicap matches at different time controls, starting with self-play. Probably limited to LC0 and SF 11. I'll keep the log here. Related to http://talkchess.com/forum3/viewtopic.php?f=2&t=73000.

Started with:
Match: LC0 30s + 1s vs LC0 15s + 0.5s
Settings: Default settings + 90000000 cache, net 59098, 4 threads, fp16, roundrobin
Hardware: Titan V + 2080Ti
Book: Silver-Suite-Large random with reversed color
Tablebases: Complete 6-piece and around 40% of 7-piece
Software: Arena 3.5.1, Windows
Result after 100 games:

lc0 59x 30s+1s - lc0 59x 15s+0.5s : 54.0/100 10-2-88 (=====11=============1==0==1=======================1===========1===1===11================1====0======) 54% +28

lc0 59x 15s+0.5s - lc0 59x 30s+1s : 46.0/100 2-10-88 (=====00=============0==1==0=======================0===========0===0===00================0====1======) 46% -28

[mmt]

200 games:
lc0 59x 30s+1s - lc0 59x 15s+0.5s :114.0/200 30-2-168
Elo: +49 -49
Ordo:
# PLAYER : RATING ERROR POINTS PLAYED (%)
1 lc0 59x 30s+1 : 2324.7 17.9 114.0 200 57
2 lc0 59x 2 15s+0.5s : 2275.3 17.9 86.0 200 43

[Guenther]

I'll be running various time handicap matches at different time controls, starting with self-play. Probably limited to LC0 and SF 11. I'll keep the log here. Related to http://talkchess.com/forum3/viewtopic.php?f=2&t=73000.

Started with:
Match: LC0 30s + 1s vs LC0 15s + 0.5s
Settings: Default settings + 90000000 cache, net 59098, 4 threads, fp16, roundrobin
Hardware: Titan V + 2080Ti
Book: Silver-Suite-Large random with reversed color
Tablebases: Complete 6-piece and around 40% of 7-piece
Software: Arena 3.5.1, Windows
Result after 100 games:

lc0 59x 30s+1s - lc0 59x 15s+0.5s : 54.0/100 10-2-88 (=====11=============1==0==1=======================1===========1===1===11================1====0======) 54% +28

lc0 59x 15s+0.5s - lc0 59x 30s+1s : 46.0/100 2-10-88 (=====00=============0==1==0=======================0===========0===0===00================0====1======) 46% -28

I don't see how this is related to the thread you give above and what you are trying to measure here?
(we already know rough values for time doubling with various programs over thousands of games depending on tc)

BTW are the games available?
(Usually, if the same entity plays against itself with time odds and the handicapped 'clone' loses some games, it might indicate
bad start positions. Moreover I am not sure time odds implementation in Arena is safe, I would use CuteChess-cli or WB here)

[Alayan]

Usually, if the same entity plays against itself with time odds and the handicapped 'clone' loses some games, it might indicate
bad start positions.

This doesn't make any sense, the version with more time plays stronger, it's expected for the one with less time to lose to it.

[Guenther]

Usually, if the same entity plays against itself with time odds and the handicapped 'clone' loses some games, it might indicate
bad start positions.

This doesn't make any sense, the version with more time plays stronger, it's expected for the one with less time to lose to it.

Ofc I wanted to write ...the not handicapped... - should be self explaining - just a typo, sorry.

[mmt]

I don't see how this is related to the thread you give above and what you are trying to measure here?
(we already know rough values for time doubling with various programs over thousands of games depending on tc)

I'm looking to see how the Elo/Ordo difference will change with longer time controls. I saw this done before but not with the latest LC0 nets and SF 11. I think it'll also be interesting to see how the curves differ for LC0 and SF.

BTW are the games available?
(Usually, if the same entity plays against itself with time odds and the handicapped 'clone' loses some games, it might indicate
bad start positions. Moreover I am not sure time odds implementation in Arena is safe, I would use CuteChess-cli or WB here)

I'll make games available periodically in bigger batches or just make a google drive directory - too much pain to upload each time I update.
The Arena is doing ok so far with self-play but it seems impossible to use it for SF vs LC0 matches because the number of threads is global. And it's definitely buggy. I didn't see the time handicap possibility in CuteChess GUI (1.0) so I didn't consider it for this match yet.

[mmt]

Match: LC0 5s + 0.5s vs LC0 2.5s + 0.25s
Settings: Default settings + 90000000 cache, net 59098, 4 threads, fp16, roundrobin
Hardware: Titan V + 2080Ti
Book: Silver-Suite-Large random with reversed color
Tablebases: Complete 6-piece and around 40% of 7-piece
Software: Arena 3.5.1, Windows

Result after 1000 games:
lc0 59x 5s/0.5s 617.5/1000 249-14-73 +85
lc0 59x 2.5s/0.25s 382.5/100 14-249-737 -85

Ordo (baseline 2300):
# PLAYER : RATING ERROR POINTS PLAYED (%)
1 lc0 5s/0.5s : 2342.0 7.5 617.5 1000 62
2 lc0 2.5s/0.25s : 2258.0 7.5 382.5 1000 38
Difference : 84

[mmt]

Match: Stockfish 11 5s + 0.5s vs Stockfish 11 2.5s + 0.25s
Settings: 31 threads, 20 GB cache
Hardware: 2x Xenon Gold 6246
Book: Silver-Suite-Large random. With reversed colors
Tablebases: Complete 6-piece and a good part of 7-piece TBs
Software: Arena 3.5.1, Windows

Results after 1000 games:
Stockfish 11 5s + 0.5s: 544.0/1000 125-37-838 +28
Stockfish 11 2.5s + 0.25s: 456.0/1000 37-125-838 -28

Ordo (baseline 2300):
# PLAYER : RATING ERROR POINTS PLAYED (%)
1 Stockfish 11 5s + 0.5s : 2315.5 7.4 544.0 1000 54
2 Stockfish 11 2.5s + 0.25s : 2284.5 7.4 456.0 1000 46
Difference: 31

[Alayan]

20GB hash for ultra-bullet ? That's unreasonable.

[mmt]

Hmm, I don't think a different value would make much of a difference, it's self-play with the same cache setting for both sides. But if less is better at these time controls then SF should figure it out itself and accordingly use less RAM.