I'll be running various time handicap matches at different time controls, starting with self-play. Probably limited to LC0 and SF 11. I'll keep the log here. Related to http://talkchess.com/forum3/viewtopic.php?f=2&t=73000.
Started with:
Match: LC0 30s + 1s vs LC0 15s + 0.5s
Settings: Default settings + 90000000 cache, net 59098, 4 threads, fp16, roundrobin
Hardware: Titan V + 2080Ti
Book: Silver-Suite-Large random with reversed color
Tablebases: Complete 6-piece and around 40% of 7-piece
Software: Arena 3.5.1, Windows
Result after 100 games:
lc0 59x 30s+1s - lc0 59x 15s+0.5s : 54.0/100 10-2-88 (=====11=============1==0==1=======================1===========1===1===11================1====0======) 54% +28
lc0 59x 15s+0.5s - lc0 59x 30s+1s : 46.0/100 2-10-88 (=====00=============0==1==0=======================0===========0===0===00================0====1======) 46% -28
Time handicap tournament. LC0 and SF11
Moderators: hgm, Rebel, chrisw
-
- Posts: 343
- Joined: Sun Aug 25, 2019 8:33 am
- Full name: .
-
- Posts: 343
- Joined: Sun Aug 25, 2019 8:33 am
- Full name: .
Re: Time handicap tournament. LC0 and SF11
200 games:
lc0 59x 30s+1s - lc0 59x 15s+0.5s :114.0/200 30-2-168
Elo: +49 -49
Ordo:
# PLAYER : RATING ERROR POINTS PLAYED (%)
1 lc0 59x 30s+1 : 2324.7 17.9 114.0 200 57
2 lc0 59x 2 15s+0.5s : 2275.3 17.9 86.0 200 43
lc0 59x 30s+1s - lc0 59x 15s+0.5s :114.0/200 30-2-168
Elo: +49 -49
Ordo:
# PLAYER : RATING ERROR POINTS PLAYED (%)
1 lc0 59x 30s+1 : 2324.7 17.9 114.0 200 57
2 lc0 59x 2 15s+0.5s : 2275.3 17.9 86.0 200 43
-
- Posts: 4611
- Joined: Wed Oct 01, 2008 6:33 am
- Location: Regensburg, Germany
- Full name: Guenther Simon
Re: Time handicap tournament. LC0 and SF11
I don't see how this is related to the thread you give above and what you are trying to measure here?mmt wrote: ↑Sat Feb 08, 2020 1:46 am I'll be running various time handicap matches at different time controls, starting with self-play. Probably limited to LC0 and SF 11. I'll keep the log here. Related to http://talkchess.com/forum3/viewtopic.php?f=2&t=73000.
Started with:
Match: LC0 30s + 1s vs LC0 15s + 0.5s
Settings: Default settings + 90000000 cache, net 59098, 4 threads, fp16, roundrobin
Hardware: Titan V + 2080Ti
Book: Silver-Suite-Large random with reversed color
Tablebases: Complete 6-piece and around 40% of 7-piece
Software: Arena 3.5.1, Windows
Result after 100 games:
lc0 59x 30s+1s - lc0 59x 15s+0.5s : 54.0/100 10-2-88 (=====11=============1==0==1=======================1===========1===1===11================1====0======) 54% +28
lc0 59x 15s+0.5s - lc0 59x 30s+1s : 46.0/100 2-10-88 (=====00=============0==1==0=======================0===========0===0===00================0====1======) 46% -28
(we already know rough values for time doubling with various programs over thousands of games depending on tc)
BTW are the games available?
(Usually, if the same entity plays against itself with time odds and the handicapped 'clone' loses some games, it might indicate
bad start positions. Moreover I am not sure time odds implementation in Arena is safe, I would use CuteChess-cli or WB here)
https://rwbc-chess.de
trollwatch:
Talkchess nowadays is a joke - it is full of trolls/idiots/people stuck in the pleistocene > 80% of the posts fall into this category...
trollwatch:
Talkchess nowadays is a joke - it is full of trolls/idiots/people stuck in the pleistocene > 80% of the posts fall into this category...
-
- Posts: 550
- Joined: Tue Nov 19, 2019 8:48 pm
- Full name: Alayan Feh
Re: Time handicap tournament. LC0 and SF11
This doesn't make any sense, the version with more time plays stronger, it's expected for the one with less time to lose to it.
-
- Posts: 4611
- Joined: Wed Oct 01, 2008 6:33 am
- Location: Regensburg, Germany
- Full name: Guenther Simon
Re: Time handicap tournament. LC0 and SF11
Ofc I wanted to write ...the not handicapped... - should be self explaining - just a typo, sorry.
https://rwbc-chess.de
trollwatch:
Talkchess nowadays is a joke - it is full of trolls/idiots/people stuck in the pleistocene > 80% of the posts fall into this category...
trollwatch:
Talkchess nowadays is a joke - it is full of trolls/idiots/people stuck in the pleistocene > 80% of the posts fall into this category...
-
- Posts: 343
- Joined: Sun Aug 25, 2019 8:33 am
- Full name: .
Re: Time handicap tournament. LC0 and SF11
I'm looking to see how the Elo/Ordo difference will change with longer time controls. I saw this done before but not with the latest LC0 nets and SF 11. I think it'll also be interesting to see how the curves differ for LC0 and SF.
I'll make games available periodically in bigger batches or just make a google drive directory - too much pain to upload each time I update.Guenther wrote: ↑Sat Feb 08, 2020 12:17 pm BTW are the games available?
(Usually, if the same entity plays against itself with time odds and the handicapped 'clone' loses some games, it might indicate
bad start positions. Moreover I am not sure time odds implementation in Arena is safe, I would use CuteChess-cli or WB here)
The Arena is doing ok so far with self-play but it seems impossible to use it for SF vs LC0 matches because the number of threads is global. And it's definitely buggy. I didn't see the time handicap possibility in CuteChess GUI (1.0) so I didn't consider it for this match yet.
-
- Posts: 343
- Joined: Sun Aug 25, 2019 8:33 am
- Full name: .
Re: Time handicap tournament. LC0 and SF11
Match: LC0 5s + 0.5s vs LC0 2.5s + 0.25s
Settings: Default settings + 90000000 cache, net 59098, 4 threads, fp16, roundrobin
Hardware: Titan V + 2080Ti
Book: Silver-Suite-Large random with reversed color
Tablebases: Complete 6-piece and around 40% of 7-piece
Software: Arena 3.5.1, Windows
Result after 1000 games:
lc0 59x 5s/0.5s 617.5/1000 249-14-73 +85
lc0 59x 2.5s/0.25s 382.5/100 14-249-737 -85
Ordo (baseline 2300):
# PLAYER : RATING ERROR POINTS PLAYED (%)
1 lc0 5s/0.5s : 2342.0 7.5 617.5 1000 62
2 lc0 2.5s/0.25s : 2258.0 7.5 382.5 1000 38
Difference : 84
Settings: Default settings + 90000000 cache, net 59098, 4 threads, fp16, roundrobin
Hardware: Titan V + 2080Ti
Book: Silver-Suite-Large random with reversed color
Tablebases: Complete 6-piece and around 40% of 7-piece
Software: Arena 3.5.1, Windows
Result after 1000 games:
lc0 59x 5s/0.5s 617.5/1000 249-14-73 +85
lc0 59x 2.5s/0.25s 382.5/100 14-249-737 -85
Ordo (baseline 2300):
# PLAYER : RATING ERROR POINTS PLAYED (%)
1 lc0 5s/0.5s : 2342.0 7.5 617.5 1000 62
2 lc0 2.5s/0.25s : 2258.0 7.5 382.5 1000 38
Difference : 84
Last edited by mmt on Tue Feb 11, 2020 2:49 am, edited 2 times in total.
-
- Posts: 343
- Joined: Sun Aug 25, 2019 8:33 am
- Full name: .
Re: Time handicap tournament. LC0 and SF11
Match: Stockfish 11 5s + 0.5s vs Stockfish 11 2.5s + 0.25s
Settings: 31 threads, 20 GB cache
Hardware: 2x Xenon Gold 6246
Book: Silver-Suite-Large random. With reversed colors
Tablebases: Complete 6-piece and a good part of 7-piece TBs
Software: Arena 3.5.1, Windows
Results after 1000 games:
Stockfish 11 5s + 0.5s: 544.0/1000 125-37-838 +28
Stockfish 11 2.5s + 0.25s: 456.0/1000 37-125-838 -28
Ordo (baseline 2300):
# PLAYER : RATING ERROR POINTS PLAYED (%)
1 Stockfish 11 5s + 0.5s : 2315.5 7.4 544.0 1000 54
2 Stockfish 11 2.5s + 0.25s : 2284.5 7.4 456.0 1000 46
Difference: 31
Settings: 31 threads, 20 GB cache
Hardware: 2x Xenon Gold 6246
Book: Silver-Suite-Large random. With reversed colors
Tablebases: Complete 6-piece and a good part of 7-piece TBs
Software: Arena 3.5.1, Windows
Results after 1000 games:
Stockfish 11 5s + 0.5s: 544.0/1000 125-37-838 +28
Stockfish 11 2.5s + 0.25s: 456.0/1000 37-125-838 -28
Ordo (baseline 2300):
# PLAYER : RATING ERROR POINTS PLAYED (%)
1 Stockfish 11 5s + 0.5s : 2315.5 7.4 544.0 1000 54
2 Stockfish 11 2.5s + 0.25s : 2284.5 7.4 456.0 1000 46
Difference: 31
-
- Posts: 550
- Joined: Tue Nov 19, 2019 8:48 pm
- Full name: Alayan Feh
Re: Time handicap tournament. LC0 and SF11
20GB hash for ultra-bullet ? That's unreasonable.
-
- Posts: 343
- Joined: Sun Aug 25, 2019 8:33 am
- Full name: .
Re: Time handicap tournament. LC0 and SF11
Hmm, I don't think a different value would make much of a difference, it's self-play with the same cache setting for both sides. But if less is better at these time controls then SF should figure it out itself and accordingly use less RAM.