The result on 100 positions in an hour or so. At short time control, as it seems to not matter.jkiliani wrote: ↑Tue Oct 02, 2018 10:12 amHi, thanks a lot for testing! The dev team found another bug in the tablebase implementation, where only repeats from the current position but not from all positions since the last zeroing move were considered. There is now a fix for this: https://github.com/LeelaChessZero/lc0/pull/412, with compiled binary https://ci.appveyor.com/api/buildjobs/5 ... %2Flc0.exe. If you could run your testing suite with the new build again this would be greatLaskos wrote: ↑Tue Oct 02, 2018 8:41 amSill misses by 3-fold, seems at lower rate, only 3 misses out of 100 positions. Time control for Lc0 is 0.5s/move, but as I already wrote, it seems to not matter, and even on easy 6-men wins it misses at pretty much the same rate. Here is the PGN of misses:crem wrote: ↑Tue Oct 02, 2018 6:48 amHi Laskos,Laskos wrote: ↑Mon Oct 01, 2018 9:16 pmNo, still misses wins as 3-fold rep. at steady rate.can00336 wrote: ↑Mon Oct 01, 2018 8:47 pmThanks for finding a bug in the implementation. A proposed fix is available here: https://github.com/LeelaChessZero/lc0/pull/408
You can download a CUDA binary to try the fix here: https://ci.appveyor.com/api/buildjobs/g ... %2Flc0.exe
We have a fix for that https://github.com/LeelaChessZero/lc0/pull/410, which we have no way to check currently. Would it be possible for you to test it? The deadline for CCCC binary submission is today, and we'll have to release v0.18.1 today with a fix.
As a last resort we can push this patch blindly, but it would be nice to have it tested.
The CUDA build of Lc0 with this patch is https://ci.appveyor.com/api/buildjobs/3 ... %2Flc0.exe
Thanks!
Thanks again!
Lc0 v18 rc2 for 2080 Syzygy implementation
Moderators: hgm, Dann Corbit, Harvey Williamson
Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Re: Lc0 v18 rc2 for 2080 Syzygy implementation
Re: Lc0 v18 rc2 for 2080 Syzygy implementation
No misses this time on 100 hard 6-men wins.Laskos wrote: ↑Tue Oct 02, 2018 10:16 amThe result on 100 positions in an hour or so. At short time control, as it seems to not matter.jkiliani wrote: ↑Tue Oct 02, 2018 10:12 amHi, thanks a lot for testing! The dev team found another bug in the tablebase implementation, where only repeats from the current position but not from all positions since the last zeroing move were considered. There is now a fix for this: https://github.com/LeelaChessZero/lc0/pull/412, with compiled binary https://ci.appveyor.com/api/buildjobs/5 ... %2Flc0.exe. If you could run your testing suite with the new build again this would be greatLaskos wrote: ↑Tue Oct 02, 2018 8:41 amSill misses by 3-fold, seems at lower rate, only 3 misses out of 100 positions. Time control for Lc0 is 0.5s/move, but as I already wrote, it seems to not matter, and even on easy 6-men wins it misses at pretty much the same rate. Here is the PGN of misses:crem wrote: ↑Tue Oct 02, 2018 6:48 amHi Laskos,Laskos wrote: ↑Mon Oct 01, 2018 9:16 pmNo, still misses wins as 3-fold rep. at steady rate.can00336 wrote: ↑Mon Oct 01, 2018 8:47 pmThanks for finding a bug in the implementation. A proposed fix is available here: https://github.com/LeelaChessZero/lc0/pull/408
You can download a CUDA binary to try the fix here: https://ci.appveyor.com/api/buildjobs/g ... %2Flc0.exe
We have a fix for that https://github.com/LeelaChessZero/lc0/pull/410, which we have no way to check currently. Would it be possible for you to test it? The deadline for CCCC binary submission is today, and we'll have to release v0.18.1 today with a fix.
As a last resort we can push this patch blindly, but it would be nice to have it tested.
The CUDA build of Lc0 with this patch is https://ci.appveyor.com/api/buildjobs/3 ... %2Flc0.exe
Thanks!
Maybe I will check on say 200 other 6-men won positions, but it seems fine now.
Re: Lc0 v18 rc2 for 2080 Syzygy implementation
Great, thank you for the whole Leela Dev team! Do you have any testing suites about holding difficult draws, or avoiding losses on TB lost positions against opponents without TB?
Re: Lc0 v18 rc2 for 2080 Syzygy implementation
I tested yesterday on 100 long 6-men draws (not sure if they are exactly hard, but many of them are), no losses. Avoiding TB losses against non-TB engine needs comparison with another Syzygy-enabled engine against that non-TB engine. Might compare Lc0_Syzygy against SF8_no_TB compared to SF8_Syzygy against SF8_no_TB on 6-men hard wins. Will depend on time control used too.
-
Javier Ros
- Posts: 185
- Joined: Fri Oct 12, 2012 10:48 am
- Location: Seville (SPAIN)
- Full name: Javier Ros
Re: Lc0 v18 rc2 for 2080 Syzygy implementation
Very interesting experiment and great work!
The love relationship between a chess engine tester and his computer can be summarized in one sentence:
Until heat do us part.
Until heat do us part.
Re: Lc0 v18 rc2 for 2080 Syzygy implementation
Out of 50 6-men TB losses, Lc0_Syzygy (the fixed one) saves 13 against SF8_no_TB, SF8_Syzygy saves 15 against SF8_no_TB. So, behaves similarly to SF8, which has a good TB implementation (and good endgame eval, which Lc0 doesn't). Might check how many saves SF dev.Laskos wrote: ↑Tue Oct 02, 2018 11:22 amI tested yesterday on 100 long 6-men draws (not sure if they are exactly hard, but many of them are), no losses. Avoiding TB losses against non-TB engine needs comparison with another Syzygy-enabled engine against that non-TB engine. Might compare Lc0_Syzygy against SF8_no_TB compared to SF8_Syzygy against SF8_no_TB on 6-men hard wins. Will depend on time control used too.
Re: Lc0 v18 rc2 for 2080 Syzygy implementation
How about fixing a bug where engine will suddenly resign during the game out of nowhere? This happens during draw or won game, and especially in long games...jkiliani wrote: ↑Tue Oct 02, 2018 10:12 amHi, thanks a lot for testing! The dev team found another bug in the tablebase implementation, where only repeats from the current position but not from all positions since the last zeroing move were considered. There is now a fix for this: https://github.com/LeelaChessZero/lc0/pull/412, with compiled binary https://ci.appveyor.com/api/buildjobs/5 ... %2Flc0.exe. If you could run your testing suite with the new build again this would be greatLaskos wrote: ↑Tue Oct 02, 2018 8:41 amSill misses by 3-fold, seems at lower rate, only 3 misses out of 100 positions. Time control for Lc0 is 0.5s/move, but as I already wrote, it seems to not matter, and even on easy 6-men wins it misses at pretty much the same rate. Here is the PGN of misses:crem wrote: ↑Tue Oct 02, 2018 6:48 amHi Laskos,Laskos wrote: ↑Mon Oct 01, 2018 9:16 pmNo, still misses wins as 3-fold rep. at steady rate.can00336 wrote: ↑Mon Oct 01, 2018 8:47 pmThanks for finding a bug in the implementation. A proposed fix is available here: https://github.com/LeelaChessZero/lc0/pull/408
You can download a CUDA binary to try the fix here: https://ci.appveyor.com/api/buildjobs/g ... %2Flc0.exe
We have a fix for that https://github.com/LeelaChessZero/lc0/pull/410, which we have no way to check currently. Would it be possible for you to test it? The deadline for CCCC binary submission is today, and we'll have to release v0.18.1 today with a fix.
As a last resort we can push this patch blindly, but it would be nice to have it tested.
The CUDA build of Lc0 with this patch is https://ci.appveyor.com/api/buildjobs/3 ... %2Flc0.exe
Thanks!
Re: Lc0 v18 rc2 for 2080 Syzygy implementation
To fix bugs, we need to first reproduce them. Can you give a .pgn file where this happened for you, as well as specifics about the binary you used and your command line parameters?
Re: Lc0 v18 rc2 for 2080 Syzygy implementation
SF_dev_Syzygy saves 23 out of 50 TB losses. Might be due to different Syzygy implementation, stronger play, etc., I don't know.Laskos wrote: ↑Tue Oct 02, 2018 12:33 pmOut of 50 6-men TB losses, Lc0_Syzygy (the fixed one) saves 13 against SF8_no_TB, SF8_Syzygy saves 15 against SF8_no_TB. So, behaves similarly to SF8, which has a good TB implementation (and good endgame eval, which Lc0 doesn't). Might check how many saves SF dev.Laskos wrote: ↑Tue Oct 02, 2018 11:22 amI tested yesterday on 100 long 6-men draws (not sure if they are exactly hard, but many of them are), no losses. Avoiding TB losses against non-TB engine needs comparison with another Syzygy-enabled engine against that non-TB engine. Might compare Lc0_Syzygy against SF8_no_TB compared to SF8_Syzygy against SF8_no_TB on 6-men hard wins. Will depend on time control used too.
-
nabildanial
- Posts: 125
- Joined: Thu Jun 05, 2014 3:29 am
- Location: Malaysia
Re: Lc0 v18 rc2 for 2080 Syzygy implementation
How does a UCI engine resign in games?Damir wrote: ↑Tue Oct 02, 2018 1:03 pmHow about fixing a bug where engine will suddenly resign during the game out of nowhere? This happens during draw or won game, and especially in long games...jkiliani wrote: ↑Tue Oct 02, 2018 10:12 amHi, thanks a lot for testing! The dev team found another bug in the tablebase implementation, where only repeats from the current position but not from all positions since the last zeroing move were considered. There is now a fix for this: https://github.com/LeelaChessZero/lc0/pull/412, with compiled binary https://ci.appveyor.com/api/buildjobs/5 ... %2Flc0.exe. If you could run your testing suite with the new build again this would be greatLaskos wrote: ↑Tue Oct 02, 2018 8:41 amSill misses by 3-fold, seems at lower rate, only 3 misses out of 100 positions. Time control for Lc0 is 0.5s/move, but as I already wrote, it seems to not matter, and even on easy 6-men wins it misses at pretty much the same rate. Here is the PGN of misses:crem wrote: ↑Tue Oct 02, 2018 6:48 amHi Laskos,Laskos wrote: ↑Mon Oct 01, 2018 9:16 pmNo, still misses wins as 3-fold rep. at steady rate.can00336 wrote: ↑Mon Oct 01, 2018 8:47 pmThanks for finding a bug in the implementation. A proposed fix is available here: https://github.com/LeelaChessZero/lc0/pull/408
You can download a CUDA binary to try the fix here: https://ci.appveyor.com/api/buildjobs/g ... %2Flc0.exe
We have a fix for that https://github.com/LeelaChessZero/lc0/pull/410, which we have no way to check currently. Would it be possible for you to test it? The deadline for CCCC binary submission is today, and we'll have to release v0.18.1 today with a fix.
As a last resort we can push this patch blindly, but it would be nice to have it tested.
The CUDA build of Lc0 with this patch is https://ci.appveyor.com/api/buildjobs/3 ... %2Flc0.exe
Thanks!