DQ 1.2 vs Bad Gyal 7 - the power of policy

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
dkappe
Posts: 263
Joined: Tue Aug 21, 2018 5:52 pm
Full name: Dietrich Kappe

DQ 1.2 vs Bad Gyal 7 - the power of policy

Post by dkappe » Thu Aug 29, 2019 4:35 pm

Two nets that, as far as I know, use similar source material.

DarkQueen uses lichess games that have been annotated by Stockfish. Bad Gyal 7 uses lichess games annotated with q and policy from Stockfish. The main difference seems to be policy. So far it's a walkover by the smaller Bad Gyal 7. Here a sample game:


MikeB
Posts: 3467
Joined: Thu Mar 09, 2006 5:34 am
Location: Pen Argyl, Pennsylvania

Re: DQ 1.2 vs Bad Gyal 7 - the power of policy

Post by MikeB » Thu Aug 29, 2019 4:56 pm

dkappe wrote:
Thu Aug 29, 2019 4:35 pm
Two nets that, as far as I know, use similar source material.

DarkQueen uses lichess games that have been annotated by Stockfish. Bad Gyal 7 uses lichess games annotated with q and policy from Stockfish. The main difference seems to be policy. So far it's a walkover by the smaller Bad Gyal 7. Here a sample game:

Bad Gyal 7 is surprising strong, it plays at World Champion strength or better on my Mac , close to 2900 - could be stronger.

brianr
Posts: 356
Joined: Thu Mar 09, 2006 2:01 pm

Re: DQ 1.2 vs Bad Gyal 7 - the power of policy

Post by brianr » Thu Aug 29, 2019 5:28 pm

FWIW, it is quite a lot stronger than that, I think, at least on the hardware below.

I trained a large 320x24 net and played against Stockfish 7 on one CPU (i5 3570K, 3.4GHz, boost to 3.8)
My net was clearly stronger on an RTX 2060 (I know, not a favorable "Leela Ratio" for SF).

Code: Select all

Score of SF7 vs T60-320x24-74000-Steps: 6 - 21 - 13  [0.313] 40
Elo difference: -136.97 +/- 94.70, LOS: 0.19 %, DrawRatio: 32.5 %
Then, I played my net against Bad Gyal 7:

Code: Select all

Score of Bad-Gyal-7-DKappe vs T60-320x24-74000-Steps: 122 - 94 - 284  [0.528] 500
Elo difference: 19.48 +/- 20.00, LOS: 97.16 %, DrawRatio: 56.8 %
On CCRL at 40/40 SF7 1 CPU is 3,245; again, my hardware is quite different.

Although larger nets should be able to learn more, they also take a lot more training time I think.
Thus the much faster speed of Bad Gyal 7 seems to more than make up for its smaller size.

To be sure, one could match SF7 vs Bad Gyal 7 directly.

Conditions:
6 piece Syzygy tablebases
2 move opening book
Time control 0:10+2.0

dkappe
Posts: 263
Joined: Tue Aug 21, 2018 5:52 pm
Full name: Dietrich Kappe

Re: DQ 1.2 vs Bad Gyal 7 - the power of policy

Post by dkappe » Thu Aug 29, 2019 6:12 pm

If I want to win a correspondence game, I’ll reach for the strongest T40 net. When I want to prepare for OTB against human opposition, I reach for Bad Gyal. It suggests moves and lines that are not always objectively best, but in practice best against human opponents. Plus they tend to be plans I can understand and follow, rather than the baroque maneuvers of a “zero” net.

This may be one of those cases where the objectively best net is not the best tool for the job.

Modern Times
Posts: 2410
Joined: Thu Jun 07, 2012 9:02 pm

Re: DQ 1.2 vs Bad Gyal 7 - the power of policy

Post by Modern Times » Thu Aug 29, 2019 6:36 pm

Where do you download Bad Gyal 7 ?
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

dkappe
Posts: 263
Joined: Tue Aug 21, 2018 5:52 pm
Full name: Dietrich Kappe

Re: DQ 1.2 vs Bad Gyal 7 - the power of policy

Post by dkappe » Thu Aug 29, 2019 6:44 pm

Modern Times wrote:
Thu Aug 29, 2019 6:36 pm
Where do you download Bad Gyal 7 ?
https://github.com/dkappe/leela-chess-w ... i/Bad-Gyal

Modern Times
Posts: 2410
Joined: Thu Jun 07, 2012 9:02 pm

Re: DQ 1.2 vs Bad Gyal 7 - the power of policy

Post by Modern Times » Thu Aug 29, 2019 7:51 pm

Thanks, I'll have a play with it.
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

MikeB
Posts: 3467
Joined: Thu Mar 09, 2006 5:34 am
Location: Pen Argyl, Pennsylvania

Re: DQ 1.2 vs Bad Gyal 7 - the power of policy

Post by MikeB » Fri Aug 30, 2019 3:58 am

brianr wrote:
Thu Aug 29, 2019 5:28 pm
FWIW, it is quite a lot stronger than that, I think, at least on the hardware below.

I trained a large 320x24 net and played against Stockfish 7 on one CPU (i5 3570K, 3.4GHz, boost to 3.8)
My net was clearly stronger on an RTX 2060 (I know, not a favorable "Leela Ratio" for SF).

Code: Select all

Score of SF7 vs T60-320x24-74000-Steps: 6 - 21 - 13  [0.313] 40
Elo difference: -136.97 +/- 94.70, LOS: 0.19 %, DrawRatio: 32.5 %
Then, I played my net against Bad Gyal 7:

Code: Select all

Score of Bad-Gyal-7-DKappe vs T60-320x24-74000-Steps: 122 - 94 - 284  [0.528] 500
Elo difference: 19.48 +/- 20.00, LOS: 97.16 %, DrawRatio: 56.8 %
On CCRL at 40/40 SF7 1 CPU is 3,245; again, my hardware is quite different.

Although larger nets should be able to learn more, they also take a lot more training time I think.
Thus the much faster speed of Bad Gyal 7 seems to more than make up for its smaller size.

To be sure, one could match SF7 vs Bad Gyal 7 directly.

Conditions:
6 piece Syzygy tablebases
2 move opening book
Time control 0:10+2.0
IMy hardware is limited, I do use OpenCl and get about 4-5K nps or more with BG, rating could be higher, I didn't test it for rating - but for my setup , it is hands down the best net for me. The bigger nets are really really slow, less than 1K nps on my weak setup. And I love the way Bad Gyal plays, very enetertaiing style of play = far more interesting on a consistent basis. It feels like it thinks "outside the box". Great work by dkappe!

Post Reply