New Giraffe (Aug 28)

[Graham Banks]

All is set up correctly.
I can create the engine and load it, but it just sits there, not registering in task manager and not showing any sign of doing anything.
If I doubleclick on the Giraffe 64-bit exe itself, that's when I get the error message.
I'll try it on my i5.

Works okay on my i5.

[matthewlai]

All is set up correctly.
I can create the engine and load it, but it just sits there, not registering in task manager and not showing any sign of doing anything.
If I doubleclick on the Giraffe 64-bit exe itself, that's when I get the error message.
I'll try it on my i5.

Works okay on my i5.

It looks like I had a bug in my build script, and it was indeed using popcnt in the non-popcnt build. I fixed it and re-uploaded the new version (same URL). The w64 version should work on all 64-bit CPUs now... for real this time!

Sorry about that and thanks for testing it!

[Laskos]

Download: https://bitbucket.org/waterreaction/gir ... 150828.zip

Windows 64-bit WB, Windows 32-bit WB, and 64-bit UCI included (thanks to Odd Gunnar Malin's Wb2Uci adapter). If you want 32-bit UCI, edit Wb2Uci.eng, and replace all "w64" by "w32".

This version is 320 Elo stronger than the last in self-play.

Most of the improvements are in eval. It is now both more accurate AND faster. The more I work Giraffe, the more I am convinced that chess is really all about eval. When I work on search, I get single digit gains if I am lucky. When I work on eval, I get double-digits all the time.

It now scores about 9750/15000 on the Strategic Test Suite at 1s per position. That's higher than almost all other engines, besides Stockfish (didn't check Komodo, because I am too cheap to buy a copy). When adjusted for the difference in NPS, I think it will have a fair chance of beating Stockfish.

I don't know if Stockfish was ever tuned based on STS results (if not, that's pretty impressive!), but Giraffe wasn't! It figured out all the knowledge by itself that happens to allow it to score very high on the STS.

This is also the first release where move ordering and node allocation are done using a second neural network. It reduces/extends (which are the same thing for a normalized probabilistic search) very aggressively, and doesn't follow a fixed LMR schedule. The neural net figures out node allocations for each move individually. When adjusted for slowdown, this resulted in a 40 Elo increase. However, it also made Giraffe slower. With the slowdown included, there's only a 20 Elo increase. This hasn't really been optimized yet, so it's just a first implementation. Just thought I would include it now since it does provide a net increase in strength.

My thesis is now also almost done, so the next release (or maybe the one after that) will be open source. After I submit my thesis I'll also have more time to do more non-machine-learning related stuff like opening book (just to save thinking time - Giraffe will generate its own opening book since it does play openings quite well already), SMP, and more search optimizations.

Thanks!

Code: Select all

STS 15 (100 positions)

Giraffe depth 1: 19/100
Fruit   depth 1: 20/100

Giraffe  1s    : 28/100
Fruit    1s    : 45/100


Giraffe - Fruit (depth 1 match)
41.0 - 59.0

Giraffe - Fruit (40/30'' match)
15.0 - 85.0

[matthewlai]

Code: Select all

STS 15 (100 positions)

Giraffe depth 1: 19/100
Fruit   depth 1: 20/100

Giraffe  1s    : 28/100
Fruit    1s    : 45/100

Giraffe - Fruit (depth 1 match)
41.0 - 59.0

Giraffe - Fruit (40/30'' match)
15.0 - 85.0

Cool thanks. By the way, Giraffe doesn't really have a concept of depth. The search is probabilistic, and I'm just using a simple formula to convert the set depth to a probabilistic threshold/node budget.

That said, "depth 1" in Giraffe does happen to coincide with normal depth 1 (1 full ply + QS). However, it won't mean anything for depths higher than 1. Giraffe also doesn't do any extensions at "depth 1".

Here are my 1s STS results on the full test suite - http://matthewlai.ca/tmp/sts_results.txt

[Laskos]

Code: Select all

STS 15 (100 positions)

Giraffe depth 1: 19/100
Fruit   depth 1: 20/100

Giraffe  1s    : 28/100
Fruit    1s    : 45/100

Giraffe - Fruit (depth 1 match)
41.0 - 59.0

Giraffe - Fruit (40/30'' match)
15.0 - 85.0

Cool thanks. By the way, Giraffe doesn't really have a concept of depth. The search is probabilistic, and I'm just using a simple formula to convert the set depth to a probabilistic threshold/node budget.

That said, "depth 1" in Giraffe does happen to coincide with normal depth 1 (1 full ply + QS). However, it won't mean anything for depths higher than 1. Giraffe also doesn't do any extensions at "depth 1".

Here are my 1s STS results on the full test suite - http://matthewlai.ca/tmp/sts_results.txt

I left it on standard opening position for more than two hours:

Code: Select all

Engine: Giraffe w64 (2048 MB)
by Matthew Lai
 14.00	 0:01 	+2.01 	1.e4 c5 2.c4 Nc6 3.Nc3 d6 4.Nf3 Nf6 5.d3 Rb8 6.a3 h6 (356.866) 228
 15.00	 0:02 	+1.21 	1.e4 c5 2.Nf3 d6 3.d3 Nf6 4.c4 Nc6 5.Nc3 (640.651) 224
 16.00	 0:06 	+0.56 	1.e4 c5 2.Nf3 d6 3.c4 Nc6 4.Nc3 g6 5.d3 Bg7 6.h3 h6 (1.338.918) 219
 17.00	 0:11 	+0.74 	1.e4 c5 2.Nf3 d6 3.c4 Nc6 4.Nc3 Nf6 5.d4 Nxd4 6.Nxd4 cxd4 7.Qxd4 e5 8.Qd3 a5 (2.534.789) 214
 18.00	 0:30 	+0.28 	1.c4 Nf6 2.Nc3 c5 3.e4 d6 4.d3 Nc6 5.Nf3 Rb8 6.a3 (6.532.897) 211
 19.00	 1:03 	+0.09 	1.e4 c5 2.c3 e6 3.d4 d5 4.e5 Qc7 5.Nf3 cxd4 6.cxd4 (13.033.073) 204
 20.00	 2:14 	+0.55 	1.e4 c5 2.c3 e6 3.d4 d5 4.e5 Nc6 5.Nf3 Nge7 6.a3 b6 7.h4 a6 (26.510.954) 197
 21.00	 5:38 	+0.08 	1.c4 c5 2.Nc3 d6 3.Nf3 Nf6 4.e4 e5 5.Be2 Nc6 6.d3 a5 7.a3 (66.010.334) 195
 22.00	11:30 	+0.50 	1.e4 e5 2.Nf3 d6 3.Nc3 c5 4.Bc4 Nc6 5.d3 h6 6.a3 Nf6 7.O-O (134.605.792) 194
 23.00	25:43 	+0.76 	1.e4 c5 2.c3 d5 3.exd5 Qxd5 4.Nf3 Nc6 5.Be2 g5 6.c4 Qf5 7.d3 h6 (295.354.772) 191
 24.00	60:09 	+0.13 	1.e4 c5 2.c3 d5 3.exd5 Qxd5 4.Nf3 Nc6 5.d4 Bg4 6.Be2 O-O-O 7.dxc5 Qxc5 8.Qa4 Nf6 9.O-O (685.285.848) 189

 25.00	146:36 	-0.06 	1.c4 c5 2.Nf3 Nf6 3.Nc3 Nc6 4.e4 e5 5.a3 d6 6.d3 a5 7.Ra2 h6 8.h3 Be6 (1.663.874.344) 189

It shows an English symmetrical four knights with 4.e4 and a slightly negative eval (I suppose this means less than 50% performance). My database shows slightly below 50% performance on this too.

[matthewlai]

It shows an English symmetrical four knights with 4.e4 and a slightly negative eval (I suppose this means less than 50% performance). My database shows slightly below 50% performance on this too.

Yeah +100.00 is the max positive non-mate score, and -100.00 is the max negative non-mate score.

I find that as training progresses, it usually starts by preferring 1. d4, then eventually switches to 1. e4, then 1. c4 (not counting very beginning where it just plays randomly).

With a7 pawn missing it's about +7.00.
b7 => +12.00
c7 => +13.00
d7 => +13.00
e7 => +13.00
f7 => +20.00
g7 => +18.00
h7 => +11.50

Not sure why it thinks f7 and g7 pawns are so important. Probably because of easier king-side attacks.

[chetday]

Matthew, thank you for your terrific chess engine. I'm having a wonderful time playing games against it. I haven't actually beaten Giraffe since the first version when I lucked out with a few wins, but I continue to enjoy playing against each update.

For what it's worth, I just finished running a 25 game Blitz tournament between Giraffe and MadChess 1.4 (also a favorite engine that I enjoy playing). This latest version of Giraffe won 74% of the games, which jives with your 2300 ELO that you estimated for Graham earlier in this thread. I have MadChess 1.4 ELO rated as 2234.

[matthewlai]

Matthew, thank you for your terrific chess engine. I'm having a wonderful time playing games against it. I haven't actually beaten Giraffe since the first version when I lucked out with a few wins, but I continue to enjoy playing against each update.

For what it's worth, I just finished running a 25 game Blitz tournament between Giraffe and MadChess 1.4 (also a favorite engine that I enjoy playing). This latest version of Giraffe won 74% of the games, which jives with your 2300 ELO that you estimated for Graham earlier in this thread. I have MadChess 1.4 ELO rated as 2234.

Thanks for your kind words! It means a lot to me. Computer vs computer chess is fun, but I still prefer watching computer vs human games (I do that a lot with Giraffe playing on ICC).

[Graham Banks]

How many cores can Giraffe use?

[matthewlai]

How many cores can Giraffe use?

Just one at the moment.