New Giraffe (Aug 28)

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

matthewlai
Posts: 793
Joined: Sun Aug 03, 2014 4:48 am
Location: London, UK

Re: New Giraffe (Aug 28)

Post by matthewlai »

Michel wrote:
matthewlai wrote:
Michel wrote:It might be interesting to apply this procedure http://rybkaforum.net/cgi-bin/rybkaforu ... ?tid=30107 to get an idea of the quality of Giraffe's eval, independently of search.
I did look into this actually. It would be a bit more difficult than Crafty and Stockfish, because of the probabilistic eval. All the score margins in search (futility, razoring, etc) would have to be changed.
I am sure you considered this but what would be the issue with using the simple conversion formula

winning probability--->pawn advantage

suggested here

https://chessprogramming.wikispaces.com ... e,+and+ELO ?
It would work for the middle linear region, but I don't know how well that will work for more extreme scores.

It may be possible to use the input instead of output of the final activation function. Then it should be more or less linear.
Disclosure: I work for DeepMind on the AlphaZero project, but everything I say here is personal opinion and does not reflect the views of DeepMind / Alphabet.
Werewolf
Posts: 2044
Joined: Thu Sep 18, 2008 10:24 pm

Re: New Giraffe (Aug 28)

Post by Werewolf »

Matthew,

Do you have data on the elo gain per doubling of hardware speed with Giraffe?

Normally it is somewhere around 65, but I don't know how neural nets work, so I'd be interested to know.

Presumably giving Giraffe more time only affects the search (as with a normal engine) - the eval is already fixed. Is that right?
matthewlai
Posts: 793
Joined: Sun Aug 03, 2014 4:48 am
Location: London, UK

Re: New Giraffe (Aug 28)

Post by matthewlai »

Werewolf wrote:Matthew,

Do you have data on the elo gain per doubling of hardware speed with Giraffe?

Normally it is somewhere around 65, but I don't know how neural nets work, so I'd be interested to know.

Presumably giving Giraffe more time only affects the search (as with a normal engine) - the eval is already fixed. Is that right?
The eval is already fixed, so during play, it should scale similarly to other engines.

I don't have the numbers, though. Sorry.
Disclosure: I work for DeepMind on the AlphaZero project, but everything I say here is personal opinion and does not reflect the views of DeepMind / Alphabet.
User avatar
Marek Soszynski
Posts: 587
Joined: Wed May 10, 2006 7:28 pm
Location: Birmingham, England

Re: New Giraffe (Aug 28)

Post by Marek Soszynski »

I'm running Giraffe 7119bddd8913 as a Winboard engine (through Wine) in Scid. It works in infinite analysis but stops at depth 29. So I make a move (such as the one it suggests) and it starts analysing from there but stops at depth 29 again. And so on, consistently.
Marek Soszynski
matthewlai
Posts: 793
Joined: Sun Aug 03, 2014 4:48 am
Location: London, UK

Re: New Giraffe (Aug 28)

Post by matthewlai »

Marek Soszynski wrote:I'm running Giraffe 7119bddd8913 as a Winboard engine (through Wine) in Scid. It works in infinite analysis but stops at depth 29. So I make a move (such as the one it suggests) and it starts analysing from there but stops at depth 29 again. And so on, consistently.
"Depth 29" is the current hard limit for node budget due to concerns with overflow. What kind of position are you looking at? I have never seen it actually getting to "depth 29" in a non-mate position.

Also, there will be a source release soon, so you'll be able to compile a native version shortly. I do almost all my testing on Linux and OSX, and have never encountered that.
Disclosure: I work for DeepMind on the AlphaZero project, but everything I say here is personal opinion and does not reflect the views of DeepMind / Alphabet.
User avatar
Marek Soszynski
Posts: 587
Joined: Wed May 10, 2006 7:28 pm
Location: Birmingham, England

Re: New Giraffe (Aug 28)

Post by Marek Soszynski »

matthewlai wrote:
Marek Soszynski wrote:I'm running Giraffe 7119bddd8913 as a Winboard engine (through Wine) in Scid. It works in infinite analysis but stops at depth 29. So I make a move (such as the one it suggests) and it starts analysing from there but stops at depth 29 again. And so on, consistently.
"Depth 29" is the current hard limit for node budget due to concerns with overflow. What kind of position are you looking at? I have never seen it actually getting to "depth 29" in a non-mate position.

Also, there will be a source release soon, so you'll be able to compile a native version shortly. I do almost all my testing on Linux and OSX, and have never encountered that.
Here's one of the positions I was studying:
[d]3r4/1pr1q1k1/p2n4/P2Pppb1/N6p/1B1R3P/5PP1/3RQ1K1 w - - 0 37

I'm certainly looking forward to a native compile of your most interesting engine.
Marek Soszynski
matthewlai
Posts: 793
Joined: Sun Aug 03, 2014 4:48 am
Location: London, UK

Re: New Giraffe (Aug 28)

Post by matthewlai »

Marek Soszynski wrote:
matthewlai wrote:
Marek Soszynski wrote:I'm running Giraffe 7119bddd8913 as a Winboard engine (through Wine) in Scid. It works in infinite analysis but stops at depth 29. So I make a move (such as the one it suggests) and it starts analysing from there but stops at depth 29 again. And so on, consistently.
"Depth 29" is the current hard limit for node budget due to concerns with overflow. What kind of position are you looking at? I have never seen it actually getting to "depth 29" in a non-mate position.

Also, there will be a source release soon, so you'll be able to compile a native version shortly. I do almost all my testing on Linux and OSX, and have never encountered that.
Here's one of the positions I was studying:
[d]3r4/1pr1q1k1/p2n4/P2Pppb1/N6p/1B1R3P/5PP1/3RQ1K1 w - - 0 37

I'm certainly looking forward to a native compile of your most interesting engine.
Yeah that position works fine for me (on OSX). It may be something to do with Wine or Scid. Do you have an engine communication log by any chance? And can you try searching the position from command line?

Code: Select all

post
setboard 3r4/1pr1q1k1/p2n4/P2Pppb1/N6p/1B1R3P/5PP1/3RQ1K1 w - - 0 37
analyze
Disclosure: I work for DeepMind on the AlphaZero project, but everything I say here is personal opinion and does not reflect the views of DeepMind / Alphabet.
User avatar
Marek Soszynski
Posts: 587
Joined: Wed May 10, 2006 7:28 pm
Location: Birmingham, England

Re: New Giraffe (Aug 28)

Post by Marek Soszynski »

matthewlai wrote: Yeah that position works fine for me (on OSX). It may be something to do with Wine or Scid. Do you have an engine communication log by any chance? And can you try searching the position from command line?

Code: Select all

post
setboard 3r4/1pr1q1k1/p2n4/P2Pppb1/N6p/1B1R3P/5PP1/3RQ1K1 w - - 0 37
analyze
From the command line, even overnight, my PC reached only:

Code: Select all

# d: 26 node budget: 1125899906842624 NPS: 53627.4                                         27 -674 2606543 1394090695 d3c3 g7g6 c3c7 e7c7 e1e5 d8e8 e5c3 c7c3 a4c3 g5f6 d1c1 e8c8 c3e2 c8c1 e2c1 f6c3 g2g3 h4g3 f2g3 c3a5                                                        
# d: 27 node budget: 4503599627370496 NPS: 53484.3
At which point I stopped the experiment.

Meanwhile the behaviour in Scid, with any position, is the same. Giraffe stops analysing (as confirmed by Task Manager) when Scid shows depth 29 is reached. This could take 25 seconds or 125, to give you an idea.
Marek Soszynski
matthewlai
Posts: 793
Joined: Sun Aug 03, 2014 4:48 am
Location: London, UK

Re: New Giraffe (Aug 28)

Post by matthewlai »

Marek Soszynski wrote:
matthewlai wrote: Yeah that position works fine for me (on OSX). It may be something to do with Wine or Scid. Do you have an engine communication log by any chance? And can you try searching the position from command line?

Code: Select all

post
setboard 3r4/1pr1q1k1/p2n4/P2Pppb1/N6p/1B1R3P/5PP1/3RQ1K1 w - - 0 37
analyze
From the command line, even overnight, my PC reached only:

Code: Select all

# d: 26 node budget: 1125899906842624 NPS: 53627.4                                         27 -674 2606543 1394090695 d3c3 g7g6 c3c7 e7c7 e1e5 d8e8 e5c3 c7c3 a4c3 g5f6 d1c1 e8c8 c3e2 c8c1 e2c1 f6c3 g2g3 h4g3 f2g3 c3a5                                                        
# d: 27 node budget: 4503599627370496 NPS: 53484.3
At which point I stopped the experiment.

Meanwhile the behaviour in Scid, with any position, is the same. Giraffe stops analysing (as confirmed by Task Manager) when Scid shows depth 29 is reached. This could take 25 seconds or 125, to give you an idea.
I have absolutely no idea how that can happen!

Is there a way to make Scid produce an engine communication log?
Disclosure: I work for DeepMind on the AlphaZero project, but everything I say here is personal opinion and does not reflect the views of DeepMind / Alphabet.
User avatar
Marek Soszynski
Posts: 587
Joined: Wed May 10, 2006 7:28 pm
Location: Birmingham, England

Re: New Giraffe (Aug 28)

Post by Marek Soszynski »

matthewlai wrote:Is there a way to make Scid produce an engine communication log?
This is for the position after 1.e4 e5 2.Nc3 Nc6 3.Bc4 Na5 *.

Code: Select all

Scid-Engine communication log file
Engine: Giraffe
Command: /home/marek/Chess/engines/Giraffe/giraffe_20150828/giraffe_w64.exe
Date: Fri Sep 04 13:15:07 BST 2015

This file was automatically generated by Scid.
It is rewritten every time an engine is started in Scid.

Engine: # Using 4 OpenMP thread(s)
NOTE  : First line from engine seen; sending it initial commands now.
Scid  : xboard
Scid  : protover 2
Scid  : ponder off
Scid  : post
Engine: # Version: 7119bddd8913
Engine: # Running in release mode
Engine: # Using static evaluator
Engine: # Using static move evaluator
Engine: feature ping=1 setboard=1 playother=0 san=0 usermove=1 time=1 draw=0 sigint=0 sigterm=0 reuse=1 analyze=1 myname="Giraffe 7119bddd8913" variants=normal colors=0 ics=0 name=0 pause=0 nps=0 debug=1 memory=0 smp=0 done=1
Engine: # GTBPath not set
Engine: Error (unknown command): ponder
Scid  : setboard r1bqkbnr/pppp1ppp/8/n3p3/2B1P3/2N5/PPPP1PPP/R1BQK1NR w KQkq - 4 4
Scid  : analyze
Engine: 1 0 0 59 c4f1 
Engine: # d: 1 node budget: 1 NPS: 313643
Engine: 2 0 0 97 c4f1 
Engine: # d: 2 node budget: 4 NPS: 278663
Engine: 3 0 0 165 c4f1 a7a6 
Engine: # d: 3 node budget: 16 NPS: 350233
Engine: 4 0 0 228 c4f1 a7a6 
Engine: # d: 4 node budget: 64 NPS: 397796
Engine: 5 0 0 338 c4f1 a7a6 a2a3 
Engine: # d: 5 node budget: 256 NPS: 412115
Engine: 6 0 0 550 c4f1 a7a6 a2a3 
Engine: # d: 6 node budget: 1024 NPS: 496314
Engine: 7 0 0 674 c4f1 a7a6 a2a3 b7b5 
Engine: # d: 7 node budget: 4096 NPS: 502928
Engine: 8 0 0 1075 c4f1 a7a6 a2a3 b7b5 b2b3 
Engine: # d: 8 node budget: 16384 NPS: 553848
Engine: 9 0 0 1525 c4f1 a7a6 a2a3 b7b5 b2b3 
Engine: # d: 9 node budget: 65536 NPS: 594785
Engine: 10 0 0 2624 c4f1 a7a6 a2a3 b7b5 b2b3 b5b4 
Engine: # d: 10 node budget: 262144 NPS: 672976
Engine: 11 0 0 3360 c4f1 a7a6 a2a3 b7b5 b2b3 b5b4 a3b4 f8b4 
Engine: # d: 11 node budget: 1048576 NPS: 640730
Engine: 12 0 0 5580 c4f1 a7a6 a2a3 b7b5 b2b3 b5b4 a3b4 f8b4 
Engine: # d: 12 node budget: 4194304 NPS: 740546
Engine: 13 0 1 8444 c4f1 a7a6 a2a3 b7b5 b2b3 b5b4 a3b4 f8b4 
Engine: # d: 13 node budget: 16777216 NPS: 698141
Engine: 14 0 2 16484 c4f1 a7a6 a2a3 b7b5 b2b3 b5b4 a3b4 f8b4 f2f3 
Engine: # d: 14 node budget: 67108864 NPS: 690050
Engine: 15 0 3 24651 c4f1 a7a6 a2a3 b7b5 b2b3 b5b4 a3b4 f8b4 f2f3 
Engine: # d: 15 node budget: 268435456 NPS: 638493
Engine: 16 0 5 39210 c4f1 a7a6 a2a3 b7b5 b2b3 b5b4 a3b4 f8b4 f2f3 c7c5 
Engine: # d: 16 node budget: 1073741824 NPS: 663116
Engine: 17 0 9 61877 c4f1 a7a6 a2a3 b7b5 b2b3 g8f6 g2g3 d7d6 d2d3 
Engine: # d: 17 node budget: 4294967296 NPS: 670680
Engine: 18 0 14 95008 c4f1 a7a6 a2a3 b7b5 b2b3 g8f6 g2g3 d7d6 d2d3 d6d5 e4d5 f6d5 
Engine: # d: 18 node budget: 17179869184 NPS: 657244
Engine: 19 0 23 148766 c4f1 a7a6 a2a3 b7b5 b2b3 g8f6 g2g3 d7d6 d2d3 d6d5 e4d5 f6d5 
Engine: # d: 19 node budget: 68719476736 NPS: 637097
Engine: 20 0 40 245989 c4f1 a7a6 a2a3 b7b5 b2b3 g8f6 g2g3 d7d6 d2d3 d6d5 e4d5 f6d5 
Engine: # d: 20 node budget: 274877906944 NPS: 613930
Engine: 21 0 63 389753 c4f1 a7a6 a2a3 b7b5 b2b3 g8f6 g2g3 d7d6 d2d3 d6d5 e4d5 f6d5 c3d5 d8d5 
Engine: # d: 21 node budget: 1099511627776 NPS: 614522
Engine: 22 0 107 665247 c4f1 a7a6 a2a3 b7b5 b2b3 g8f6 g2g3 d7d6 d2d3 d6d5 e4d5 f6d5 c3d5 d8d5 
Engine: # d: 22 node budget: 4398046511104 NPS: 617239
Engine: 23 0 165 1005892 c4f1 a7a6 a2a3 b7b5 b2b3 g8f6 g2g3 d7d6 d2d3 d6d5 e4d5 f6d5 c3d5 d8d5 
Engine: # d: 23 node budget: 17592186044416 NPS: 606536
Engine: 24 0 240 1483596 c4f1 a7a6 a2a3 b7b5 b2b3 g8f6 g2g3 c7c5 c3d5 f6d5 e4d5 b5b4 a3b4 c5b4 
Engine: # d: 24 node budget: 70368744177664 NPS: 616674
Engine: 25 0 357 2270577 c4f1 a7a6 a2a3 b7b5 b2b3 g8f6 g2g3 c7c5 c3d5 f6d5 e4d5 d7d6 g1e2 g7g6 c2c3 
Engine: # d: 25 node budget: 281474976710656 NPS: 635400
Engine: 26 0 549 3470353 c4f1 a7a6 a2a3 b7b5 b2b3 g8f6 g2g3 c7c5 c3d5 f6d5 e4d5 f7f5 d1f3 g7g6 d2d3 f8e7 
Engine: # d: 26 node budget: 1125899906842624 NPS: 631674
Engine: 27 0 883 5493176 c4f1 a7a6 a2a3 b7b5 b2b3 g8f6 g2g3 c7c5 c3d5 f6d5 e4d5 d7d6 g1f3 g7g6 c2c3 c5c4 b3c4 b5c4 
Engine: # d: 27 node budget: 4503599627370496 NPS: 621527
Engine: 28 0 1446 8949928 c4f1 a7a6 a2a3 b7b5 d1h5 d7d6 b2b3 a5c6 h5f3 g7g6 f3d3 f8e7 g1e2 c8d7 e1d1 e7h4 f2f3 
Engine: # d: 28 node budget: 18014398509481984 NPS: 618748
Engine: 29 0 2542 15680388 c4f1 a7a6 a2a3 d8f6 d1h5 a5c6 c3d5 f6d6 g1f3 g7g6 h5h3 g8e7 d5e7 c6e7 d2d3 f7f6 e1d1 
Engine: # d: 29 node budget: 72057594037927936 NPS: 616788
There is no more. The engine stops - but remains in memory ready for a new move.
Marek Soszynski