Scorpio 2.8.9 with TensorRT support

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

Daniel Shawul
Posts: 4185
Joined: Tue Mar 14, 2006 11:34 am
Location: Ethiopia

Scorpio 2.8.9 with TensorRT support

Post by Daniel Shawul »

Hi,

Scorpio 2.8.0 MCTS+NN with TensorFlow and TensorRT backend is available here

https://github.com/dshawul/Scorpio/releases

Changes:

a) Implemented TensorRT backend with the option of FLOAT/HALF/INT8 precision calculations.
There are now two formats for the network UFF on the GPU, and PB on the CPU.
The TensorRT backend needs UFF files but if you compile for both Tensorflow and TensorRT backends
you can use both files.

b) Major bug reveal: Slicing of inputs for multi-gpu training slowed down network inference by 2x.

On a P100 GPU I am getting 60 kn/s as compared to the 10kn/s I used to get with the previous version.
I still need to train a strong network from ccrl+cegt games which i will do later.

Please let me know if the GPU version of windows works!
If your GPU does not support HALF precision change the float_type to FLOAT.

Code: Select all

#######################################
# NN
# use_nn                 - turn off/on neural network
# nn_path                - path to neural network
# device_type            - CPU or GPU
# delay                  - Sleep threads for this amount. Set this to 1
#                          if you are using way too many threads than you
#                          have physical cores (maybe when >16x more).
# float_type             - inference precison for tensorrt = FLOAT/HALF/INT8  
#######################################
use_nn                   1
nn_path                  nets-ccrl/net-6x64.uff
device_type              GPU
n_devices                1
delay                    0
float_type               HALF
Note that the GPU version is compiled only with TensorRT backend so it only works with UFF format graphs.

Also, I have added a step by step guide on how to compile egbbdll on linux machines here

https://github.com/dshawul/egbbdll

regards,
Daniel
Top
Damir
Posts: 2801
Joined: Mon Feb 11, 2008 3:53 pm
Location: Denmark
Full name: Damir Desevac

Re: Scorpio 2.8.9 with TensorRT support

Post by Damir »

Thanks Daniel. It is nice to see competition to Leela. Can You tell us how much this version of Scorpio is to Leela in playing strength is ?
Should Leela Team be worried, since there has not been any improvement with Leela for over a month now ? :) :)
Top
Norbert Raimund Leisner
Posts: 1643
Joined: Tue May 20, 2008 4:57 pm
Location: Augsburg - Germany

Re: Scorpio 2.8.9 with TensorRT support

Post by Norbert Raimund Leisner »

The last working Scorpio Edition on my computer Intel i5 Quad , 4 GB RAM (4 CPUs) is v2.8.6 for Windows 7 x64 (both executable files of each version)

2.8.7-2.8.8-2.8.9 can be embedded under WinBoard Interface or Arena GUI, but before the first ply the engine crash each time:

unhandled exeception error in win32 application 788/ 4232/ 4688/ 4824 / etc. are the debug informations. > scorpio.exe exited unexpectedly

What can be done? I have only installed https://github.com/dshawul/Scorpio/rele ... cts-nn.zip - https://github.com/dshawul/Scorpio/rele ... -nn.tar.gz and extracted these folders

Best wishes,
Norbert
Top
Daniel Shawul
Posts: 4185
Joined: Tue Mar 14, 2006 11:34 am
Location: Ethiopia

Re: Scorpio 2.8.9 with TensorRT support

Post by Daniel Shawul »

Norbert Raimund Leisner wrote: ↑Sun Oct 14, 2018 3:10 pm The last working Scorpio Edition on my computer Intel i5 Quad , 4 GB RAM (4 CPUs) is v2.8.6 for Windows 7 x64 (both executable files of each version)

2.8.7-2.8.8-2.8.9 can be embedded under WinBoard Interface or Arena GUI, but before the first ply the engine crash each time:

unhandled exeception error in win32 application 788/ 4232/ 4688/ 4824 / etc. are the debug informations. > scorpio.exe exited unexpectedly

What can be done? I have only installed https://github.com/dshawul/Scorpio/rele ... cts-nn.zip - https://github.com/dshawul/Scorpio/rele ... -nn.tar.gz and extracted these folders

Best wishes,
Norbert
Norbert, are you trying to run the CPU or GPU version. First thing to do it to try and run with without using neural networks.
Set in scorpio.ini use_nn 0. That should work. If that is the case, ad the path to egbbdll in the "Path" environment variable. This is actually
needed only for the GPU version. For the CPU version just setting the nn_path in scorpio.ini should work right away.

Daniel
Top
Daniel Shawul
Posts: 4185
Joined: Tue Mar 14, 2006 11:34 am
Location: Ethiopia

Re: Scorpio 2.8.9 with TensorRT support

Post by Daniel Shawul »

Daniel Shawul wrote: ↑Sun Oct 14, 2018 11:51 pm
Norbert Raimund Leisner wrote: ↑Sun Oct 14, 2018 3:10 pm The last working Scorpio Edition on my computer Intel i5 Quad , 4 GB RAM (4 CPUs) is v2.8.6 for Windows 7 x64 (both executable files of each version)

2.8.7-2.8.8-2.8.9 can be embedded under WinBoard Interface or Arena GUI, but before the first ply the engine crash each time:

unhandled exeception error in win32 application 788/ 4232/ 4688/ 4824 / etc. are the debug informations. > scorpio.exe exited unexpectedly

What can be done? I have only installed https://github.com/dshawul/Scorpio/rele ... cts-nn.zip - https://github.com/dshawul/Scorpio/rele ... -nn.tar.gz and extracted these folders

Best wishes,
Norbert
Norbert, are you trying to run the CPU or GPU version. First thing to do it to try and run with without using neural networks.
Set in scorpio.ini use_nn 0. That should work. If that is the case, ad the path to egbbdll in the "Path" environment variable. This is actually
needed only for the GPU version. For the CPU version just setting the nn_path in scorpio.ini should work right away.

Daniel
Ofcourse, you also need to download network files
https://github.com/dshawul/Scorpio/rele ... pd-10m.zip

And pick one of 2x32, 6x64 or 12x128 net. For the CPU use the PB files and for the GPU use the UFF files.
That was probably why it was crashing for you when you tried to run it.

Daniel
Top
User avatar
CMCanavessi
Posts: 1142
Joined: Thu Dec 28, 2017 4:06 pm
Location: Argentina

Re: Scorpio 2.8.9 with TensorRT support

Post by CMCanavessi »

Hi Daniel, I'm trying to make it work in my pc and it crashes all the time.

My scorpio.ini looks like this:

Code: Select all

#############################################
# log on/off   - turn logging on and off
# resign       - resign value in centipawns
#############################################
log                 off
resign              800
##################################################################
# Hashtable sizes in mb.
#      ht - main hash table shared by ALL threads
#     eht - evalualtion hash table allocated PER thread
#     pht - pawn hash table allocated PER thread
# Usually 1m or 2mb is enough for pawn hash table, but the others
# can be set as high as possible
##################################################################
ht  	            256
eht                 8
pht                 1
##################################################################
# montecarlo - Use montecarlo tree search (MCTS) if set to 1
# treeht - Maximum size of tree to store in memory given in MB.
#          Note that this memory is not allocated at start up; it only
#          specifies the maximum limit the tree could grow to.
#          Note: Don't think you are making scorpio weak by not setting
#          this memory to high value, infact treeht=0 is the strongest 
#          MCTS setting.
# frac_freeze_tree    [0 - 100] - freeze tree after this percentage of time is spent
# frac_alphabeta      [0 - 100] - use alpha-beta search this percent of the time
# frac_abrollouts     [0 - 100] - use alpha-beta rollouts search this percent of the time
# frac_width          [0 - 1e3] - determines amount of progressive widening
# mcts_strategy_depth [0 - 64]  - Lookahead depth for MCTS rollouts
# alphabeta_depth     [0 - 64]  - Fixed search depth for standard alphabeta searches at the leaves
# evaluate_depth      [0 - 64]  - Fixed search depth for evaluation of new children during expansion
# backup_type         [0 - 6]   - minmax = 0, averaging = 1, mixed = 2, for mixing score with 
#                                 previous ID use 3,4,5 resp. 6 is classic MCTS.
# reuse_tree                    - reuse MCTS tree between moves
##################################################################
montecarlo          1
treeht              12800
frac_freeze_tree    100
frac_alphabeta      0
frac_abrollouts     20
frac_width          100
mcts_strategy_depth 30
alphabeta_depth     16
evaluate_depth      0
backup_type         6
reuse_tree          1
#############################################################################
# Parallel search setup
#   mt            - number of processors
#   smp_depth     - split depth for smp
#   cluster_depth - split depth for cluster search
#   message_poll_nodes - number nodes to search before checking for message
#############################################################################
mt                  1
smp_depth           8
cluster_depth       12
message_poll_nodes  20
smp_type            YBW
cluster_type        YBW 
##########################################################################
# EGBB
# egbb_path              - path to endgame egbbs 
# egbb_cache_size        - cache size for egbbs
# egbb_ply_limit_percent - percent of plies from the root to egbb probing
# egbb_depth_limit       - depth from the leaves to egbb probing
# egbb_load_type         - egbb load type with the following values
#                    0 = none are loaded
#                    1 = all 3/4 men are loaded
#                    2 = Not implemented yet
#                    3 = all 5 men are loaded
##########################################################################
egbb_path                F:\Scorpio\
egbb_cache_size          32
egbb_load_type           3
egbb_depth_limit         6
#######################################
# NN
# use_nn                 - turn off/on neural network
# nn_path                - path to neural network
# device_type            - CPU or GPU
# delay                  - Sleep threads for this amount. Set this to 1
#                          if you are using way too many threads than you
#                          have physical cores (maybe when >16x more).
# float_type             - inference precison for tensorrt = FLOAT/HALF/INT8  
#######################################
use_nn                   1
nn_path                  F:\Scorpio\net-20x256.uff
device_type              GPU
n_devices                1
delay                    0
float_type               HALF
And I have the following files in F:\Scorpio

Code: Select all

F:\Scorpio>DIR
 El volumen de la unidad F es Downloads
 El número de serie del volumen es: 4C45-B71C

 Directorio de F:\Scorpio

26/10/2018  17:17    <DIR>          .
26/10/2018  17:17    <DIR>          ..
26/08/2018  01:33        67.318.784 cublas64_100.dll
26/08/2018  01:33           416.256 cudart64_100.dll
21/09/2018  04:19       338.195.968 cudnn64_7.dll
06/10/2018  19:35            82.944 egbbdll64.dll
24/10/2018  20:37        90.170.334 net-20x256.uff
25/09/2018  19:55       110.258.688 nvinfer.dll
25/09/2018  19:55         2.358.784 nvparsers.dll
               7 archivos    608.801.758 bytes
               2 dirs  606.091.501.568 bytes libres

F:\Scorpio>
There's a line in the Arena debug file that reads "2018-10-26 17:20:21,537<--1:EgbbProbe not Loaded!" which I don't know if it's normal or not, here's the full output:

Code: Select all

2018-10-26 17:20:20,715*1*--------------------Starting engine 1 Scorpio 2.8.9 x64---------------------
2018-10-26 17:20:20,716*1*Configured Engine 1 Type:   WB2
2018-10-26 17:20:20,716*1*Engine 1 dir: F:\Chess\Engines Windows\Scorpio\2.8.9
2018-10-26 17:20:20,716*1*Engine 1 commandline: "F:\Chess\Engines Windows\Scorpio\2.8.9\scorpio.exe" 
2018-10-26 17:20:20,847*1*Child Process Prio Adj:  PID 55048 conhost.exe
2018-10-26 17:20:20,882*1*Engine 1 ProcessID: 70696
2018-10-26 17:20:20,882*1*Engine 1 Prio:32 ThreadPrio:0
2018-10-26 17:20:20,882-->1:xboard
2018-10-26 17:20:20,882-->1:protover 2
2018-10-26 17:20:20,919<--1:feature done=0
2018-10-26 17:20:20,919-->1:accepted done
2018-10-26 17:20:20,919<--1:ht 16777216 X 16 = 256.0 MB
2018-10-26 17:20:20,920<--1:eht 524288 X 8 = 8.0 MB
2018-10-26 17:20:20,968<--1:pht 32768 X 24 = 0.8 MB
2018-10-26 17:20:20,968<--1:treeht 335539200 X 40 = 12799.8 MB
2018-10-26 17:20:20,969<--1:processors [1]
2018-10-26 17:20:20,970<--1:feature name=1 myname="Scorpio_2.8.9 MCTS+NN"
2018-10-26 17:20:20,970-->1:accepted name
2018-10-26 17:20:20,970<--1:feature sigint=0 sigterm=0
2018-10-26 17:20:20,970-->1:accepted sigint
2018-10-26 17:20:20,970-->1:accepted sigterm
2018-10-26 17:20:20,993<--1:feature setboard=1 draw=0 colors=0
2018-10-26 17:20:20,993-->1:accepted setboard
2018-10-26 17:20:20,993-->1:accepted draw
2018-10-26 17:20:20,993-->1:accepted colors
2018-10-26 17:20:20,993<--1:feature smp=0 memory=0
2018-10-26 17:20:20,993-->1:accepted smp
2018-10-26 17:20:20,993-->1:accepted memory
2018-10-26 17:20:21,014<--1:feature option="log -check 0"
2018-10-26 17:20:21,014-->1:accepted option
2018-10-26 17:20:21,014<--1:feature option="clear_hash -button"
2018-10-26 17:20:21,014-->1:accepted option
2018-10-26 17:20:21,015<--1:feature option="resign -spin 800 100 30000"
2018-10-26 17:20:21,015-->1:accepted option
2018-10-26 17:20:21,036<--1:feature option="cores -spin 1 1 4096"
2018-10-26 17:20:21,036-->1:accepted option
2018-10-26 17:20:21,036<--1:feature option="ht -spin 256 1 131072"
2018-10-26 17:20:21,036-->1:accepted option
2018-10-26 17:20:21,057<--1:feature option="eht -spin 8 1 16384"
2018-10-26 17:20:21,057-->1:accepted option
2018-10-26 17:20:21,057<--1:feature option="pht -spin 1 1 256"
2018-10-26 17:20:21,057-->1:accepted option
2018-10-26 17:20:21,078<--1:feature option="egbb_path -path F:\Scorpio\"
2018-10-26 17:20:21,078-->1:accepted option
2018-10-26 17:20:21,079<--1:feature option="nn_path -path F:\Scorpio\net-20x256.uff"
2018-10-26 17:20:21,079-->1:accepted option
2018-10-26 17:20:21,080<--1:feature option="egbb_cache_size -spin 32 1 16384"
2018-10-26 17:20:21,080-->1:accepted option
2018-10-26 17:20:21,101<--1:feature option="egbb_load_type -spin 3 0 3"
2018-10-26 17:20:21,101-->1:accepted option
2018-10-26 17:20:21,122<--1:feature option="egbb_depth_limit -spin 6 0 64"
2018-10-26 17:20:21,122-->1:accepted option
2018-10-26 17:20:21,143<--1:feature option="egbb_ply_limit_percent -spin 75 0 100"
2018-10-26 17:20:21,143-->1:accepted option
2018-10-26 17:20:21,164<--1:feature option="n_devices -spin 1 1 128"
2018-10-26 17:20:21,164-->1:accepted option
2018-10-26 17:20:21,185<--1:feature option="device_type -combo *CPU /// GPU "
2018-10-26 17:20:21,185-->1:accepted option
2018-10-26 17:20:21,206<--1:feature option="delay -spin 0 0 1000"
2018-10-26 17:20:21,206-->1:accepted option
2018-10-26 17:20:21,206<--1:feature option="float_type -combo FLOAT /// *HALF  /// INT8 "
2018-10-26 17:20:21,206-->1:accepted option
2018-10-26 17:20:21,227<--1:feature option="smp_type -combo *YBW /// ABDADA /// SHT "
2018-10-26 17:20:21,227-->1:accepted option
2018-10-26 17:20:21,227<--1:feature option="smp_depth -spin 8 1 10"
2018-10-26 17:20:21,227-->1:accepted option
2018-10-26 17:20:21,248<--1:feature option="use_singular -check 1"
2018-10-26 17:20:21,248-->1:accepted option
2018-10-26 17:20:21,269<--1:feature option="use_probcut -check 0"
2018-10-26 17:20:21,269-->1:accepted option
2018-10-26 17:20:21,290<--1:feature option="singular_margin -spin 32 0 1000"
2018-10-26 17:20:21,290-->1:accepted option
2018-10-26 17:20:21,311<--1:feature option="probcut_margin -spin 195 0 1000"
2018-10-26 17:20:21,311-->1:accepted option
2018-10-26 17:20:21,311<--1:feature option="aspiration_window -spin 10 0 100"
2018-10-26 17:20:21,311-->1:accepted option
2018-10-26 17:20:21,332<--1:feature option="UCTKmin -spin 0 0 100"
2018-10-26 17:20:21,332-->1:accepted option
2018-10-26 17:20:21,332<--1:feature option="UCTKmax -spin 30 0 100"
2018-10-26 17:20:21,332-->1:accepted option
2018-10-26 17:20:21,353<--1:feature option="reuse_tree -check 1"
2018-10-26 17:20:21,353-->1:accepted option
2018-10-26 17:20:21,353<--1:feature option="backup_type -combo *MINMAX AVERAGE MIX MINMAX_MEM AVERAGE_MEM MIX_MEM CLASSIC"
2018-10-26 17:20:21,353-->1:accepted option
2018-10-26 17:20:21,353<--1:feature option="frac_alphabeta -spin 0 0 100"
2018-10-26 17:20:21,353-->1:accepted option
2018-10-26 17:20:21,374<--1:feature option="frac_freeze_tree -spin 100 0 100"
2018-10-26 17:20:21,374-->1:accepted option
2018-10-26 17:20:21,374<--1:feature option="frac_abrollouts -spin 20 0 100"
2018-10-26 17:20:21,374-->1:accepted option
2018-10-26 17:20:21,375<--1:feature option="frac_width -spin 100 0 1000"
2018-10-26 17:20:21,375-->1:accepted option
2018-10-26 17:20:21,396<--1:feature option="mcts_strategy_depth -spin 30 0 100"
2018-10-26 17:20:21,396-->1:accepted option
2018-10-26 17:20:21,396<--1:feature option="alphabeta_depth -spin 16 1 100"
2018-10-26 17:20:21,396-->1:accepted option
2018-10-26 17:20:21,417<--1:feature option="evaluate_depth -spin 0 0 100"
2018-10-26 17:20:21,417-->1:accepted option
2018-10-26 17:20:21,417<--1:feature option="montecarlo -check 1"
2018-10-26 17:20:21,417-->1:accepted option
2018-10-26 17:20:21,417<--1:feature option="treeht -spin 12799 0 131072"
2018-10-26 17:20:21,417-->1:accepted option
2018-10-26 17:20:21,438<--1:feature done=1
2018-10-26 17:20:21,438-->1:accepted done
2018-10-26 17:20:21,438-->1:option device_type=GPU
2018-10-26 17:20:21,452*1*Child Process Prio Adj:  PID 55048 conhost.exe
2018-10-26 17:20:21,516-->1:new
2018-10-26 17:20:21,516-->1:random
2018-10-26 17:20:21,516-->1:level 0 1 1
2018-10-26 17:20:21,516-->1:post
2018-10-26 17:20:21,516-->1:hard
2018-10-26 17:20:21,516-->1:name Tucano 7.06 x64
2018-10-26 17:20:21,537<--1:EgbbProbe not Loaded!
2018-10-26 17:20:21,537<--1:loading_time = 0s
2018-10-26 17:20:21,718<--1:Hello Tucano7.06x64!
2018-10-26 17:20:48,917-->1:computer
2018-10-26 17:20:48,926*1*Start calc, move no: 0
2018-10-26 17:20:48,926-->1:level 0 1 1
2018-10-26 17:20:48,926-->1:new
2018-10-26 17:20:48,926-->1:random
2018-10-26 17:20:48,926-->1:level 0 1 1
2018-10-26 17:20:48,926-->1:post
2018-10-26 17:20:48,926-->1:hard
2018-10-26 17:20:48,926-->1:name Tucano 7.06 x64
2018-10-26 17:20:48,926-->1:computer
2018-10-26 17:20:48,927-->1:time 6000
2018-10-26 17:20:48,927-->1:otim 6000
2018-10-26 17:20:48,927-->1:go
2018-10-26 17:20:48,956<--1:Hello Tucano7.06x64!
2018-10-26 17:20:48,956<--1:[st = 4600ms, mt = 29250ms , hply = 0 , moves_left 45]
2018-10-26 17:20:56,749-->1:?
Do you have any suggestions?
Follow my tournament and some Leela gauntlets live at http://twitch.tv/ccls
Top
Daniel Shawul
Posts: 4185
Joined: Tue Mar 14, 2006 11:34 am
Location: Ethiopia

Re: Scorpio 2.8.9 with TensorRT support

Post by Daniel Shawul »

You need to add the path to egbbdll in the "Path" environment variable.

After that please also set mt=128 and use delay = 1 in the scorpio.ini

Daniel
Top
Hai
Posts: 598
Joined: Sun Aug 04, 2013 1:19 pm

Re: Scorpio 2.8.9 with TensorRT support

Post by Hai »

Daniel Shawul wrote: ↑Sun Oct 07, 2018 3:06 pm Hi,

Scorpio 2.8.0 MCTS+NN with TensorFlow and TensorRT backend is available here

https://github.com/dshawul/Scorpio/releases

Changes:

a) Implemented TensorRT backend with the option of FLOAT/HALF/INT8 precision calculations.
There are now two formats for the network UFF on the GPU, and PB on the CPU.
The TensorRT backend needs UFF files but if you compile for both Tensorflow and TensorRT backends
you can use both files.

b) Major bug reveal: Slicing of inputs for multi-gpu training slowed down network inference by 2x.

On a P100 GPU I am getting 60 kn/s as compared to the 10kn/s I used to get with the previous version.
I still need to train a strong network from ccrl+cegt games which i will do later.

Please let me know if the GPU version of windows works!
If your GPU does not support HALF precision change the float_type to FLOAT.

Code: Select all

#######################################
# NN
# use_nn                 - turn off/on neural network
# nn_path                - path to neural network
# device_type            - CPU or GPU
# delay                  - Sleep threads for this amount. Set this to 1
#                          if you are using way too many threads than you
#                          have physical cores (maybe when >16x more).
# float_type             - inference precison for tensorrt = FLOAT/HALF/INT8  
#######################################
use_nn                   1
nn_path                  nets-ccrl/net-6x64.uff
device_type              GPU
n_devices                1
delay                    0
float_type               HALF
Note that the GPU version is compiled only with TensorRT backend so it only works with UFF format graphs.

Also, I have added a step by step guide on how to compile egbbdll on linux machines here

https://github.com/dshawul/egbbdll

regards,
Daniel
Can you tell us more about the INT8 precision?
Top
Jamal Bubker
Posts: 326
Joined: Mon May 24, 2010 4:32 pm

Re: Scorpio 2.8.9 with TensorRT support

Post by Jamal Bubker »

Thanks Daniel for this new release ! 8-) 8-)
Top
Hai
Posts: 598
Joined: Sun Aug 04, 2013 1:19 pm

Re: Scorpio 2.8.9 with TensorRT support

Post by Hai »

Does it loses many elo points when using int8 precision calculations?
Top

Return to “Computer Chess Club: General Topics”