Scorpio 3.0.2

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Modern Times
Posts: 2420
Joined: Thu Jun 07, 2012 9:02 pm

Re: Scorpio 3.0.2

Post by Modern Times » Mon Oct 07, 2019 6:17 pm

I failed to get the last version of Scorpio running after spending quite a lot of time on it, so decided enough was enough. I don't mind having to carry out some detailed steps, as long as there is a high degree of certainty that it will work at the end. And there seems something fundamentally weird with running the engine from a batch file in the GUI. I'd rather not do that either. I think that was only necessary to give the path, and I'm happy setting the system variable myself on that.
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

Daniel Shawul
Posts: 3762
Joined: Tue Mar 14, 2006 10:34 am
Location: Ethiopia
Contact:

Re: Scorpio 3.0.2

Post by Daniel Shawul » Mon Oct 07, 2019 7:11 pm

IIRC you were able to get it working on the command line but not in a GUI. Now UCI is supported natively so you could try that instead.
The installation is easy: download and double-click the installer

https://github.com/dshawul/Scorpio/rele ... nstall.bat

I have now made it so that it generates the calibrate.dat file, needed for INT8, during installation so there is almost nothing the user has to do manually now.
If the user prefers, he can edit scorpio.ini to change networks to use, floating point precision, number of GPUs etc...

About guarantees, nothing in life is guarnteed, so you will have to try and find out or not :)

Daniel

supersharp77
Posts: 764
Joined: Sat Jul 05, 2014 5:54 am
Location: Southwest USA

Re: Scorpio 3.0.2

Post by supersharp77 » Mon Oct 07, 2019 11:02 pm

Modern Times wrote:
Mon Oct 07, 2019 6:17 pm
I failed to get the last version of Scorpio running after spending quite a lot of time on it, so decided enough was enough. I don't mind having to carry out some detailed steps, as long as there is a high degree of certainty that it will work at the end. And there seems something fundamentally weird with running the engine from a batch file in the GUI. I'd rather not do that either. I think that was only necessary to give the path, and I'm happy setting the system variable myself on that.
Amen Brother..I'm with you on that case...Had lots of trouble getting Scorpio to work from the earliest times..v2.76 and v2.77 were a big problems...(ini file).. v2.7 x32 was infected by some sort of code/virus my antivirus programs liked to eat and now with this "Scorpio net fiasco situation" "I've about reached my limits of patience" Too much time on setup..(exe works but no moves made x64)...I do like Scorpio although results have been spotty in most/all of my tournaments with Scorpio not winning or placing in the top ten in none that I can remember (new or old versions) The JA versions seemed to be easiest to get to work with Scorpio Beta playing well in some of my tourneys for some reason..2.79 Is working but not much success with 2.88 (ini file) 2.9 and 3.0 (ini file issues/problems making the program not able to make moves)...No luck in Fritz, Chess OK and Shredder GUI's its a sad situation..Scorpios ini file setup needs a complete rework..setup too complex, exe can't seem to find the net files...BIG TROUBLE..NEEDS WORK! 8-)

Daniel Shawul
Posts: 3762
Joined: Tue Mar 14, 2006 10:34 am
Location: Ethiopia
Contact:

Re: Scorpio 3.0.2

Post by Daniel Shawul » Tue Oct 08, 2019 12:29 am

pohl4711 wrote:
Mon Oct 07, 2019 3:23 pm
Daniel Shawul wrote:
Mon Oct 07, 2019 3:21 pm
The net 42510 (an SE net) has problems with INT8 and TensorRT -- Werner found out the bug, and it will produce garbage result with INT8.
I still have not found the reason why it does not work with TensorRT when smaller nets of same architecure, such as badgyal, work just fine with INT8. However, you can still use 42510 in HALF precision mode, but it will be slower.. I recommend to use 32930 till i figure out the reason.
Ah. OK. Same here. 42510 plays horrible with INT8...32930 seems to work

If you find the problem, it would be nice to get Net T40B.4-160 from J.Horthos, which is the best Net so far and it is 20x256 SE, too. No need for a 42510 Net, which is outdated.
It seems the behaviour is random. I got the J13-410, which is a 24x320 net, working with INT8 (it is SE, WDL+convolutions head) and it works fine).
I don't know how strong a net it is but i can updload it.
Maddex-SE which has exactly same architecture as T40 alsow works fine with INT8.
But T40 and T40B.4-160 do not work with INT8...

pohl4711
Posts: 1119
Joined: Sat Sep 03, 2011 5:25 am
Location: Berlin, Germany
Contact:

Re: Scorpio 3.0.2

Post by pohl4711 » Tue Oct 08, 2019 7:09 am

Daniel Shawul wrote:
Tue Oct 08, 2019 12:29 am


It seems the behaviour is random. I got the J13-410, which is a 24x320 net, working with INT8 (it is SE, WDL+convolutions head) and it works fine).
I don't know how strong a net it is but i can updload it.
Maddex-SE which has exactly same architecture as T40 alsow works fine with INT8.
But T40 and T40B.4-160 do not work with INT8...
Strange...
But a upload of J13-410 would be interesting, of course. It is a large net, so a fast computing with INT8 could be a strong playing solution.
But J13B.2-178 would be somewhat stronger - around +10 Elo.
("J13B.2 attempt to improve on J13 using the same method as for T40B.4")

I would love to do a testrun with that net and Scorpio. But before, I will test Scorpio 3.0.2 with Net 32930 in the near future. Because I have a testresult of lc0 with Net 32930 already, so this testrun will make a comparison possible (does lc0 or Scorpio play better with the same net?!).
Stay tuned!

https://www.sp-cc.de/lc0-testing.htm

User avatar
Werner
Posts: 2428
Joined: Wed Mar 08, 2006 9:09 pm

Re: Scorpio 3.0.2

Post by Werner » Tue Oct 08, 2019 10:01 am

Ok, it works here:



...and I am quite sure this Scorpio scores better than LCZero with the same net.
Werner

pohl4711
Posts: 1119
Joined: Sat Sep 03, 2011 5:25 am
Location: Berlin, Germany
Contact:

Re: Scorpio 3.0.2

Post by pohl4711 » Tue Oct 08, 2019 12:24 pm

Werner wrote:
Tue Oct 08, 2019 10:01 am
Ok, it works here:

[White "Scorpio 3.0.2 MCTS+NN J13B.2-200"]
...and I am quite sure this Scorpio scores better than LCZero with the same net.
Cool !!!

I will test that. Now, there is no need for testing Scorpio with the old 32930 Net, anymore. I will test Scorpio (INT8-speed, of course) with that Terminator-Net (J13B.2-200)
And I will test lc0 with the same net, too. 3000 games each. Then, we will se, which engine uses the net better...
Because I have 2 RTX2060-Notebooks, now, the testrun of Scorpio will start now (will take 6 days). And the lc0-testrun will start Friday(?), when the Allie 0.5dev testrun is finished.

Stay tuned!

From Scorpio-website (important): "Oct 7: Added WDL head support. T40 were problematic with INT8 so those are replaced with a Terminator net J13B.2-200 from JHortos. It is a 24x320 net and uses SE, WDL head, convolution policy. To use this net you have to add wdl_head 1 in scorpio.ini, otherwise result will be bad although it may seem it is working properly."

Daniel Shawul
Posts: 3762
Joined: Tue Mar 14, 2006 10:34 am
Location: Ethiopia
Contact:

Re: Scorpio 3.0.2

Post by Daniel Shawul » Tue Oct 08, 2019 1:29 pm

Indeed make sure to use lateest scorpio exe that has "WDL value head" support, and also set "wdl_head 1".
I am afraid many people are going to make this mistake since wdl_head is not on by default -- only net that needs it is the J13B.2-200.
The difference between the two is that without WDL, only the winning probablity without draws is optimized i.e. W, while the latter optimizes
W+D/2. It is almost indistinguishable the difference between these two if you run it from the start position.

with WDL

Code: Select all

$ ./scorpio.sh wdl_head 1 float_type INT8 nn_path ../nets-lczero/J13B.2-200.uff go quit

# [st = 10835ms, mt = 29250ms , hply = 0 , moves_left 10]
63 32 109 14906  d2-d4 Ng8-f6 c2-c4 e7-e6 g2-g3 Bf8-b4 Bc1-d2 Bb4-e7 Bf1-g2 d7-d5 Ng1-f3 Ke8-g8 Ke1-g1 c7-c6 Qd1-c2 Nb8-d7
64 33 222 31567  d2-d4 Ng8-f6 c2-c4 e7-e6 g2-g3 d7-d5 Bf1-g2 Bf8-b4 Bc1-d2 Bb4-e7 Ng1-f3 Ke8-g8 Ke1-g1 c7-c6 Qd1-c2 Nb8-d7 Rf1-d1 b7-b6
65 34 331 49028  d2-d4 Ng8-f6 c2-c4 e7-e6 g2-g3 Bf8-b4 Bc1-d2 Bb4-e7 Bf1-g2 d7-d5 Ng1-f3 Ke8-g8 Ke1-g1 c7-c6 Qd1-c2 Nb8-d7 Rf1-d1 b7-b6 Bd2-f4
66 33 440 69364  d2-d4 Ng8-f6 c2-c4 e7-e6 g2-g3 Bf8-b4 Bc1-d2 Bb4-e7 Qd1-c2 d7-d5 Bf1-g2 Nb8-d7 Ng1-f3 Ke8-g8 Ke1-g1 c7-c6 Rf1-d1 b7-b6 Bd2-f4 Bc8-a6 Nb1-d2 Nf6-h5 Bf4-e3 Ra8-c8 Ra1-c1 h7-h6
67 33 549 88845  d2-d4 Ng8-f6 c2-c4 e7-e6 g2-g3 a7-a6 Bf1-g2 d7-d5 Ng1-f3 d5xc4 Ke1-g1 Nb8-c6 e2-e3 Bc8-d7 Qd1-e2 b7-b5 b2-b3 c4xb3 a2xb3 Bf8-e7 Bc1-d2 Ke8-g8 Rf1-c1 Ra8-a7 Nb1-c3
68 33 658 107778  d2-d4 Ng8-f6 c2-c4 e7-e6 g2-g3 Bf8-b4 Bc1-d2 Bb4-e7 Bf1-g2 d7-d5 Ng1-f3 Ke8-g8 Ke1-g1 c7-c6 Qd1-c2 Nb8-d7 Rf1-d1 b7-b6 Bd2-f4 Bc8-a6 Nb1-d2
69 33 767 127638  d2-d4 d7-d5 c2-c4 d5xc4 e2-e3 e7-e6 Bf1xc4 Ng8-f6 Ng1-f3 c7-c5 Ke1-g1 a7-a6 d4xc5 Bf8xc5 Qd1xd8 Ke8xd8 Bc4-e2 Kd8-e7 Nb1-d2 Bc8-d7
70 33 848 143499  d2-d4 d7-d5 c2-c4 d5xc4 e2-e3 e7-e6 Bf1xc4 Ng8-f6 Ng1-f3 c7-c5 Ke1-g1 a7-a6 d4xc5 Bf8xc5 Qd1xd8 Ke8xd8 Bc4-e2 Kd8-e7 Nb1-d2 Bc8-d7
# Move   Value=(V,P,V+P)   Policy  Visits                  PV
# ----------------------------------------------------------------------------------
#  1   (0.548,0.530,0.539)  17.94   34382   e2-e4 e7-e5 Ng1-f3 Nb8-c6 Bf1-c4 Bf8-c5 Ke1-g1 Ng8-f6 d2-d3 d7-d6 c2-c3 a7-a6 Rf1-e1 h7-h6 a2-a4 Bc5-a7 Nb1-d2 Ke8-g8 h2-h3 Bc8-e6
#  2   (0.552,0.530,0.541)  17.43   83106   d2-d4 d7-d5 c2-c4 d5xc4 e2-e3 e7-e6 Bf1xc4 Ng8-f6 Ng1-f3 c7-c5 Ke1-g1 a7-a6 d4xc5 Bf8xc5 Qd1xd8 Ke8xd8 Bc4-e2 Kd8-e7 Nb1-d2 Bc8-d7
#  3   (0.545,0.530,0.537)   9.22    9218   Ng1-f3 d7-d5 d2-d4 Ng8-f6 c2-c4 e7-e6 g2-g3 Bf8-b4 Bc1-d2 Bb4-e7 Bf1-g2 Ke8-g8 Ke1-g1 c7-c6 Qd1-c2 Nb8-d7 Rf1-d1 b7-b6 Bd2-f4 Bc8-a6
#  4   (0.546,0.530,0.538)   8.21    8818   c2-c4 e7-e5 g2-g3 Ng8-f6 Bf1-g2 Bf8-c5 d2-d3 Nb8-c6 Nb1-c3 Ke8-g8 a2-a3 a7-a5 e2-e3 Bc5-a7 Ra1-b1 h7-h6 b2-b4 a5xb4 a3xb4 d7-d6 Qd1-c2
#  5   (0.528,0.530,0.528)   5.80    1990   e2-e3 Ng8-f6 Ng1-f3 c7-c5 d2-d4 d7-d5 Bf1-e2 Nb8-c6 c2-c4 d5xc4 Be2xc4 c5xd4 e3xd4 e7-e6
#  6   (0.518,0.530,0.518)   4.87    1203   g2-g3 d7-d5 Ng1-f3 c7-c5 Bf1-g2 e7-e6 Ke1-g1 Ng8-f6 c2-c4 d5-d4 d2-d3 Nb8-c6 e2-e3 Bf8-d6 e3xd4
#  7   (0.520,0.530,0.520)   3.65     897   c2-c3 Ng8-f6 d2-d4 d7-d5 Ng1-f3 e7-e6 Bc1-f4 Bf8-d6 e2-e3 c7-c5 Bf4-g3 Ke8-g8 Nb1-d2
without WDL

Code: Select all

$ ./scorpio.sh float_type INT8 nn_path ../nets-lczero/J13B.2-200.uff go quit
63 63 109 12995  d2-d4 Ng8-f6 c2-c4 e7-e6 g2-g3 d7-d5 Bf1-g2 Bf8-b4 Bc1-d2 Bb4-e7 Qd1-c2 Nb8-d7 Ng1-f3 Ke8-g8 Ke1-g1 c7-c6 Rf1-d1 b7-b6 Bd2-f4 Bc8-a6 Nb1-d2
64 68 218 31694  d2-d4 Ng8-f6 c2-c4 e7-e6 g2-g3 Bf8-b4 Bc1-d2 Bb4-e7 Qd1-c2 d7-d5 Bf1-g2 Nb8-d7 Ng1-f3 Ke8-g8 Ke1-g1 c7-c6 Bd2-f4 b7-b6 Rf1-d1 Bc8-a6 Nb1-d2 Nf6-h5 Bf4-e3 Ra8-c8 Ra1-c1 h7-h6 Nf3-e5
65 66 327 48947  d2-d4 Ng8-f6 c2-c4 e7-e6 g2-g3 a7-a6 Bf1-g2 d7-d5 Ng1-f3 d5xc4 Nf3-e5 Bf8-b4 Nb1-c3 Nf6-d5 Ke1-g1 Nd5xc3 b2xc3 Bb4xc3 Ra1-b1 Qd8xd4 Bc1-f4 Qd4xd1 Rf1xd1 g7-g5 Bf4xg5 Nb8-d7 Ne5xd7
66 66 436 69078  d2-d4 Ng8-f6 c2-c4 e7-e6 g2-g3 a7-a6 Bf1-g2 d7-d5 Nb1-d2 c7-c5 Ng1-f3 Nb8-c6 Ke1-g1 h7-h6 c4xd5 e6xd5 Qd1-c2 c5-c4 e2-e4 Nc6-b4 Qc2-b1 Bc8-e6 Nf3-e5 Ra8-c8 a2-a3 Nb4-c6
67 68 545 89395  d2-d4 Ng8-f6 c2-c4 e7-e6 g2-g3 a7-a6 Bf1-g2 d7-d5 Nb1-d2 c7-c5 Ng1-f3 Nb8-c6 Ke1-g1 h7-h6 Nd2-b3 d5xc4 Nb3xc5 Nc6xd4 Nc5xb7 Nd4xe2 Qd1xe2 Bc8xb7 Rf1-d1 Bb7-d5 Nf3-e5 Bf8-e7 Ne5xc4 Ke8-g8 b2-b3
68 69 654 113233  d2-d4 Ng8-f6 c2-c4 e7-e6 g2-g3 a7-a6 Bf1-g2 d7-d5 Nb1-d2 c7-c5 Ng1-f3 Nb8-c6 Ke1-g1 h7-h6 Nd2-b3 d5xc4 Nb3xc5 Nc6xd4 Nc5xb7 Nd4xe2 Qd1xe2 Bc8xb7 Rf1-d1 Bb7-d5 Nf3-e5 Bf8-e7 Ne5xc4 Ke8-g8 b2-b3 Qd8-c7 Bc1-b2 Qc7-b7
69 71 764 129316  d2-d4 Ng8-f6 c2-c4 e7-e6 g2-g3 d7-d5 Bf1-g2 a7-a6 Ng1-f3 d5xc4 Nf3-e5 Bf8-b4 Nb1-c3 Nf6-d5 Ke1-g1 Nd5xc3 b2xc3 Bb4xc3 Ra1-b1 Qd8xd4 Qd1-a4 b7-b5 Qa4-a3 Qd4xe5 Bc1-f4 Qe5-f6 Bf4xc7 b5-b4 Qa3-a5 Ke8-g8
70 71 764 129339  d2-d4 Ng8-f6 c2-c4 e7-e6 g2-g3 d7-d5 Bf1-g2 a7-a6 Ng1-f3 d5xc4 Nf3-e5 Bf8-b4 Nb1-c3 Nf6-d5 Ke1-g1 Nd5xc3 b2xc3 Bb4xc3 Ra1-b1 Qd8xd4 Qd1-a4 b7-b5 Qa4-a3 Qd4xe5 Bc1-f4 Qe5-f6 Bf4xc7 b5-b4 Qa3-a5 Ke8-g8
# Move   Value=(V,P,V+P)   Policy  Visits                  PV
# ----------------------------------------------------------------------------------
#  1   (0.606,0.461,0.533)  17.94   22890   e2-e4 e7-e5 Ng1-f3 Nb8-c6 Bf1-b5 Ng8-f6 Ke1-g1 Nf6xe4 d2-d4 Ne4-d6 Bb5xc6 d7xc6 d4xe5 Nd6-f5 Qd1xd8 Ke8xd8 Nb1-c3 Kd8-e8 h2-h3 h7-h5 Nc3-e2 b7-b6 Rf1-d1 Bc8-b7 Ne2-d4 Nf5xd4 Nf3xd4 Ra8-d8 Bc1-f4
#  2   (0.606,0.461,0.533)  17.43   89759   d2-d4 Ng8-f6 c2-c4 e7-e6 g2-g3 d7-d5 Bf1-g2 a7-a6 Ng1-f3 d5xc4 Nf3-e5 Bf8-b4 Nb1-c3 Nf6-d5 Ke1-g1 Nd5xc3 b2xc3 Bb4xc3 Ra1-b1 Qd8xd4 Qd1-a4 b7-b5 Qa4-a3 Qd4xe5 Bc1-f4 Qe5-f6 Bf4xc7 b5-b4 Qa3-a5 Ke8-g8
#  3   (0.595,0.461,0.528)   9.22    7743   Ng1-f3 d7-d5 d2-d4 Ng8-f6 c2-c4 e7-e6 g2-g3 Bf8-b4 Bc1-d2 Bb4-e7 Bf1-g2 Ke8-g8 Qd1-c2 Nb8-d7 Ke1-g1 c7-c6 Rf1-d1 b7-b6 Bd2-f4 Bc8-a6 Nb1-d2 Nf6-h5 Bf4-e3 Ra8-c8 Ra1-c1
#  4   (0.592,0.461,0.526)   8.21    5021   c2-c4 e7-e5 g2-g3 Ng8-f6 Bf1-g2 Bf8-c5 d2-d3 Nb8-c6 Nb1-c3 Ke8-g8 a2-a3 a7-a5 e2-e3 Bc5-a7 Ra1-b1 h7-h6 b2-b4 a5xb4 a3xb4 d7-d6 Qd1-c2 Nc6-e7
#  5   (0.559,0.461,0.510)   5.80    1084   e2-e3 Ng8-f6 d2-d4 e7-e6 Ng1-f3 b7-b6 Bf1-d3 d7-d5 b2-b3 Bc8-b7
#  6   (0.537,0.461,0.499)   4.87     605   g2-g3 d7-d5 Ng1-f3 c7-c5 Bf1-g2 e7-e6 Ke1-g1 Ng8-f6 c2-c4 d5-d4 d2-d3 Nb8-c6 e2-e3 Bf8-d6 e3xd4
#  7   (0.547,0.461,0.504)   3.65     516   c2-c3 Ng8-f6 Ng1-f3 d7-d5 d2-d4 e7-e6 Bc1-f4 Bf8-d6 e2-e3 c7-c5 Bf4-g3 Ke8-g8 Nb1-d2
#  8   (0.524,0.461,0.492)   3.16     305   Nb1-c3 d7-d5 d2-d4 Ng8-f6 Bc1-f4 a7-a6 e2-e3 e7-e6 g2-g4 Bf8-b4 Ng1-e2 Ke8-g8 a2-a3 Bb4-a5 b2-b4 Ba5-b6
I am going to run a tournament between these two to see how they behave. I expect the WDL head to be better since I expect the W head to screw up in some position, but who knows maybe optimizing just W gives a more aggresive engine ( I am already surprized that it seems to work Ok).

pohl4711
Posts: 1119
Joined: Sat Sep 03, 2011 5:25 am
Location: Berlin, Germany
Contact:

Re: Scorpio 3.0.2

Post by pohl4711 » Tue Oct 08, 2019 2:53 pm

Daniel Shawul wrote:
Tue Oct 08, 2019 1:29 pm
Indeed make sure to use lateest scorpio exe that has "WDL value head" support, and also set "wdl_head 1".
I am afraid many people are going to make this mistake since wdl_head is not on by default -- only net that needs it is the J13B.2-200.
Of course. I use Scorpio 3.0.2 and set wdl_head to 1 in the scorpio.ini file.

pohl4711
Posts: 1119
Joined: Sat Sep 03, 2011 5:25 am
Location: Berlin, Germany
Contact:

Re: Scorpio 3.0.2

Post by pohl4711 » Wed Oct 09, 2019 9:11 am

Update: The testrun of Scorpio 3.0.2 with wdl_head 1 and Net J13B.2-200 is a disaster: After 380 games the score is only 28.2% vs. 6 AB-Engines, which means a SPCC-Elo of 3200, which is really, really bad (Scorpio 3 Maddex has 3343 SPCC-Elo). So, there is definitly something very wrong with Scorpio when using that net.
So, I started the testrun of Scorpio 3.0.2 Net 32930 this morning and after 100 games the result is much better: 60.5%, which is at the same level, that Lc0 32930 reached in my testings (60%). So, with N32930 all works fine, but with Net J13B.2-200 not...

https://www.sp-cc.de/lc0-testing.htm

Post Reply