Scorpio 2.8

[Evert]

It is good enough if you set "treeht = 0" (have to test this but I expect it to be very similar to 2.8 unless i have a bug). However, if you have treeht>0, it gets weaker with every increase of memory; lower values of treeht are better strengthwise.

Wait, so why would you set it to anything other than 0?
I find it very counter-intuitive that increasing memory doesn't improve strength. Why isn't it better to store more of the tree?

[Daniel Shawul]

I made a couple of bug fixes for MCTS and updated the download.
There was also a serious bug that hindered the treeht=0 version.
The standard version is not changed so no need to download again if you are only interested in that.

First of all, the MCTS version with treeht=128 or larger is not going to be as strong as scorpio 2.8 so don't be surprized if it looses 10-0. But it should be 500-600 elo stronger than previous version scorpio 2.7.9 MCTS.

With treeht=0, it is more or less same strength as scorpio 2.8, but this is not interesting to me. A quick match result:

Code: Select all

Finished game 37 (scorpio vs scorpio-mcts): 1-0 {Black resigns}
Score of scorpio vs scorpio-mcts: 12 - 10 - 15  [0.527] 37

What treeht=0 means is that the root and its children are stored in memory but the rest of the tree is not. At the leaves standard recursive alpha-beta search is called. This is very much like what other engines do by having a different root search, so not very much interesting.

If you set treeht=1, it will use about 25000 nodes to store the upper parts of the tree (maybe depth = 5) .

On the start position with treeht=0

Code: Select all

ht 4194304 X 16 = 64.0 MB
eht 1048576 X 8 = 8.0 MB
pht 32768 X 24 = 0.8 MB
treeht 0 X 40 = 0.0 MB
processors [1]
EgbbProbe not Loaded!
loading_time = 0s
[st = 5000ms, mt = 5000ms , hply = 0]
2 49 0 254  d2-d4 d7-d5
3 49 0 796  e2-e4 e7-e5
4 49 0 1476  e2-e4 e7-e5 Nb1-c3 Nb8-c6
5 38 1 1833  e2-e4 e7-e5 Qd1-f3
5 30 1 2279  d2-d4 d7-d5 Ng1-f3
6 40 1 2674  d2-d4 e7-e5 d4xe5 Ng8-e7
6 49 1 3840  d2-d4 d7-d5 Ng1-f3 Nb8-c6 Nb1-c3 Ng8-f6
7 39 1 4543  d2-d4 d7-d5 Ng1-f3 Nb8-c6 Qd1-d2
7 24 2 5227  d2-d4 d7-d5 e2-e4 d5xe4 a2-a3
7 30 2 6969  Ng1-f3 d7-d5 d2-d4 Nb8-c6 Nb1-c3 Ng8-f6 e2-e3
8 40 2 7786  Ng1-f3 d7-d5 d2-d4 e7-e5 d4xe5 Bf8-e7
8 49 3 11798  e2-e4 e7-e5 Ng1-f3 Nb8-c6 Nb1-c3 Ng8-f6
9 49 4 15893  d2-d4 d7-d5 Nb1-c3 Nb8-c6 Ng1-f3 Ng8-f6 e2-e3 e7-e6
10 43 5 20782  e2-e4 e7-e5 Ng1-f3 Nb8-c6 Nb1-c3 Ng8-f6 d2-d4 e5xd4 Nf3xd4 Nc6xd4
11 53 8 32928  e2-e4 e7-e5 Ng1-f3 Nb8-c6 Nb1-c3 Ng8-f6 d2-d4 e5xd4 Nf3xd4 Nc6xd4 Qd1xd4 g7-g6 Qd4-d1
11 60 12 52132  e2-e4 d7-d5 e4xd5 c7-c6 d5xc6 Nb8xc6 Ng1-f3 e7-e5 Bf1-b5 Bf8-d6 Ke1-g1
12 62 16 71089  e2-e4 e7-e5 Ng1-f3 Nb8-c6 d2-d4 e5xd4 Nf3xd4 d7-d5 Nb1-c3 Nc6xd4 Qd1xd4 d5xe4 Qd4xe4 Ng8-e7
13 52 19 93143  d2-d4 d7-d5 g2-g3 Nb8-c6 Ng1-h3 Bc8xh3 Bf1xh3 Ng8-f6
13 50 22 106253  e2-e4 e7-e5 Ng1-f3 d7-d5 Nb1-c3 d5xe4 Nf3xe5 f7-f6 Ne5-g4 Nb8-c6 Bf1-c4
14 40 29 147900  e2-e4 e7-e5 Ng1-f3 Ng8-f6 Nb1-c3 Nb8-c6 d2-d4 e5xd4 Nf3xd4 Bf8-c5 Nd4-f3 Ke8-g8 g2-g3
14 38 37 196588  Ng1-f3 d7-d5 e2-e3 Nb8-c6 d2-d4 Ng8-f6 Bf1-d3 e7-e6 h2-h4 Bf8-d6 c2-c4
15 28 43 233134  d2-d4 d7-d5 g2-g3 Nb8-c6 Nb1-c3 Ng8-f6 Bf1-g2 e7-e6 Ng1-f3 Bc8-d7 a2-a3 a7-a6 Ke1-g1
15 38 51 280526  e2-e4 e7-e5 Ng1-f3 Ng8-f6 Nb1-c3 Nb8-c6 d2-d4 e5xd4 Nf3xd4 Bf8-c5 Nd4-f3 Ke8-g8 Bf1-d3 d7-d5
16 28 76 456979  e2-e4 e7-e5 Ng1-f3 Nb8-c6 d2-d4 e5xd4 Nf3xd4 Ng8-f6 Nd4xc6 b7xc6 Nb1-c3 Bf8-d6 Bf1-d3 Ke8-g8 Bd3-c4 Qd8-e7
16 13 85 522742  d2-d4 d7-d5 g2-g3 Nb8-c6 Nb1-c3 Ng8-f6 e2-e3 e7-e6 Bf1-h3
16 38 137 890584  Ng1-f3 d7-d5 e2-e3 Nb8-c6 d2-d4 Ng8-f6 Bf1-d3 e7-e6 Bd3-b5 Bf8-d6 Bb5xc6 b7xc6 h2-h4
17 28 146 941979  Nb1-c3 d7-d5 e2-e4 d5xe4 Nc3xe4 Ng8-f6 d2-d3 Nb8-c6 Ne4xf6 g7xf6 Bc1-e3 e7-e5 f2-f3
17 44 183 1197117  Ng1-f3 d7-d5 e2-e3 Nb8-c6 d2-d4 Ng8-f6 Bf1-d3 e7-e6 Bd3-b5 Bf8-d6 Nb1-d2 Nf6-g4 Bb5-a4
18 34 204 1349610  d2-d4 Ng8-f6 Ng1-f3 d7-d5 Nb1-c3 e7-e6 Bc1-f4 Nb8-c6 e2-e3 a7-a6 Qd1-b1
18 42 247 1639561  d2-d4 Ng8-f6 Ng1-f3 d7-d5 Nb1-c3 Nb8-c6 e2-e3 e7-e6 Bf1-d3 Bc8-d7 a2-a3 a7-a6 Ke1-g1 Bf8-d6 Bd3-e2 Ke8-g8 Rf1-e1 h7-h6
19 32 280 1857088  d2-d4 Ng8-f6 Ng1-f3 d7-d5 Nb1-c3 e7-e6 e2-e3 Bf8-e7 Bf1-d3 Ke8-g8 Ke1-g1 c7-c5 a2-a3 Bc8-d7 Bc1-d2 Qd8-b6 d4xc5 Be7xc5 Nf3-e5 Qb6xb2 Bd3xh7
19 30 324 2161415  d2-d4 Ng8-f6 Ng1-f3 d7-d5 Nb1-c3 e7-e6 e2-e3 Bf8-e7 Bf1-d3 Ke8-g8 Ke1-g1 c7-c5 a2-a3 Bc8-d7 Bc1-d2 Qd8-b6 d4xc5 Be7xc5 Nf3-e5 Qb6xb2 Bd3xh7
20 29 435 2902478  Ng1-f3 d7-d5 e2-e3 Ng8-f6 Nb1-c3 e7-e6 d2-d4 Bf8-e7 Bf1-d3 Ke8-g8 Ke1-g1 c7-c5 d4xc5 Be7xc5 Bd3-e2 Nb8-c6 Nc3-b5 a7-a6 h2-h3 a6xb5
nodes = 3360051 <63 qnodes> time = 5050ms nps = 665356
Tree&#58; nodes = 21 depth = 0/0 pps = 124 visits = 627 
      qsearch_calls = 20 search_calls = 596

Note that the tree has 21 nodes = 1 root + 20 children.
The qsearch_calls is done for every child (20)
The search_calls is the number of recursive alpha-beta searches done from leaves using different depths. This parameter should go be 0 with high values of treeht because there shouldn't be any search_calls since all the tree is stored in memory.

With treeht=1

Code: Select all

ht 4194304 X 16 = 64.0 MB
eht 1048576 X 8 = 8.0 MB
pht 32768 X 24 = 0.8 MB
treeht 26214 X 40 = 1.0 MB
processors [1]
EgbbProbe not Loaded!
loading_time = 0s
[st = 5000ms, mt = 5000ms , hply = 0]
2 49 0 870  d2-d4 d7-d5
3 49 1 3593  e2-e4 e7-e5
4 47 1 7310  e2-e4 Ng8-f6 Nb1-c3
5 49 1 8410  e2-e4 e7-e5 Nb1-c3 Nb8-c6
6 42 2 13343  d2-d4 Ng8-f6 Ng1-f3
7 42 3 24812  Ng1-f3 Ng8-f6 d2-d3
8 41 5 53862  e2-e4 e7-e5 Nb1-c3 d7-d6 d2-d4
9 31 8 93271  d2-d4 d7-d5 Nb1-c3 Nb8-c6 e2-e3 Ng8-f6
9 36 11 136754  Ng1-f3 Nb8-c6 d2-d4 d7-d5 e2-e3 Ng8-f6
10 36 12 148919  d2-d4 d7-d5 Ng1-f3 Ng8-f6 e2-e3 Nb8-c6 Nb1-c3 b7-b6 h2-h3
11 41 15 161507  Ng1-f3 Ng8-f6 Nb1-c3 d7-d5 d2-d3 Nb8-c6 e2-e4
12 38 19 192379  Nb1-c3 d7-d5 d2-d4 Ng8-f6 Ng1-f3 e7-e6 e2-e3 Bf8-d6 h2-h3 Ke8-g8
13 38 25 231938  Ng1-f3 d7-d5 d2-d4 Ng8-f6 Nb1-c3 e7-e6 e2-e3 Bf8-d6 Bf1-d3 Ke8-g8 Nc3-e2
14 29 31 266609  e2-e4 e7-e5 Nb1-c3 Nb8-c6 Ng1-f3 Ng8-f6 d2-d3 d7-d5 Nc3-b5
15 39 36 296453  e2-e4 e7-e5 Ng1-f3 Nb8-c6 Nb1-c3 d7-d6 d2-d4 b7-b5
15 27 47 380454  e2-e4 e7-e5 Ng1-f3 Nb8-c6 d2-d4 e5xd4 Nf3xd4 Ng8-f6 Bc1-e3 Nc6xd4
16 37 54 421316  e2-e4 e7-e5 Nb1-c3 Nb8-c6 Ng1-f3 Ng8-f6 Bf1-c4 h7-h6 Ke1-g1 Bf8-e7 d2-d4 Nc6xd4 Nf3xd4
16 43 82 631083  Ng1-f3 d7-d5 d2-d4 c7-c6 e2-e3 h7-h6 Nb1-c3 Bc8-g4 e3-e4
17 33 95 721255  d2-d4 Ng8-f6 e2-e3 d7-d5 Ng1-f3 Nb8-c6 Bf1-e2 h7-h6 Ke1-g1 e7-e6 h2-h3 Bc8-d7 Be2-d3 Bf8-d6
17 27 129 960809  e2-e4 e7-e5 Nb1-c3 Nb8-c6 Ng1-f3 Ng8-f6 h2-h3 d7-d5 b2-b4 d5xe4
18 33 170 1249159  Ng1-f3 d7-d5 d2-d4 Nb8-c6 Nb1-c3 Ng8-f6 a2-a3 e7-e6 e2-e3 Bf8-d6 Bf1-d3 Ke8-g8 Bc1-d2 h7-h6 Ke1-g1 a7-a6 h2-h3
19 29 220 1593606  Ng1-f3 Ng8-f6 d2-d3 Nb8-c6 e2-e4 e7-e5 Nb1-c3 d7-d5 a2-a3 d5xe4 d3xe4 Qd8xd1 Ke1xd1 Bc8-e6 Ra1-a2
20 28 296 2097269  Ng1-f3 Ng8-f6 d2-d3 Nb8-c6 c2-c4 e7-e6 Nb1-c3 d7-d5 h2-h3 d5xc4 Bc1-e3
21 32 415 2881588  d2-d4 d7-d5 Nb1-c3 Ng8-f6 Ng1-f3 Nb8-c6 e2-e3 e7-e6 Bf1-d3 Bf8-d6 Ke1-g1 e6-e5 e3-e4 d5xe4 d4xe5 e4xd3 e5xd6 Qd8xd6
nodes = 3464165 <60 qnodes> time = 5045ms nps = 686653
Tree&#58; nodes = 26222 depth = 2/9 pps = 10964 visits = 55318 
      qsearch_calls = 25322 search_calls = 15927

Here 26222 nodes are used to store the upper part of the tree which has depth=2/9 (average=2 and maximum=9)

With treeht=128

Code: Select all

ht 4194304 X 16 = 64.0 MB
eht 1048576 X 8 = 8.0 MB
pht 32768 X 24 = 0.8 MB
treeht 3355392 X 40 = 128.0 MB
processors [1]
EgbbProbe not Loaded!
loading_time = 0s
[st = 5000ms, mt = 5000ms , hply = 0]
2 49 0 870  d2-d4 d7-d5
3 49 1 3593  e2-e4 e7-e5
4 47 1 7310  e2-e4 Ng8-f6 Nb1-c3
5 49 1 8410  e2-e4 e7-e5 Nb1-c3 Nb8-c6
6 42 2 13343  d2-d4 Ng8-f6 Ng1-f3
7 42 3 24812  Ng1-f3 Ng8-f6 d2-d3
8 41 5 53862  e2-e4 e7-e5 Nb1-c3 d7-d6 d2-d4
9 31 8 93271  d2-d4 d7-d5 Nb1-c3 Nb8-c6 e2-e3 Ng8-f6
9 36 14 184164  Ng1-f3 Nb8-c6 d2-d4 d7-d5 e2-e3 Ng8-f6
10 36 21 283253  d2-d4 d7-d5 Ng1-f3 Ng8-f6 e2-e3 Nb8-c6
11 31 42 651584  Nb1-c3 d7-d5 d2-d4 Nb8-c6 e2-e3 Ng8-f6
12 35 92 1452928  Nb1-c3 d7-d5 d2-d4 Ng8-f6 Ng1-f3 Nb8-c6 h2-h3 e7-e6
13 35 146 2357319  d2-d4 d7-d5 Ng1-f3 Ng8-f6 Nb1-c3 Nb8-c6 a2-a3 e7-e6
14 35 280 4510988  e2-e4 e7-e5 Nb1-c3 Nb8-c6 Ng1-f3 Ng8-f6 a2-a3 d7-d6
nodes = 8071921 <64 qnodes> time = 5021ms nps = 1607632
Tree&#58; nodes = 952348 depth = 1/14 pps = 213310 visits = 1071032 
      qsearch_calls = 924019 search_calls = 0

Here you can see there are no search calls, and all the internal nodes visited (952348) are stored in memory.
Clearly this is the weakest setup, but this is what I wanted to be tested, and what I will be improving in the future

regards,
Daniel

[Daniel Shawul]

It is good enough if you set "treeht = 0" (have to test this but I expect it to be very similar to 2.8 unless i have a bug). However, if you have treeht>0, it gets weaker with every increase of memory; lower values of treeht are better strengthwise.

Wait, so why would you set it to anything other than 0?
I find it very counter-intuitive that increasing memory doesn't improve strength. Why isn't it better to store more of the tree?

When you store everything in memory, you are going to use MCTS with alpha-beta rollouts and none of the heuristic enhanced standard alpha-beta search. The MCTS search is currently not at the level of the alpha-beta search but I guess that storing some of the upper parts of the tree shouldn't hurt much. The move ordering could be even better since you can do a sort by size of subtree for all internal nodes (similar to what is done at the root in standard alpha-beta searchers.

treeht=1 stores 26000 nodes and shouldn't be much weaker. There are somethings I am currently not doing (f.i scout search) in the MCTS part that will make it weaker.

See my other post for details.

Daniel

[Daniel Shawul]

It is good enough if you set "treeht = 0" (have to test this but I expect it to be very similar to 2.8 unless i have a bug). However, if you have treeht>0, it gets weaker with every increase of memory; lower values of treeht are better strengthwise.

Wait, so why would you set it to anything other than 0?
I find it very counter-intuitive that increasing memory doesn't improve strength. Why isn't it better to store more of the tree?

It turns out treeht=128 will equal treeht=0 with long time controls, and it might even be better. You gave me an idea that may give all treeht=x settings to be of equal strength. Spend the first say 30% of time building the tree, then freeze it and make search calls at the leaves. This should work equally well at all time controls.

I was using short time searches so i thought treeht=128 will do it, but actually what I want tested is treeht=infinity

Daniel

[supersharp77]

It is good enough if you set "treeht = 0" (have to test this but I expect it to be very similar to 2.8 unless i have a bug). However, if you have treeht>0, it gets weaker with every increase of memory; lower values of treeht are better strengthwise.

Wait, so why would you set it to anything other than 0?
I find it very counter-intuitive that increasing memory doesn't improve strength. Why isn't it better to store more of the tree?

When you store everything in memory, you are going to use MCTS with alpha-beta rollouts and none of the heuristic enhanced standard alpha-beta search. The MCTS search is currently not at the level of the alpha-beta search but I guess that storing some of the upper parts of the tree shouldn't hurt much. The move ordering could be even better since you can do a sort by size of subtree for all internal nodes (similar to what is done at the root in standard alpha-beta searchers.

treeht=1 stores 26000 nodes and shouldn't be much weaker. There are somethings I am currently not doing (f.i scout search) in the MCTS part that will make it weaker.

See my other post for details.

Daniel

Thank You Daniel.....As you know the Chess World Loves Scorpio..
AR

[Marek Soszynski]

It's been over a week. Your impressions of ScorpioMC?

The Windows version occasionally crashes while the Linux version is okay (but then I am running via Wine).

I haven't been able to get Scorpio bitbases going at all.

Tactically, treeht=0 is stronger than =1.

Unfortunately, I didn't discover any positions in which ScorpioMC gave notably good results (up to 2 hours infinite analysis) compared to other engines.

All the same, I hope the author continues his intriguing work.

[glennsamuel32]

I can not create Scorpio as new engine in Fritz GUI. The engine field is empty...

Did you try installing it with winboard? It installed with Wb2Uci under Fritz 11 here.

I cannot increase the threads...
Seems to be stuck on 1 in Fritz 15 using Wb2Uci...
Can you share your working Wb2Uci settings ?

[Daniel Shawul]

@Marek, You shouldn't expect the AlphaZero like combination of tactics and strategy in scorpioMCTS. My approach of MCTS should give exactly same style as a standard alpha-beta engine. What ScorpioMCTS aims to prove is that it is possible to get to the tactical level of standard alpha-beta engines using MCTS search that does alpha-beta rollouts. Using the exact approach of AlphaZero doesn't seem to result in a remotely strong engine of any sort due to lack of tactics (see f.i leela-chess results so far).

You can try to mix in alpha-beta and MCTS 50-50% just to get a different style by setting frac_alphabeta = 0.5. The defalt is 100% alphabeta frac_alphabeta=1 which is why you didn't see any changes of style from normal engines.

Daniel

[Marek Soszynski]

@Marek, You shouldn't expect the AlphaZero like combination of tactics and strategy in scorpioMCTS. My approach of MCTS should give exactly same style as a standard alpha-beta engine. What ScorpioMCTS aims to prove is that it is possible to get to the tactical level of standard alpha-beta engines using MCTS search that does alpha-beta rollouts. Using the exact approach of AlphaZero doesn't seem to result in a remotely strong engine of any sort due to lack of tactics (see f.i leela-chess results so far).

You can try to mix in alpha-beta and MCTS 50-50% just to get a different style by setting frac_alphabeta = 0.5. The defalt is 100% alphabeta frac_alphabeta=1 which is why you didn't see any changes of style from normal engines.

Daniel

Point taken about AlphaZero. All the same, with the only command line parameters mt=4, montecarlo=1

Upon several minutes' infinite analysis of 1.e4 e5 2.Nc3 Nc6 3.Bc4 Na5

scorpio-nopop is switching between Bxf7 and Bd4 with >2 plus score for both.

What's with 4.Bc4-d5 after so much time?

[tpoppins]

It appears that currently Scorpio MCTS performs at least 500 Elo below vanilla Scorpio. In a test 40/4 gauntlet vs. 24 engines within +/-200 Elo of plain Scorpio 2.7.9 (including the latter and vanilla 2.8) it achieved nine draws out of 960 games.

Pitting it against engines in the lower part of the Elo spectrum reveals that its performance is quite inconsistent, not to say semi-random, too: it may score 50% vs. a 2200-rated engine and only 8% vs. a 2100-rated one. Here are the results from a 40/40 gauntlet, stopped after 6 rounds:

Scorpio 2.8 MCTS 64-bit 40/40 Mini-gauntlet

Code: Select all

Scorpio - Absolute Zero 2.4.7.2 64-bit  2291    1.0 - 11.0    +0/=2/-10    8.33%
Scorpio - Barbarossa 0.4.0 64-bit       2099    1.0 - 11.0    +1/=0/-11    8.33%
Scorpio - Beowulf 2.4a 32-bit           2212    6.0 - 6.0     +5/=2/-5    50.00%
Scorpio - BigLion 2.23x                 2059    7.5 - 4.5     +6/=3/-3    62.50%
Scorpio - CT800 1.12 64-bit             2217    2.0 - 10.0    +1/=2/-9    16.67%
Scorpio - Dorpsgek Dillinger 64-bit     2202    3.5 - 8.5     +3/=1/-8    29.17%
Scorpio - Drosophila 1.5 64-bit         2500*   0.0 - 12.0    +0/=0/-12    0.00%
Scorpio - Embla 1.0.1 64-bit            2042    4.5 - 7.5     +3/=3/-6    37.50%
Scorpio - Esc 1.16                      2300*   0.0 - 12.0    +0/=0/-12    0.00%
Scorpio - FireFly 2.7.0 64-bit          2285    3.5 - 8.5     +1/=5/-6    29.17%
Scorpio - Jabba 1.0 32-bit              2052    6.5 - 5.5     +5/=3/-4    54.17%
Scorpio - Myrddin 0.87 64-bit           2381    2.0 - 10.0    +0/=4/-8    16.67%
Scorpio - NagaSkaki 5.12                2183    4.5 - 7.5     +4/=1/-7    37.50%
Scorpio - Napoleon 1.7 64-bit           2129    2.0 - 10.0    +0/=4/-8    16.67%
Scorpio - Robocide 20170718 64-bit      2211    1.0 - 11.0    +0/=2/-10    8.33%
Scorpio - Sungorus 1.4 64-bit           2312    3.0 - 9.0     +2/=2/-8    25.00%
Scorpio - Vice 1.0 64-bit               2100*   3.5 - 8.5     +2/=3/-7    29.17%
Scorpio - Zevra 1.8.2 r612 64-bit       2300*   6.5 - 5.5     +4/=5/-3    54.17%

216 games

Games: PGN
Test conditions:
Dual E5-2690v2 @3 GHz
Cute Chess GUI 2018-02-03
4-men EGBBs for Scorpio, 5-men EGTBs for the few engines that support them
FEOBOS v20.1 Contempt-3 book (custom edit)
TC: 40/20 (adapted for CCRL)
montecarlo=1
treeht=hash=256

Some of the engines above have yet to achieve a 40/40 rating, so their ratings (marked with the asterisk *) are guesstimates based on their 40/4 ratings.

I would love to see a Scorpio MCTS on CCRL's lists; however, it appears that its current MCTS development is still at a very early stage.