STS rating v13.1 for Lc0 0.21.2 with nodes = 1

[mesilikas]

How can i force lc0 to play with 1 node per move?

In Nibbler select Engine/Node limit/1
Node 1.png

And if someone can beat lc0 with 1 node and the next level (10 nodes) is too strong how can we set for example 4 nodes for a move?

[Max]

And if someone can beat lc0 with 1 node and the next level (10 nodes) is too strong how can we set for example 4 nodes for a move?

Nibbler has atm no option to choose 4 nodes as search limit. But you can help yourself!

- locate the file main.js in your Nibbler directory (resources/app)
- open main.js with a plain text editor like notepad (not word)
- find label: "Node limit" (at line 1105 for Nibbler version 1.0.2)
- change a checkbox to your needs (in the example 50000 in lines 1132,1134,1136 and 1139)
btw: already changed 100,000,000 to 100,000 in this screenshot for my needs too
- save main.js
- now you can select your node limit (50000 in our example) inside Nibbler

Or kindly ask fohristiwhirl, the author of Nibbler to add such a feature. He is a very nice person!

main.js.png

[mesilikas]

Thanks for your help Max!

[Max]

Seems like the 56-Network hit some wall. It peaked around 56216 and since then looses STS points.

Did anybody notice, if this translates to playing games too

net56216.png

Code: Select all

STS	Net
===========
1646  56012
2117  56031
2143  56052
2194  56090
2194  56120
2215  56072	
2241  56251
2248  56235
2256  56136
2270  56200
2271  56186
2273  56148
2274  56158
2284  56210
2293  56192
2293  56224
2302  56216

[Max]

Policy head T60 surpassed T40 (net 42850 got 2581) at STS suite

Code: Select all

Net   STS
----------
60030 1360
60065 1923
60100 2112
60150 2220
60200 2258
60300 2329
60400 2375
60500 2413
60660 2437
60780 2476 - last before LR drop
60880 2536
60960 2602

[peter]

Policy head T60 surpassed T40 (net 42850 got 2581) at STS suite

Thanks for the update!

[Master Om]

I just want to say that I think the distinction between "tactical" and "positional" problems is rather arbitrary and not so useful, because in real chess games good positional moves are found by tactical details. For example, let's say that rook on the 7th rank is usually good (with whatever conditions you want to specify). One engine may find some odd-looking move that after a deep search results in getting a rook to the 7th rank because preventing it loses material. Is this tactical or positional? I wonder if there is some set of problems taken from high level human games where the right move is difficult but 90% or so agreed upon as best, without distinguishing between tactical and positional problems? That might be a test with some predictive power for elo ratings.

I am not sure I would agree, more so seeing the playing of this weirdo called Leela. We do know what a "tactical" problem or puzzle is, don't we? In fact so much emphasis in this forum is about some tactical puzzles, that to me it became a clear, albeit often a bit obnoxious topic. We can find positions which per se don't pose any tactical complications, and aren't they, brushing aside elementary tactics, the majority of in-game chess positions? I do not know a strong human's perspective on that, maybe there are few "quiet" moves for humans, and even a strong human is wary of some hidden tactics move upon move. But this concept that "tactical" and "positional" are hard to separate came to me with the top regular AB engines, where I can clearly see that what I call "positional" strength is due to deeper search and deeper tactics. In case of regular engines, "positional" strength came mostly as a side effect of deeper, tactically accurate search. But Leela doesn't play this game. Leela can easily miss a three-mover shot, but in real games that is not what usually happens.

If we know what "tactics" is, then we know that WAC suite is a very tactical test-suite. I trimmed it from 300 to 145 positions which have a unique, game-changing solution. Komodo solves all of them in under 5 seconds/position, and the vast majority of them at depths 1-12 (125/145 solved), literally in 1-30 milliseconds.

Code: Select all

Engine: Komodo 13.02 64-bit (192 MB)
by Don Dailey, Larry Kaufman, Mark Lefler

1      sec    ->       142/145 
2      sec    ->       143/145 
3      sec    ->       143/145 
4      sec    ->       144/145 
5      sec    ->       145/145 

  n/s: 7.202.528  
  TotTime: 2:33m    SolTime: 11s
  Ply: 0   Positions:145   Avg Nodes:       0   Branching = 0.00
  Ply: 1   Positions:113   Avg Nodes:    1956   Branching = 0.00
  Ply: 2   Positions: 97   Avg Nodes:    4834   Branching = 2.47
  Ply: 3   Positions: 82   Avg Nodes:    6860   Branching = 1.42
  Ply: 4   Positions: 74   Avg Nodes:    9254   Branching = 1.35
  Ply: 5   Positions: 66   Avg Nodes:   14131   Branching = 1.53
  Ply: 6   Positions: 57   Avg Nodes:   19534   Branching = 1.38
  Ply: 7   Positions: 50   Avg Nodes:   26732   Branching = 1.37
  Ply: 8   Positions: 43   Avg Nodes:   36925   Branching = 1.38
  Ply: 9   Positions: 39   Avg Nodes:   50848   Branching = 1.38
  Ply:10   Positions: 32   Avg Nodes:   91416   Branching = 1.80
  Ply:11   Positions: 26   Avg Nodes:  134044   Branching = 1.47
  Ply:12   Positions: 20   Avg Nodes:  235666   Branching = 1.76
  Ply:13   Positions: 16   Avg Nodes:  372846   Branching = 1.58
  Ply:14   Positions: 12   Avg Nodes:  529047   Branching = 1.42
  Ply:15   Positions: 10   Avg Nodes:  794787   Branching = 1.50
  Ply:16   Positions:  7   Avg Nodes: 1289637   Branching = 1.62
  Ply:17   Positions:  6   Avg Nodes: 1792729   Branching = 1.39
  Ply:18   Positions:  5   Avg Nodes: 2959558   Branching = 1.65
  Ply:19   Positions:  4   Avg Nodes: 4611414   Branching = 1.56
  Ply:20   Positions:  4   Avg Nodes: 5795014   Branching = 1.26
  Ply:21   Positions:  3   Avg Nodes: 9113716   Branching = 1.57
  Ply:22   Positions:  2   Avg Nodes: 4529585   Branching = 0.50
  Ply:23   Positions:  1   Avg Nodes:16790212   Branching = 3.71

Here is the number of new solutions by depth of Komodo:


WAC_depth.jpg


Leela (42620) has a big, irrecuperable trouble with this very easy for Komodo tactical suite:

1s/position
score=98/145 [averages on correct positions: depth=4.1 time=0.07 nodes=498]

10s/poisition
score=105/145 [averages on correct positions: depth=4.4 time=0.14 nodes=1620]


In 10 seconds per position (on strong GPU), Leela fares worse than Komodo in milliseconds to depth 9. Leela misses tactical 2-3-4 mover shots quite easily. Can we say that "Leela is weak tactically"? And all in all, beats the crap out of Komodo on my PC due to "something else"? If I call this "something else" as roughly the "positional play", I am outside the usual terminology? Yes, as a patzer human player, I can hardly grasp what exactly "positional play" means, so I use engines, and for example these extreme positional/tactical test suites. If 2 years ago with usual engines, the separation tactical/positional was unclear indeed, with Leela this separation is quite extreme. So, I am unprepared now to blur the separation tactical/positional (was more prepared 2 years ago).

Tactical-positional is a falser than false dichotomy. They are both words associated with weak human players learning chess. Firstly, weak player knows about material 95331. Then he blunders around, leaves pieces en prise and learns to guard his material. Then he learns there are things called tactics, maybe a knight fork or something. From then on he plays tactics, always looking for a trick. Some few then learn there’s a bit more, called positional, like double pawn. These few then play tactics to try and get some positional advantage. That’s where it stops, also with most chess programmers. Everything is a combination of positional things and some lookahead tactics. The belief being that chess is entirely tactics with a bit of position knowledge. Eg, chess is won by tactics.
Fast forward to 2018. Whoops. Everything everybody knew was wrong. But they are not really very sure why, so they carry on babbling using language that doesn’t actually fit, tactics this, positional that, bla bla. Like Sisyphus, they want to progress up the hill, but they have this giant tactics-positional dichotomy stone that keeps rolling backwards. Worst, probably, are these 75 move deep SF lines people, who want to prove everything, but can’t.
Throw away the words, they were useful in learning, but are a handicap to understanding deeper. There is no tactics. Everything is positional. Except for beginners.

I somewhat agree on this. At patzer level chess is tactics. When u go to SGM level games are decided in endgames which are not tactics but positional based on strategy. Sometimes both are synonymous. A tactical player understand position well and a positional player never misses a tactics as he has understanding of the position. Also I can relate what KAI is saying. If we match Houdini tactical with sf dev ,sf dev will beat it most of the times. Most tactical engines have less elo. So form where this elo comes ? may b positional chess. Tactics doesnot work with them which have solid defenses which is a positional aspect of chess.

[Max]

Small update with some newer nets at different sizes.

T60 rose slowly but steadily above 2650 points. Network 59533 took with 2378 points the lead over older 128x10 experiments. And Sergio's weights with T40 data are still on top.

Code: Select all

STS	Size	Net   
------------------------------
2677	384x30 	T40-1573
2670	384x30 	T40-1705
2655	320x24 	61927 
2631	320x24 	T13-swa-330000
2585	256x20	LS 12.1
2581	256x20 	42850
2560	192x16	J20-swa-440000
2378	128x10	59533
2291	128x10	LD2
2233	 80x7	11258-se

[peter]

Thanks for the update!

[Max]

As first network, Sergio's latest 384x30-t40-1808.pb.gz scores over 2700 STS points (with nodes = 1).

I'm very curious how it translates to games.

Code: Select all

STS	Size	Net   
--------------------------
2711	384x30 	T40-1808
2686	384x30 	T60-2-1538
2662	320x24 	62000
2441	128x10  59670