End of Era is there: SF is finally beaten!

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
henk2
Posts: 30
Joined: Mon Jan 14, 2019 6:55 am
Full name: Henk Verbaasdonk

Re: End of Era is there: SF is finally beaten!

Post by henk2 » Fri Feb 15, 2019 2:30 pm

CMCanavessi wrote:
Fri Feb 15, 2019 2:16 pm
smatovic wrote:
Fri Feb 15, 2019 1:53 pm
CMCanavessi wrote:
Fri Feb 15, 2019 1:48 pm
smatovic wrote:
Fri Feb 15, 2019 1:12 pm
Graham Banks wrote:
Fri Feb 15, 2019 10:05 am
M ANSARI wrote:
Fri Feb 15, 2019 9:46 am
I think GPU's being used for chess will be only a temporary thing as for chess a much better solution would be to use cards that are specifically designed to do NN and nothing to do with graphics. This will probably become very common very soon and some kind of hardware equation needs to be agreed upon if AB vs NN engine matches will take place. Most likely SF and Komodo and Houdini will become hybrid engines that use both NN and AB and thus will make full use of the CPU as well as the GPU.
Wouldn't that mean that a user couldn't run any other programs that use the GPU, even if there are many cores free?
Just did a quick check on Windows 7 with AMD Radeon HD 7750,
i was able to run two Zeta v099k* instances in Winboard (ponder off) in a match,
and to play Rome Total War in 3D battle mode at the same time.

*note that currently Zeta utilizes a gpu only with about 50% but memory usage is configurable.

--
Srdja
How strong is Zeta?
est. 2000+ CCRL Elo on an highend GPU

https://www.chessprogramming.org/Zeta#N ... g_Strength

--
Srdja
Cool, I will give it a test. With my GTX 1080 (non TI), how many workers should I use? May I keep increasing the number until the nps won't go up anymore? Is there a sweet spot? Is there some auto-detection routine? Is "guessconfigx" enough?
This thread talks about the effect of workers and speed using different GPUs.

viewtopic.php?topic_view=threads&p=711259&t=63517

smatovic
Posts: 833
Joined: Wed Mar 10, 2010 9:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic
Contact:

Re: End of Era is there: SF is finally beaten!

Post by smatovic » Fri Feb 15, 2019 2:37 pm

CMCanavessi wrote:
Fri Feb 15, 2019 2:16 pm
smatovic wrote:
Fri Feb 15, 2019 1:53 pm
CMCanavessi wrote:
Fri Feb 15, 2019 1:48 pm
How strong is Zeta?
est. 2000+ CCRL Elo on an highend GPU

https://www.chessprogramming.org/Zeta#N ... g_Strength

--
Srdja
Cool, I will give it a test. With my GTX 1080 (non TI), how many workers should I use? May I keep increasing the number until the nps won't go up anymore? Is there a sweet spot? Is there some auto-detection routine? Is "guessconfigx" enough?
The GTX 1080 has 2560 cores, divided by 32 cores per SIMD-Unit this results in 80 workers.

You can try 160 workers, but this is similar to HyperThreading on CPUs, it increases total nps, but can lower nps per worker.

The guessconfigx command doubles the workers size as long there is an 1.8 speedup,
and guesses some memory config.

There is an benchsmp command implemented in Zeta to test parallel speedups,
so you can try custom settings in workers and memory size,
start the engine in command line and type these commands:

Code: Select all

new
sd 12
st 2000
benchsmp
Note from the Zeta README file:

################################################################################
### WARNING
################################################################################
It is recommended to run the engine on an discrete GPU,
without display connected,
otherwise system and display can freeze or crash during computation.

Some GPU drivers have an timeout of 5 seconds if GPU is connected to display.
So make sure to use an discrete GPU or set proper time controls or disable
the Watch Dog timeout in your driver settings.

Windows OS have an internal gpu timeout, double click the .reg file
"SetWindowsGPUTimeoutTo20s.reg"
and reboot the OS to set the timeout to 20 seconds.

AMD GPUs may have an driver specific timeout of about 360 to 3600 seconds

smatovic
Posts: 833
Joined: Wed Mar 10, 2010 9:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic
Contact:

Re: End of Era is there: SF is finally beaten!

Post by smatovic » Fri Feb 15, 2019 3:33 pm

henk2 wrote:
Fri Feb 15, 2019 2:14 pm
smatovic wrote:
Fri Feb 15, 2019 1:53 pm
est. 2000+ CCRL Elo on an highend GPU

https://www.chessprogramming.org/Zeta#N ... g_Strength

--
Srdja
I wonder how much it improved node per node from a GTX480 vs an R9 Fury X.
The Fury should be 2-3 times as fast. So I guess CCRL Elo should be 2100-2150 at least.

2500 CCRL on an RTX2080ti?
My GTX 480 resides now in silicon heaven,
and my Fury X needs an watercooling upgrade,
so i can't test it at the moment,
but about 2100 Elo for Fury X and 2300 for RTX 2080 TI could be possible.

--
Srdja

corres
Posts: 1578
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: End of Era is there: SF is finally beaten!

Post by corres » Fri Feb 15, 2019 5:24 pm

corres wrote:
Fri Feb 15, 2019 9:50 am
Graham Banks wrote:
Fri Feb 15, 2019 9:16 am
How can one know whether the engines are playing on equal terms?
How can you compare CPU and GPU and state you've found a balance?
Nohow.
There are only subjective methods to make comparison.
Moreover AB engines are made basically for playing middle games and endgames.
An AB engines will be banished from a competition if it has a built-in opening book.
An NN type engines has definitively a built-in opening + middle game + endgame book.
AB engines have their own benefit and their own drawback and NN type engines have these too.
It is obvious the development of hardware and software gives more opportunity for enhancing
the chess power of the NN type engines than for an AB type engines.
But the problem of chess can not be solved by an NN type engine either.
Note
I think the performance of Stockfish against NN engines would be enhanced if developers of Stockfish would make test not only against the actual master of Stockfish but against the best NN engines and if they would modify in some measure the source of Stockfish according to the result of tests.
Until now to make these tests was problematic because the fast changing in the power of NN engines.
But to now the chess power of NN engines reach near a plateau so there is sense to make tests with NN type engines too.

MikeB
Posts: 3391
Joined: Thu Mar 09, 2006 5:34 am
Location: Pen Argyl, Pennsylvania

Re: End of Era is there: SF is finally beaten!

Post by MikeB » Sat Feb 16, 2019 5:49 am

corres wrote:
Fri Feb 15, 2019 5:24 pm
corres wrote:
Fri Feb 15, 2019 9:50 am
Graham Banks wrote:
Fri Feb 15, 2019 9:16 am
How can one know whether the engines are playing on equal terms?
How can you compare CPU and GPU and state you've found a balance?
Nohow.
There are only subjective methods to make comparison.
Moreover AB engines are made basically for playing middle games and endgames.
An AB engines will be banished from a competition if it has a built-in opening book.
An NN type engines has definitively a built-in opening + middle game + endgame book.
AB engines have their own benefit and their own drawback and NN type engines have these too.
It is obvious the development of hardware and software gives more opportunity for enhancing
the chess power of the NN type engines than for an AB type engines.
But the problem of chess can not be solved by an NN type engine either.
Note
I think the performance of Stockfish against NN engines would be enhanced if developers of Stockfish would make test not only against the actual master of Stockfish but against the best NN engines and if they would modify in some measure the source of Stockfish according to the result of tests.
Until now to make these tests was problematic because the fast changing in the power of NN engines.
But to now the chess power of NN engines reach near a plateau so there is sense to make tests with NN type engines too.
Whoever figures out how to combine the Chess CPU power of a Stockfish like engine with the Chess GPU of an Lc0 like engine will get there first, SO many computers today have 24 , 36 or even 72 cores - take the best x moves from Lc0 and dedicate an AB search on those moves using one of those CPUs with A/B search engine like SF. Someone with some coding skills should be looking at that, Good luck!

User avatar
M ANSARI
Posts: 3408
Joined: Thu Mar 16, 2006 6:10 pm

Re: End of Era is there: SF is finally beaten!

Post by M ANSARI » Sat Feb 16, 2019 6:29 am

MikeB wrote:
Sat Feb 16, 2019 5:49 am
corres wrote:
Fri Feb 15, 2019 5:24 pm
corres wrote:
Fri Feb 15, 2019 9:50 am
Graham Banks wrote:
Fri Feb 15, 2019 9:16 am
How can one know whether the engines are playing on equal terms?
How can you compare CPU and GPU and state you've found a balance?
Nohow.
There are only subjective methods to make comparison.
Moreover AB engines are made basically for playing middle games and endgames.
An AB engines will be banished from a competition if it has a built-in opening book.
An NN type engines has definitively a built-in opening + middle game + endgame book.
AB engines have their own benefit and their own drawback and NN type engines have these too.
It is obvious the development of hardware and software gives more opportunity for enhancing
the chess power of the NN type engines than for an AB type engines.
But the problem of chess can not be solved by an NN type engine either.
Note
I think the performance of Stockfish against NN engines would be enhanced if developers of Stockfish would make test not only against the actual master of Stockfish but against the best NN engines and if they would modify in some measure the source of Stockfish according to the result of tests.
Until now to make these tests was problematic because the fast changing in the power of NN engines.
But to now the chess power of NN engines reach near a plateau so there is sense to make tests with NN type engines too.
Whoever figures out how to combine the Chess CPU power of a Stockfish like engine with the Chess GPU of an Lc0 like engine will get there first, SO many computers today have 24 , 36 or even 72 cores - take the best x moves from Lc0 and dedicate an AB search on those moves using one of those CPUs with A/B search engine like SF. Someone with some coding skills should be looking at that, Good luck!

Very true ... I think that SF even on 1000 cores would still lose to Lc0 some pretty spectacular games. I mean just look at yesterday's loss ... out of the opening and each engine plays a few moves by itself and you have this position

[d]r2q1k1r/3b1pp1/1p2p2p/p2pP2P/P1p2P2/3B4/2P1QBP1/R3K2R w KQ - 0 20

SF has reacted to Lc0 pawn attack on the Kingside by giving up a bishop temporarily to where it seemed like Lc0 would have to give it back for a couple of pawns. But here Lc0 uncorks Bg6!!! giving up the bishop without any pawn compensation.


[d]r2q1k1r/3b1pp1/1p2p1Bp/p2pP2P/P1p2P2/8/2P1QBP1/R3K2R b KQ - 0 20

It was really hard to see how this was going to work out well for Lc0 ... but the entire plan was to entomb the rook on h8 and the continuation for Lc0 was to give up a total of 3 pawn to make that happen. It somehow made it work and was even very happy to exchange queens and go for what seemed like a super slow strangulation 3 pawns down. If you want to understand why NN engines are extremely strong and very difficult for AB to handle, it is games like this. There is just no calculating through something like this.


Here is what it looked like after about 30 moves … YES … Lc0 is 3 pawns down but SF is totally helpless as the rook on h8 can't get developed!

[d]2r1b1kr/6p1/4P1Pp/1p1p1P2/p1p3p1/2P5/3K1B2/RR6 w - - 0 32


And the full game for anyone interested ... very powerful game done in A0 style !!!


henk2
Posts: 30
Joined: Mon Jan 14, 2019 6:55 am
Full name: Henk Verbaasdonk

Re: End of Era is there: SF is finally beaten!

Post by henk2 » Sat Feb 16, 2019 9:40 am

M ANSARI wrote:
Sat Feb 16, 2019 6:29 am
MikeB wrote:
Sat Feb 16, 2019 5:49 am
corres wrote:
Fri Feb 15, 2019 5:24 pm
corres wrote:
Fri Feb 15, 2019 9:50 am
Graham Banks wrote:
Fri Feb 15, 2019 9:16 am
How can one know whether the engines are playing on equal terms?
How can you compare CPU and GPU and state you've found a balance?
Nohow.
There are only subjective methods to make comparison.
Moreover AB engines are made basically for playing middle games and endgames.
An AB engines will be banished from a competition if it has a built-in opening book.
An NN type engines has definitively a built-in opening + middle game + endgame book.
AB engines have their own benefit and their own drawback and NN type engines have these too.
It is obvious the development of hardware and software gives more opportunity for enhancing
the chess power of the NN type engines than for an AB type engines.
But the problem of chess can not be solved by an NN type engine either.
Note
I think the performance of Stockfish against NN engines would be enhanced if developers of Stockfish would make test not only against the actual master of Stockfish but against the best NN engines and if they would modify in some measure the source of Stockfish according to the result of tests.
Until now to make these tests was problematic because the fast changing in the power of NN engines.
But to now the chess power of NN engines reach near a plateau so there is sense to make tests with NN type engines too.
Whoever figures out how to combine the Chess CPU power of a Stockfish like engine with the Chess GPU of an Lc0 like engine will get there first, SO many computers today have 24 , 36 or even 72 cores - take the best x moves from Lc0 and dedicate an AB search on those moves using one of those CPUs with A/B search engine like SF. Someone with some coding skills should be looking at that, Good luck!

Very true ... I think that SF even on 1000 cores would still lose to Lc0 some pretty spectacular games. I mean just look at yesterday's loss ... out of the opening and each engine plays a few moves by itself and you have this position

[d]r2q1k1r/3b1pp1/1p2p2p/p2pP2P/P1p2P2/3B4/2P1QBP1/R3K2R w KQ - 0 20

SF has reacted to Lc0 pawn attack on the Kingside by giving up a bishop temporarily to where it seemed like Lc0 would have to give it back for a couple of pawns. But here Lc0 uncorks Bg6!!! giving up the bishop without any pawn compensation.


[d]r2q1k1r/3b1pp1/1p2p1Bp/p2pP2P/P1p2P2/8/2P1QBP1/R3K2R b KQ - 0 20

It was really hard to see how this was going to work out well for Lc0 ... but the entire plan was to entomb the rook on h8 and the continuation for Lc0 was to give up a total of 3 pawn to make that happen. It somehow made it work and was even very happy to exchange queens and go for what seemed like a super slow strangulation 3 pawns down. If you want to understand why NN engines are extremely strong and very difficult for AB to handle, it is games like this. There is just no calculating through something like this.


Here is what it looked like after about 30 moves … YES … Lc0 is 3 pawns down but SF is totally helpless as the rook on h8 can't get developed!

[d]2r1b1kr/6p1/4P1Pp/1p1p1P2/p1p3p1/2P5/3K1B2/RR6 w - - 0 32


And the full game for anyone interested ... very powerful game done in A0 style !!!

A 1000 core Clusterfish may not be suffering from the same horizon effect Stock and Bluefish are.
After all, it would get 800-1000 million NPS.

A 1000 core Stockfish would be like regular stockfish getting 1450 minutes + 3 minutes per move vs Leela getting only 120 minutes + 15sec per move.
It could still lose but the ELO difference between 1000fish and Leela would be around 150-200. So Leela would be in deep trouble.

User avatar
George Tsavdaris
Posts: 1575
Joined: Thu Mar 09, 2006 11:35 am

Re: End of Era is there: SF is finally beaten!

Post by George Tsavdaris » Sat Feb 16, 2019 9:51 am

henk2 wrote:
Sat Feb 16, 2019 9:40 am
A 1000 core Clusterfish may not be suffering from the same horizon effect Stock and Bluefish are.
After all, it would get 800-1000 million NPS.

A 1000 core Stockfish would be like regular stockfish getting 1450 minutes + 3 minutes per move vs Leela getting only 120 minutes + 15sec per move.
It could still lose but the ELO difference between 1000fish and Leela would be around 150-200. So Leela would be in deep trouble.
And a 100000 core Stockfish would be even stronger so Leela would be in even bigger deep trouble!
After his son's birth they've asked him:
"Is it a boy or girl?"
YES! He replied.....

Javier Ros
Posts: 181
Joined: Fri Oct 12, 2012 10:48 am
Location: Seville (SPAIN)
Full name: Javier Ros

Re: End of Era is there: SF is finally beaten!

Post by Javier Ros » Sat Feb 16, 2019 10:18 am

M ANSARI wrote:
Sat Feb 16, 2019 6:29 am
Very true ... I think that SF even on 1000 cores would still lose to Lc0 some pretty spectacular games. I mean just look at yesterday's loss ... out of the opening and each engine plays a few moves by itself and you have this position
Thanks for the game, it is absolutely amazing!!

I am a follower of chess computers since the 80s and I must say that I never thought we would ever see such a radical revolution.
As another example of this revolution I show a game of one of my tests.
The game was played from the 8th position of Balsa_Top25 Test Suite, Sicilian Najdorf betwwen Stockfish dev 2-3-19 on i7 980X 6 cores and AntiFish1.0 on 2xGTX1060 at 90 minutes plus 30 sec time control.

Stockfish has just eaten a pawn and is evaluated with +1.30/34 26 sec.,
Image
Lc0 payed 41...Rd7 +0.46/16 and Stockfish didn't eat the second pawn at a5 after thinking 391 sec 42.Qe3 +0.16/36.
If 42.Qxa5 Stockfish gives the variant

Analysis by Stockfish 030219 64 BMI2x4:

42...Dh6 43.Dc3 Ad8 44.Cxd6 Cg3+ 45.hxg3 hxg3+ 46.Rg1 Ab6+ 47.Rf1 Dh1+ 48.Re2 Dxg2+ 49.Rd1 Ad4 50.Dd2 Dxf3+ 51.De2 Dxe2+ 52.Rxe2 Txd6 53.Rf3 Rg7 54.Rxg3 Rf6 55.Rf3 Td7 56.Tbd1 Ab2 57.a4 Aa3 58.Th1 Ac5 59.Re2 e4 60.Ta1 Ta7 61.Tad1 Ab4 62.Re3 Te8 63.Th3 Ac5+ 64.Re2 Td7 65.Rf1 Ab4 66.Ta1 Ta7 67.Td1 Ad6 68.Td2 Tg7 69.Td1 Tee7 70.Re2 Th7 71.Tdh1 Txh3 72.Txh3 Ac5 73.Rf1 Re5 74.Th6 Tg7
Las negras están mejor: -/+ (-0.94) Profundidad: 43/85 00:13:17 4052MN

while AF prefers 42...Bg5

{Lc0 v0.20.2AF10:} (42. .. Bg5 43. Qb6 Bf4 44. a4 Qf6
45. a5 Qg5 46. Qf2 Rf8 47. a6 Rh7 48. Nxd6 Bg3 49. Qe3 Bf4 50. Qf2 Ng3+ 51.
Kg1 h3 52. a7 e4 53. fxe4 hxg2 54. Ra1 Qh4 55. Qxg2 Bxd6 56. a8=Q Bc5+ 57.
Re3 Bxe3+ 58. Qf2 Bxf2+ {-1.07/16}))

After that AF delights us with a ride of the knight

Image

and finally won the game

The love relationship between a chess engine tester and his computer can be summarized in one sentence:
Until heat do us part.

User avatar
M ANSARI
Posts: 3408
Joined: Thu Mar 16, 2006 6:10 pm

Re: End of Era is there: SF is finally beaten!

Post by M ANSARI » Sat Feb 16, 2019 10:20 am

I have to admit that was very funny :D I forget sometimes that this is a geeky forum and every word is taken literally. OK so maybe 1000 core SF would have seen Bg6! and would have avoided getting its rook trapped. But pretty impressed that a 1000 core SF is 150 to 200 ELO stronger than Lc0 playing at the moment! I need to get one of those setups!

Post Reply