Stockfish 1.3

[BBauer]

Hi,

Werner Schüle wrote:

The new Stockfish 1.3 x64 was tested on 1, 2 and 4 CPUs:
Stockfish 1.3 x64 4CPU with 2925 elos after 349 games. This is 40 points behind version 1.2default.
Stockfish 1.3 x64 2CPU with 2908 elos after 317 games. This is close to version 1.2default.
Stockfish 1.3 x64 1CPU with 2884 elos after 289 games. This is with +52 to version 1.2default a clear improvement. So we have the question: normal statistical errors or something wrong with the smp-versions?

Is he right supposing errors in the SMP Version?

Kind regards
Bernhard

[mcostalba]

Hi,

Werner Schüle wrote:

The new Stockfish 1.3 x64 was tested on 1, 2 and 4 CPUs:
Stockfish 1.3 x64 4CPU with 2925 elos after 349 games. This is 40 points behind version 1.2default.
Stockfish 1.3 x64 2CPU with 2908 elos after 317 games. This is close to version 1.2default.
Stockfish 1.3 x64 1CPU with 2884 elos after 289 games. This is with +52 to version 1.2default a clear improvement. So we have the question: normal statistical errors or something wrong with the smp-versions?

Is he right supposing errors in the SMP Version?

Kind regards
Bernhard

Hi have not touches any SMP code, but it is difficult to say that some bug is not crept in as an indirect change. Anyhow if this week results confirm this numbers I think I will have to try to understund what's happened. Unfortunaltly I don't have a SMP PC, so it is a bit difficult, but if the elo is confirmed it means there is more then 90 ELO difference bewteen single CPU and SMP, so this possible bug could be not very subtle.

[Mike S.]

I've got a great result with Stockfish 1.3 versus R232a, in bullet: 10.5-9.5.

But meanwhile, I noticed a problem in multi-pv: Unlike Glaurung, Stockfish often shows identical evals for all variations. Here, for example from the starting position and after depth 15:

1. in Fritz/1 thread:

Analysis by Stockfish 1.3 JA-si:

1. +/= (0.52): 1.e4 e5 2.Nf3 Nc6 3.Bb5 Nf6 4.0-0 Bd6 5.d4 Nxd4 6.Nxd4 exd4 7.Qxd4 a6 8.e5 axb5 9.exd6
2. +/= (0.52): 1.d4 Nf6
3. +/= (0.52): 1.Nf3 Nf6

2. in Arena/2 threads:

Stockfish 1.3:

(...)

---------------------------------------------------------------------------
15 00:20 19.694.705 956.936 +0,49 1.Nf3 Nf6
15 00:20 19.694.705 956.936 +0,49 1.d4 Nf6
15 00:20 19.694.705 956.982 +0,49 1.e4 e5 2.Nf3 Nc6 3.Bb5 Nf6 4.0-0 Bc5 5.Nc3 Qe7 6.Nd5 Nxd5 7.exd5 a6 8.Bd3 e4

[mcostalba]

I've got a great result with Stockfish 1.3 versus R232a, in bullet: 10.5-9.5.

But meanwhile, I noticed a problem in multi-pv: Unlike Glaurung, Stockfish often shows identical evals for all variations. Here, for example from the starting position and after depth 15:

1. in Fritz/1 thread:

Analysis by Stockfish 1.3 JA-si:

1. +/= (0.52): 1.e4 e5 2.Nf3 Nc6 3.Bb5 Nf6 4.0-0 Bd6 5.d4 Nxd4 6.Nxd4 exd4 7.Qxd4 a6 8.e5 axb5 9.exd6
2. +/= (0.52): 1.d4 Nf6
3. +/= (0.52): 1.Nf3 Nf6

2. in Arena/2 threads:

Stockfish 1.3:

(...)

---------------------------------------------------------------------------
15 00:20 19.694.705 956.936 +0,49 1.Nf3 Nf6
15 00:20 19.694.705 956.936 +0,49 1.d4 Nf6
15 00:20 19.694.705 956.982 +0,49 1.e4 e5 2.Nf3 Nc6 3.Bb5 Nf6 4.0-0 Bc5 5.Nc3 Qe7 6.Nd5 Nxd5 7.exd5 a6 8.Bd3 e4

Thanks for the report !

I have not touched multi-pv code too, but now I defenitly need to give a good look, not now that I am very tired and I am unable to think....

But if the SMP bug turns-out true, well, this would be a GREAT NEWS !!!

It means that Stockfish is just a bug away from a very good result...hmmmmm....too good to be true.

P.S: Please could you post bullet time control and your hardaware description...also used book would be useful. Thanks in advance.

[Eelco de Groot]

I've got a great result with Stockfish 1.3 versus R232a, in bullet: 10.5-9.5.

But meanwhile, I noticed a problem in multi-pv: Unlike Glaurung, Stockfish often shows identical evals for all variations. Here, for example from the starting position and after depth 15:

1. in Fritz/1 thread:

Analysis by Stockfish 1.3 JA-si:

1. +/= (0.52): 1.e4 e5 2.Nf3 Nc6 3.Bb5 Nf6 4.0-0 Bd6 5.d4 Nxd4 6.Nxd4 exd4 7.Qxd4 a6 8.e5 axb5 9.exd6
2. +/= (0.52): 1.d4 Nf6
3. +/= (0.52): 1.Nf3 Nf6

2. in Arena/2 threads:

Stockfish 1.3:

(...)

---------------------------------------------------------------------------
15 00:20 19.694.705 956.936 +0,49 1.Nf3 Nf6
15 00:20 19.694.705 956.936 +0,49 1.d4 Nf6
15 00:20 19.694.705 956.982 +0,49 1.e4 e5 2.Nf3 Nc6 3.Bb5 Nf6 4.0-0 Bc5 5.Nc3 Qe7 6.Nd5 Nxd5 7.exd5 a6 8.Bd3 e4

Michael and Marco, Joona already posted his fix for this behaviour in the Tournaments and Matches Forum: Stockfish v1.3.1_JA running in Fritz GUI thread This version is probably a bit faster though Is not really Multi-PV but you can't have everything!

Very strange that the multithreaded version of Stockfish 1.3.1 would seem to scale less well! Maybe it is just that the single core version is already doing so well that with more threads there is diminishing returns..

Regards
Eelco

[Mike S.]

Yes, the match was:

1m+2s, D945/3,4 GHz (~P4)
128 MB hash each, ponder off
Windows Vista 32, Arena 2.0.1
S.Canbaz' Top 25 openings
(but only 10 used --> 20 games)
3/4- and some 5-piece Nalimovs (f.Rybka)
both engines 32 bit/singlecore

Code: Select all

1   Rybka 2.3.2a   ½110½½1½01½½1½0½10½0   10.5/20  52,5%
2   Stockfish 1.3  ½001½½0½10½½0½1½01½1    9.5/20  47,5%

It was discussed in this posting (in german language):
http://forum.computerschach.de/cgi-bin/ ... l?tid=1115

In the Rybka forum, I have also added Stockfish 1.3's results in the PET endgame test to some other results. The singecore version solved more than on two cores, which seems to point to the mp problem, too.

http://rybkaforum.net/cgi-bin/rybkaforu ... ?tid=10866

[BBauer]

Hi,
for me the PET test (10sec) gave
Crafty 23.0 2proc 38/50
Stockfish1.3 1proc 35/50
Stockfish1.3 2proc 37/50

So I cannot see any shortcomings in the 2proc version of Stockfish1.3
My test is done in 64-bit Ubuntu using my own compile.

Could it be that the source code is ok but the windows compile of Stockfish1.3 is somewhat overoptimized?

Kind regards
Bernhard