stability of evaluation: stockfish versus hiarcs

[ozziejoe]

Has anybody systematically looked at the stability of prefered choice,e.g., how many time an engine changes its mind about the best move.

I've noticed that hiarcs is very stable, e.g., its first move tends to stay its first move, and this is perhaps because it uses alot of knowledge? Stockfish gets to a higher depth than hiarcs, but its evaluation bounces around a bit.

check out this position and watch it happen

[d] rn1qkbnr/pb1p1p1p/1p2p1p1/8/2Pp4/5NP1/PP2PPBP/RNBQ1RK1 w kq - 0 6

[Mike S.]

its first move tends to stay its first move

But stability is bad if the move chosen is bad. If an engine would always keep the move it chose at depth 1 ply, it would be very stable and at the same time very weak, obviously. In other words, some "instability" is required to change to a better move, now and then.

I recall old statistical results saying that the pm changes from depth x to depth x+1, in 15% or 16% of the cases.

I agree that it may look irritating if the pm changes often, in some positions or with some engines more than with others. But I assume that also has a lot to do with several moves being within a narrow margin, sometimes. (I admit I didn't try your example position now.)

[D]rn1qkbnr/pb1p1p1p/1p2p1p1/8/2Pp4/5NP1/PP2PPBP/RNBQ1RK1 w kq - 0 6

[yanquis1972]

stockfish is very 'bouncy' but this seems like an choice, here's that position about after a minute w/ SF 2.1 --

[+1.01] d=26 6.Qxd4 Nf6 7.Nc3 Nc6 8.Qh4 Be7 9.Bg5 Nh5 10.Rfd1 h6 11.Bxe7 Qxe7 12.Qe4 Nf6 13.Qe3 Rd8 14.Rac1 d6 15.Nb5 Kf8 16.Qa3 d5 17.cxd5 Qxa3 18.bxa3 Nxd5 19.e4 Nf6 20.Nd6 (0:00:56) 182221kN
[+1.05] d=25 6.Qxd4 Nf6 7.Nc3 Nc6 8.Qh4 Be7 9.Bg5 Nh5 10.Rfd1 h6 11.Bxe7 Qxe7 12.Qe4 Nf6 13.Qe3 Rd8 14.Rac1 g5 15.Nb5 d5 16.Ne5 O–O 17.cxd5 Nxe5 18.Qxe5 Bxd5 19.Rc7 Rd7 20.Bxd5 (0:00:40) 126285kN
[+0.84] d=24 6.Qxd4 Nf6 7.Nc3 Nc6 8.Qh4 Be7 9.Bg5 Nh5 10.Rfd1 h6 11.Bxe7 Qxe7 12.Qe4 Nf6 13.Qe3 Rd8 14.Rac1 g5 15.Nb5 d5 16.Ne5 O–O 17.cxd5 Nxe5 18.Qxe5 Bxd5 19.Bxd5 Rxd5 (0:00:32) 102551kN
[+0.96] d=23 6.Qxd4 Nf6 7.Nc3 Nc6 8.Qh4 Be7 9.Bg5 Nh5 10.Rfd1 h6 11.Bxe7 Qxe7 12.Nb5 Qxh4 13.Nxh4 Ke7 14.Nd6 Rab8 15.Nxf7 Kxf7 16.Rxd7+ Kf6 17.Rxb7 Rxb7 18.Bxc6 Rbb8 19.Rd1 g5 (0:00:22) 68261kN
[+1.01] d=22 6.Qxd4 Nf6 7.Nc3 Nc6 8.Qh4 Be7 9.Bg5 Nh5 10.Rad1 Bxg5 11.Nxg5 h6 12.Nge4 Qxh4 13.Nd6+ Ke7 14.gxh4 Bc8 15.e3 Nf6 16.Ncb5 Ng4 17.h3 Nf6 18.Bxc6 dxc6 19.Nc7 (0:00:18) 52582kN
[+1.05] d=21 6.Qxd4 Nf6 7.Nc3 Nc6 8.Qh4 Be7 9.Bg5 Nh5 10.Rad1 h6 11.Bxe7 Qxe7 12.Qe4 Nf6 13.Qe3 g5 14.Ne5 O–O 15.Bxc6 Bxc6 16.h4 Nh5 17.Ne4 (0:00:14) 39706kN
[+1.01] d=20 6.Qxd4 Nf6 7.Nc3 Nc6 8.Qh4 Be7 9.Bg5 Nh5 10.Rad1 h6 11.Bxe7 Qxe7 12.Qe4 f5 13.Qd3 O–O–O 14.Nb5 Kb8 15.Qc3 e5 16.Nh4 Qe6 17.Nd6 (0:00:05) 9569kN
[+1.01] d=19 6.Qxd4 Nf6 7.Nc3 Nc6 8.Qh4 Be7 9.Bg5 Nh5 10.Rad1 h6 11.Bxe7 Qxe7 12.Qe4 f5 13.Qd3 O–O–O 14.Nb5 Kb8 15.Qc3 e5 16.Nh4 Qe6 17.Nd6 (0:00:04) 7079kN
[+1.01] d=18 6.Qxd4 Nf6 7.Nc3 Nc6 8.Qh4 Be7 9.Bg5 Nh5 10.Rad1 h6 11.Bxe7 Qxe7 12.Qe4 f5 13.Qd3 O–O–O 14.Nb5 Kb8 15.Qc3 e5 16.Nh4 Qe6 17.Nd6 (0:00:03) 5550kN
[+1.09] d=17 6.Qxd4 Nf6 7.Nc3 Nc6 8.Qh4 Be7 9.Bg5 Nh5 10.Rad1 h6 11.Bxe7 Qxe7 12.Qe4 f5 13.Qd3 O–O–O 14.Nb5 Kb8 15.Qe3 d5 16.Ne5 (0:00:03) 3342kN
[+1.01] d=16 6.Qxd4 Nf6 7.Nc3 Nc6 8.Qh4 Be7 9.Bg5 Nh5 10.Rad1 h6 11.Bxe7 Qxe7 12.Qe4 f5 13.Qd3 O–O–O 14.Nb5 Kb8 15.Nfd4 Nf6 (0:00:02) 2090kN
[+0.96] d=15 6.Qxd4 Nf6 7.Nc3 Nc6 8.Qh4 Be7 9.Bg5 Nh5 10.Nb5 O–O 11.Rfd1 Bxg5 12.Qxg5 Qxg5 13.Nxg5 Nf6 14.Nd6 (0:00:02) 1386kN
[+0.84] d=14 6.Qxd4 Nf6 7.Nc3 Nc6 8.Qh4 Be7 9.Bg5 Nh5 10.Nb5 O–O 11.Bxe7 Nxe7 12.Nd6 Bxf3 13.Bxf3 Nc6 14.Qxd8 Rfxd8 15.Bxh5 gxh5 (0:00:02) 1245kN
[+1.05] d=13 6.Qxd4 Nf6 7.Nc3 Nc6 8.Qd3 Be7 9.Rd1 O–O 10.a3 Re8 11.Bh6 a6 12.Bg5 d6 13.Bxf6 Bxf6 14.Qxd6 Qxd6 15.Rxd6 Bxc3 16.bxc3 (0:00:01) 444kN
[+1.25] d=12 6.Qxd4 Nf6 7.Nc3 Nc6 8.Qd3 Bg7 9.Nb5 O–O 10.Nd6 Qb8 11.Rd1 Rd8 12.Nxb7 Qxb7 13.Ne5 Nxe5 14.Bxb7 Nxd3 15.Rxd3 (0:00:01) 230kN
[+1.25] d=11 6.Qxd4 Nf6 7.Nc3 Nc6 8.Qd3 Bg7 9.Bg5 O–O 10.Rad1 Nb4 11.Qd6 a5 12.a3 (0:00:01) 175kN
[+0.96] d=10 6.Qxd4 Nf6 7.Nc3 Nc6 8.Qh4 Be7 9.Qh6 Ng8 10.Qh3 Nf6 11.Ne5 O–O (0:00:01) 103kN
[+1.09] d=9 6.Qxd4 Nf6 7.Bg5 Be7 8.Nc3 Nc6 9.Qf4 O–O 10.Ne5 d5 11.cxd5 Nxe5 12.Qxe5 exd5 (0:00:01) 37kN
[+1.09] d=8 6.Qxd4 Nf6 7.Bg5 Be7 8.Nc3 Nc6 9.Qf4 O–O 10.Ne5 d5 11.cxd5 Nxe5 12.Qxe5 exd5 (0:00:01) 32kN
[+1.21] d=7 6.Qxd4 f6 7.Qd3 Nc6 8.a3 Ne5 9.Nxe5 Bxg2 (0:00:01) 13kN

about as stable as ive ever seen stockfish in terms of both move choice & evaluation.

[tano-urayoan]

You could read this topics:

http://www.open-chess.org/viewtopic.php?f=5&t=1042

http://www.open-chess.org/viewtopic.php?f=5&t=1234