Odd Arena bug found while testing sts

[jwes]

Arena apparently occasionally adds a solution move to an epd test position. Save this position as a file, use some engine in Arena to analyze it, and look at the analyses.log file.

1r1n1rk1/3qp2p/P2p2p1/1p6/5pP1/1p3P1P/5PB1/R1QR2K1 w - - bm Bf1; c0 "Bf1=10, Qb2=7, Qc3=7, Qd2=6"; id "Open Files & Diagnonals.005";

I get

5) Bf1; c0 "Bf1=10, Qb2
Searching move: Bg2-f1, Ra1-b1
Best move (SOS 5.1 for Arena): Ra1-b1
identical moves! Found in: 00:01

I don't know where Arena gets Ra1-b1.

[swami]

Maybe it's not a bug but perhaps it's when an engine was just changing the move in the last second? (note that the test is only 10 seconds/position)

With 1 minute/ per position, I expect that you will not see this kind of move-changing till last second

[jwes]

Maybe it's not a bug but perhaps it's when an engine was just changing the move in the last second? (note that the test is only 10 seconds/position)

With 1 minute/ per position, I expect that you will not see this kind of move-changing till last second

AFICT, Searching Move is the term Arena uses for the bm field in the epd record. Best Move is the move found by the program.

[swami]

Maybe it's not a bug but perhaps it's when an engine was just changing the move in the last second? (note that the test is only 10 seconds/position)

With 1 minute/ per position, I expect that you will not see this kind of move-changing till last second

AFICT, Searching Move is the term Arena uses for the bm field in the epd record. Best Move is the move found by the program.

I'm not sure I follow, are you saying that Arena mixes up latest Searching move with best move, and occasionally doesn't add the best move from engine in a log file and eventually wrongly awards points?

[jwes]

Maybe it's not a bug but perhaps it's when an engine was just changing the move in the last second? (note that the test is only 10 seconds/position)

With 1 minute/ per position, I expect that you will not see this kind of move-changing till last second

AFICT, Searching Move is the term Arena uses for the bm field in the epd record. Best Move is the move found by the program.

I'm not sure I follow, are you saying that Arena mixes up latest Searching move with best move, and occasionally doesn't add the best move from engine in a log file and eventually wrongly awards points?

No, I am saying that Arena sometimes adds a second solution to a position, and it can happen that programs can find this second move, and be counted right, though that move is not a solution given in the epd record.

[swami]

Maybe it's not a bug but perhaps it's when an engine was just changing the move in the last second? (note that the test is only 10 seconds/position)

With 1 minute/ per position, I expect that you will not see this kind of move-changing till last second

AFICT, Searching Move is the term Arena uses for the bm field in the epd record. Best Move is the move found by the program.

I'm not sure I follow, are you saying that Arena mixes up latest Searching move with best move, and occasionally doesn't add the best move from engine in a log file and eventually wrongly awards points?

No, I am saying that Arena sometimes adds a second solution to a position, and it can happen that programs can find this second move, and be counted right, though that move is not a solution given in the epd record.

Yes, I saw that. Arena appears to be adding extra moves the most in STS 2 and STS 3 (nearly 10 positions). Not sure where the move came from?

I guess I'll have to use ChessGUI from now onwards.

Have you also tried Arena 1.1? It's an old supposedly stable version.

[David Dahlem]

Hi Swami

I'm using Arena 1.1. If you recall, i had problems with STS suites getting a different score when repeating the test. Perhaps this was what was happening to me. At any rate, i had to stop using STS for this reason.

Regards
Dave

[Matthias Gemuh]

No, I am saying that Arena sometimes adds a second solution to a position, and it can happen that programs can find this second move, and be counted right, though that move is not a solution given in the epd record.

Yes, I saw that. Arena appears to be adding extra moves the most in STS 2 and STS 3 (nearly 10 positions). Not sure where the move came from?

I guess I'll have to use ChessGUI from now onwards.

Have you also tried Arena 1.1? It's an old supposedly stable version.

Tests may take longer in ChessGUI because of its basic philosophy:
after each game or test position, engines are unloaded.
This way, there are no previously filled hash tables messing up things (infinitesimally).

Matthias.

[jwes]

No, I am saying that Arena sometimes adds a second solution to a position, and it can happen that programs can find this second move, and be counted right, though that move is not a solution given in the epd record.

Yes, I saw that. Arena appears to be adding extra moves the most in STS 2 and STS 3 (nearly 10 positions). Not sure where the move came from?

I guess I'll have to use ChessGUI from now onwards.

Have you also tried Arena 1.1? It's an old supposedly stable version.

Tests may take longer in ChessGUI because of its basic philosophy:
after each game or test position, engines are unloaded.
This way, there are no previously filled hash tables messing up things (infinitesimally).

Matthias.

Any chance of this being made an option?

[Matthias Gemuh]

No, I am saying that Arena sometimes adds a second solution to a position, and it can happen that programs can find this second move, and be counted right, though that move is not a solution given in the epd record.

Yes, I saw that. Arena appears to be adding extra moves the most in STS 2 and STS 3 (nearly 10 positions). Not sure where the move came from?

I guess I'll have to use ChessGUI from now onwards.

Have you also tried Arena 1.1? It's an old supposedly stable version.

Tests may take longer in ChessGUI because of its basic philosophy:
after each game or test position, engines are unloaded.
This way, there are no previously filled hash tables messing up things (infinitesimally).

Matthias.

Any chance of this being made an option?

Not in game play.
I may consider the option for testsuites but dread the work involved.