First test with Firebird 1.2 not very promising

[beram]

First test with Firebird 1.2 not very promising

Rybka 3 at 4m2sec still beaten, but less convincingly

Dualcore T4300 2100MHZ, Blitz 4m+2s Nunn2 50 games testmatch

1 FireBird 1.2 x64 +17/=22/-11 56.00% 28.0/50
2 Rybka 3 +11/=22/-17 44.00% 22.0/50


Dualcore T4300 2100MHZ, Blitz 4m+2s Nunn2 50 games testmatch

1 FireBird 1.0 beta x64 +24/=19/-7 67.00% 33.5/50
2 Rybka 3 +7/=19/-24 33.00% 16.5/50

While watching some of the games though, I noticed as with earlier versions of Firebird again better evaluation of unbalanced material positions and also better endgame play with both sides Queen with pawns.

Regards Bram

[Xaake]

That is interesting, I tried to find Firebird 1.1 to test against Firebird 1.2 but didn't find it anywhere, I had a beta of 1.0 though and tried it against 1.2 and 1.2 won 80% of the games at 1 min games.

A note though is that I have no idea how I should set up stuff to get it fair. I basically just added Firebird 1.0 beta and Firebird 1.2 into Arena GUI (no opening books, default settings etc.) and started a tournament with 20 games at 1min. The computer I used was a non SSE2 capable 32bit machine, so that may affect the result too.

Anyway, I am not putting too much trust in my tests, but am very interested to see how other peoples tests work out.

[beram]

That is interesting, I tried to find Firebird 1.1 to test against Firebird 1.2 but didn't find it anywhere, I had a beta of 1.0 though and tried it against 1.2 and 1.2 won 80% of the games at 1 min games.

A note though is that I have no idea how I should set up stuff to get it fair. I basically just added Firebird 1.0 beta and Firebird 1.2 into Arena GUI (no opening books, default settings etc.) and started a tournament with 20 games at 1min. The computer I used was a non SSE2 capable 32bit machine, so that may affect the result too.

Anyway, I am not putting too much trust in my tests, but am very interested to see how other peoples tests work out.

Best you can Play engine test match under Fritz gui or with a restricted book. For instance Nunn match (2x 10 games) or Nunn2 (2x 25 games) where each engine playes one openingsposition with white and one with black. 1m games are perhaps too short to draw conclusions, but with 5 or 10 m games or 4m2sec and playing a lot of games you can get a real picture of underlying strength.

good luck Bram

[beram]

at longer time contro 4/40,4/40,4 far more draws and Firebird leading 19-16

Dualcore T4300 2100MHZ, 4m/40+4m/40+4m Nunn2 testmatch after 35 of the 50 games


1 FireBird 1.2 x64 +7/=24/-4 54.29% 19.0/35
2 Rybka 3 +4/=24/-7 45.71% 16.0/35

[WuShock]

FB 1.2 x64 = 45 , +22 , =46 , -7
Rybka 3 x64 = 30

4'2" / i7@ 3.75 / 3 cores ea / 512 MB / ponder off / R3 contempt=0 /

klo openings / Robbo TripleBases / R3 no egtb /

[beram]

FB 1.2 x64 = 45 , +22 , =46 , -7
Rybka 3 x64 = 30

4'2" / i7@ 3.75 / 3 cores ea / 512 MB / ponder off / R3 contempt=0 /

klo openings / Robbo TripleBases / R3 no egtb /

That is a good result of 60 % by Firebird 1.2 on a very fast system

Have you also tested at longer timecontrols ?
Just curious whether Firebird holds his superiority than

Thx Bram

[beram]

Complete results at 4/40.
Firebird wins 29-21, so remarkably a better result for Firebird 1.2 at the longer timecontrol

Dualcore T4300 2100MHZ, 4m/40+4m/40+4m Nunn2 testmatch

1 FireBird 1.2 x64 +15/=28/-7 58.00% 29.0/50
2 Rybka 3 +7/=28/-15 42.00% 21.0/50

[Xaake]

Results for Firebird 1.1 vs 1.2 using arena:

50 games, first 40 moves 10 minutes, rest 2 minutes

FB1.2 25.5
FB1.1 24.5

Same opening book (mainbook_7moves.abk)
Cache 256MB

I think I got it right this time since the score is as close as to be within the margin of error. 1.1 and 1.2 seems to have the same strength.