Robbers assulting victim!

[Michael Sherwin]

Code: Select all

1: FireBird_10_beta_x64     68.5/99 
2: Igorrit_0086v9_x64       62.5/99 
3: IvanHoe_v73_x64          48.5/99
4: RobboLito_0085g3_x64     47.5/98
5: Tankist 1.2 32-bit       34.5/98
6: Stockfish-16s-x64-ja     34.5/99

296 of 1500 games played

Code: Select all

-----------------FireBird_10_beta_x64-----------------
FireBird_10_beta_x64 - Igorrit_0086v9_x64   : 10.5/19 4-2-13   55%   +35
FireBird_10_beta_x64 - IvanHoe_v73_x64      : 13.0/19 7-0-12   68%  +131
FireBird_10_beta_x64 - RobboLito_0085g3_x64 : 13.5/20 7-0-13  68%  +131
FireBird_10_beta_x64 - Stockfish-16s-x64-ja : 14.5/19 13-3-3   76%  +200
FireBird_10_beta_x64 - Tankist 1.2 32-bit   : 15.0/19 11-0-8  79%  +230
-----------------Igorrit_0086v9_x64-----------------
Igorrit_0086v9_x64 - FireBird_10_beta_x64   : 8.5/19 2-4-13   45%   -35
Igorrit_0086v9_x64 - IvanHoe_v73_x64        : 11.5/20 6-3-11   58%   +56
Igorrit_0086v9_x64 - RobboLito_0085g3_x64   : 12.0/19 6-1-12   63%   +92
Igorrit_0086v9_x64 - Stockfish-16s-x64-ja   : 15.0/19 12-1-6   79%  +230
Igorrit_0086v9_x64 - Tankist 1.2 32-bit     : 15.0/20 11-1-8   75%  +191
-----------------IvanHoe_v73_x64-----------------
IvanHoe_v73_x64 - FireBird_10_beta_x64      : 6.0/19 0-7-12   32%  -131
IvanHoe_v73_x64 - Igorrit_0086v9_x64        : 8.5/20 3-6-11   43%   -49
IvanHoe_v73_x64 - RobboLito_0085g3_x64      : 9.0/19 4-5-10  47%   -21
IvanHoe_v73_x64 - Stockfish-16s-x64-ja      : 13.0/19 9-2-8   68%  +131
IvanHoe_v73_x64 - Tankist 1.2 32-bit        : 10.0/19 3-2-14   53%   +21
-----------------RobboLito_0085g3_x64-----------------
RobboLito_0085g3_x64 - FireBird_10_beta_x64 : 6.5/20 0-7-13   33%  -123
RobboLito_0085g3_x64 - Igorrit_0086v9_x64   : 7.0/19 1-6-12   37%   -92
RobboLito_0085g3_x64 - IvanHoe_v73_x64      : 10.0/19 5-4-10   53%   +21
RobboLito_0085g3_x64 - Stockfish-16s-x64-ja : 11.5/20 7-4-9   58%   +56
RobboLito_0085g3_x64 - Tankist 1.2 32-bit   : 12.0/19 6-1-12   63%   +92
-----------------Stockfish-16s-x64-ja-----------------
Stockfish-16s-x64-ja - FireBird_10_beta_x64 : 4.5/19 3-13-3   24%  -200
Stockfish-16s-x64-ja - Igorrit_0086v9_x64   : 4.0/19 1-12-6  21%  -230
Stockfish-16s-x64-ja - IvanHoe_v73_x64      : 6.0/19 2-9-8   32%  -131
Stockfish-16s-x64-ja - RobboLito_0085g3_x64 : 8.5/20 4-7-9   43%   -49
Stockfish-16s-x64-ja - Tankist 1.2 32-bit   : 11.0/20 9-7-4   55%   +35
-----------------Tankist 1.2 32-bit-----------------
Tankist 1.2 32-bit - FireBird_10_beta_x64   : 4.0/19 0-11-8   21%  -230
Tankist 1.2 32-bit - Igorrit_0086v9_x64     : 5.0/20 1-11-8   25%  -191
Tankist 1.2 32-bit - IvanHoe_v73_x64        : 9.0/19 2-3-14   47%   -21
Tankist 1.2 32-bit - RobboLito_0085g3_x64   : 7.0/19 1-6-12   37%   -92
Tankist 1.2 32-bit - Stockfish-16s-x64-ja   : 9.0/20 7-9-4   45%   -35

[gerold]

Thanks Michael.

Looks like i will have to switch to 64 bit.

Best,
Gerold.

[Michael Sherwin]

Code: Select all

1: FireBird_10_beta_x64 82.5/122 
2: Igorrit_0086v9_x64   75.5/122 
3: RobboLito_0085g3_x64 61.0/122 
4: IvanHoe_v73_x64      60.0/122 
5: Tankist 1.2 32-bit   44.5/122 
6: Stockfish-16s-x64-ja 42.5/122 

366 of 1500 games played

Code: Select all

-----------------FireBird_10_beta_x64-----------------
FireBird_10_beta_x64 - Igorrit_0086v9_x64   : 13.5/24 5-2-17    56%   +42
FireBird_10_beta_x64 - IvanHoe_v73_x64      : 18.0/25 11-0-14   72%  +164
FireBird_10_beta_x64 - RobboLito_0085g3_x64 : 15.5/25 7-1-17    62%   +85
FireBird_10_beta_x64 - Stockfish-16s-x64-ja : 17.5/24 14-3-7    73%  +173
FireBird_10_beta_x64 - Tankist 1.2 32-bit   : 18.0/24 12-0-12   75%  +191
-----------------Igorrit_0086v9_x64-----------------
Igorrit_0086v9_x64 - FireBird_10_beta_x64   : 10.5/24 2-5-17    44%   -42
Igorrit_0086v9_x64 - IvanHoe_v73_x64        : 14.0/25 7-4-14    56%   +42
Igorrit_0086v9_x64 - RobboLito_0085g3_x64   : 14.5/24 7-2-15    60%   +70
Igorrit_0086v9_x64 - Stockfish-16s-x64-ja   : 18.0/24 13-1-10   75%  +191
Igorrit_0086v9_x64 - Tankist 1.2 32-bit     : 18.5/25 13-1-11   74%  +182
-----------------IvanHoe_v73_x64-----------------
IvanHoe_v73_x64 - FireBird_10_beta_x64      : 7.0/25 0-11-14    28%  -164
IvanHoe_v73_x64 - Igorrit_0086v9_x64        : 11.0/25 4-7-14    44%   -42
IvanHoe_v73_x64 - RobboLito_0085g3_x64      : 11.5/24 5-6-13    48%   -14
IvanHoe_v73_x64 - Stockfish-16s-x64-ja      : 17.5/24 13-2-9    73%  +173
IvanHoe_v73_x64 - Tankist 1.2 32-bit        : 13.0/24 5-3-16    54%   +28
-----------------RobboLito_0085g3_x64-----------------
RobboLito_0085g3_x64 - FireBird_10_beta_x64 : 9.5/25 1-7-17     38%   -85
RobboLito_0085g3_x64 - Igorrit_0086v9_x64   : 9.5/24 2-7-15     40%   -70
RobboLito_0085g3_x64 - IvanHoe_v73_x64      : 12.5/24 6-5-13    52%   +14
RobboLito_0085g3_x64 - Stockfish-16s-x64-ja : 14.0/25 9-6-10    56%   +42
RobboLito_0085g3_x64 - Tankist 1.2 32-bit   : 15.5/24 8-1-15    65%  +108
-----------------Stockfish-16s-x64-ja-----------------
Stockfish-16s-x64-ja - FireBird_10_beta_x64 : 6.5/24 3-14-7     27%  -173
Stockfish-16s-x64-ja - Igorrit_0086v9_x64   : 6.0/24 1-13-10    25%  -191
Stockfish-16s-x64-ja - IvanHoe_v73_x64      : 6.5/24 2-13-9     27%  -173
Stockfish-16s-x64-ja - RobboLito_0085g3_x64 : 11.0/25 6-9-10    44%   -42
Stockfish-16s-x64-ja - Tankist 1.2 32-bit   : 12.5/25 9-9-7     50%    ±0
-----------------Tankist 1.2 32-bit-----------------
Tankist 1.2 32-bit - FireBird_10_beta_x64   : 6.0/24 0-12-12    25%  -191
Tankist 1.2 32-bit - Igorrit_0086v9_x64     : 6.5/25 1-13-11    26%  -182
Tankist 1.2 32-bit - IvanHoe_v73_x64        : 11.0/24 3-5-16    46%   -28
Tankist 1.2 32-bit - RobboLito_0085g3_x64   : 8.5/24 1-8-15     35%  -108
Tankist 1.2 32-bit - Stockfish-16s-x64-ja   : 12.5/25 9-9-7     50%    ±0

[George Tsavdaris]

[quote="Michael Sherwin"]

Code: Select all

1: FireBird_10_beta_x64 82.5/122 
2: Igorrit_0086v9_x64   75.5/122 
3: RobboLito_0085g3_x64 61.0/122 
4: IvanHoe_v73_x64      60.0/122 
5: Tankist 1.2 32-bit   44.5/122 
6: Stockfish-16s-x64-ja 42.5/122 

366 of 1500 games played

Nice result for Firebird so far. I wonder why it does so well in your computer. In mine it doesn't seem so much strong. Iggorit for sure appears stronger than Firebird in my tests.
Perhaps it's the 64bit thing.

[Michael Sherwin]

Code: Select all

1: FireBird_10_beta_x64 82.5/122 
2: Igorrit_0086v9_x64   75.5/122 
3: RobboLito_0085g3_x64 61.0/122 
4: IvanHoe_v73_x64      60.0/122 
5: Tankist 1.2 32-bit   44.5/122 
6: Stockfish-16s-x64-ja 42.5/122 

366 of 1500 games played

Nice result for Firebird so far. I wonder why it does so well in your computer. In mine it doesn't seem so much strong. Iggorit for sure appears stronger than Firebird in my tests.
Perhaps it's the 64bit thing.

The results have been so consistent that I have decided to stop the tournament here as I need my computer for other things. If FireBird for 32 bits was compiled using GCC then I would say that it could be the 64 bit thing as GCC is much better at 64 bits. If there are moderate changes to FB's eval then there is the possibility that it just simply does well in the first 10+ positions in my (sherwin50.pgn) test set. Or any other guess.

Or, as Bob has demonstrated time and time again, 10's of thousands of games are needed between very similar engines to prove which is better and by how much.

[George Tsavdaris]

Code: Select all

1: FireBird_10_beta_x64 82.5/122 
2: Igorrit_0086v9_x64   75.5/122 
3: RobboLito_0085g3_x64 61.0/122 
4: IvanHoe_v73_x64      60.0/122 
5: Tankist 1.2 32-bit   44.5/122 
6: Stockfish-16s-x64-ja 42.5/122 

366 of 1500 games played

Nice result for Firebird so far. I wonder why it does so well in your computer. In mine it doesn't seem so much strong. Iggorit for sure appears stronger than Firebird in my tests.
Perhaps it's the 64bit thing.

The results have been so consistent that I have decided to stop the tournament here as I need my computer for other things. If FireBird for 32 bits was compiled using GCC then I would say that it could be the 64 bit thing as GCC is much better at 64 bits. If there are moderate changes to FB's eval then there is the possibility that it just simply does well in the first 10+ positions in my (sherwin50.pgn) test set. Or any other guess.

Yes so many parameters that one can never be sure about anything in this short time.

Or, as Bob has demonstrated time and time again, 10's of thousands of games are needed between very similar engines to prove which is better and by how much.

Yes i now have started to be on that side now.
For example i plan to play with my limited resources some thousand of games too to see the difference in the above tournament:

Time control 1'+0", 128 MB hash, all 32 bit and 1 CPU.
All games are from predefined positions that all engines will play one with black and one with white against all other.
I plan to play around 2016 games(from 504 starting positions) or even more.

Until now i have:

Code: Select all

                               1             2             3                
1   Igorrit 0.086v9_w32       *****       105.0 - 85.0   138.0 - 52.0     243.0/380  63.9%
2   FireBird 1.0 beta w32  85.0 - 105.0       *****      122.0 - 68.0     207.0/380  54.1%
3   Rybka 3 1-cpu 32-bit   52.0 - 138.0   68.0 - 122.0       *****        120.0/380  31.8%

But the interesting thing is the graph with the % performance of each engine compared to the chronological order of the games played(horizontal line has the number of games each engine played):



(Comments i wrote about the graph on Rybkaforum):

It can be seen that after 32 games for each engine, Iggorit seemed to be stronger than Firebird in the tournament conditions. From then on Firebird started a good performance and even after 144 games for each engine(total games played at this point was 144·3/2 = 216) it was leading. Rybka 3 was well below of both leaders.

So if one have played only 100 games for example it could mistakenly think Firebird is stronger than Iggorit in these conditions.
But then after about 144 games suddenly situation changes and Iggorit takes over. And retains its lead even now after 308 games with a good difference.

But in the last 80 games there should be noted a tendency of Rybka 3 to increase her % performance. So perhaps after 600 games it would even catch up with the leaders and even take the lead, although the probability for this is low of course.

The increasing performance tendency of Rybka 3 seems to have been stabilized.

But i wonder for how much and if anything abnormal or extraordinary will happen in the future?