Extraordinary performance of Strelka 5.

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Osipov Jury
Posts: 186
Joined: Mon Jan 21, 2008 1:07 pm
Location: Russia

Re: Extraordinary performance of Strelka 5.

Post by Osipov Jury » Mon Oct 24, 2011 7:13 am

Rebel wrote:
MM wrote: Hi, i watched many games against Houdini. Well, their play is very similar but evaluation is very different, i think some important changes have been made in Strelka 5.
Score is multiplied by 4 or something like that.

Put S5 in analysis mode from the start position, you get something like 0.68
Score is not multiplied or obfuscated. It is the real internal evaluation.
I went through the Rybka code forwards and backwards and took many things.

MM
Posts: 766
Joined: Sun Oct 16, 2011 9:25 am

Re: Extraordinary performance of Strelka 5.

Post by MM » Mon Oct 24, 2011 8:00 am

Osipov Jury wrote:
Rebel wrote:
MM wrote: Hi, i watched many games against Houdini. Well, their play is very similar but evaluation is very different, i think some important changes have been made in Strelka 5.
Score is multiplied by 4 or something like that.

Put S5 in analysis mode from the start position, you get something like 0.68
Score is not multiplied or obfuscated. It is the real internal evaluation.
Hi Jury,

now your engine seems to be the closest one to houdini, do you think you are going to make some other modifications to improve it?

Thank you

Regards
MM

A Distel
Posts: 3618
Joined: Thu Dec 30, 2010 12:33 pm

Re: Extraordinary performance of Strelka 5.

Post by A Distel » Mon Oct 24, 2011 8:13 am

Houdini wrote:
Laskos wrote:http://www.talkchess.com/forum/viewtopi ... w=&start=0

Look down there on the second graph for Strelka 5. As the evaluation goes, every monotonic function is invertible.

Kai
One reason for the difference in evaluations is that Jury (purposely?) forgot to divide the output by 2 to obtain centipawns - Houdini uses 1/200 pawn units internally.

Seeing the results published on CEGT, I actually feel some sympathy for Norman's complaints about rating lists. If Strelka is readily adopted by the CEGT people, why continue to exclude Ivanhoe or Fire?

With Strelka we have an engine that is, by its author's admission, the direct result of reverse engineering Houdini 1.5. Unsurprisingly it produces identical results in move choices (see the similarity diagram) and in Elo strength (see the CEGT results).

Personally I don't mind Houdini (in whatever form) occupying all the upper ranks of the rating lists, but I'm not sure it adds to the rating list's relevance ;).

Robert

Null Move Test:

[Event "6s/Move"]

[Black "Houdini 1.5 x64"]
[Result "0-1"]
[PlyCount "24"]

Fritz 13 Uci Book.ctg
1. Na3 {0} e5 {0} 2. Nb1 {2} d5 {7} 3. Na3 {2}
Nc6 {6} 4. Nb1 {2} Nf6 {9} 5. Na3 {2} Be7 {5} 6. Nb1 {3} O-O {5} 7. Na3 {3} d4
{10} 8. Nb1 {4} e4 {9} 9. Na3 {2} Qd5 {16} 10. Nb1 {2} Ng4 {4} 11. Na3 {2} e3 {
1} 12. Nb1 {2} exf2# {0} 0-1

*******************************************************************************************

Null Move Test:

[Event "6s/Move"]

[Black "Strelka 5"]
[Result "0-1"]
[PlyCount "26"]

Fritz 13 Uci Book.ctg
1. Na3 {0} e5 {0} 2. Nb1 {2} d5 {7} 3. Na3 {1}
Nc6 {5} 4. Nb1 {2} Nf6 {8} 5. Na3 {2} Be6 {9} 6. Nb1 {2} Bc5 {11} 7. Na3 {2}
O-O {11} 8. Nb1 {2} d4 {4} 9. Na3 {2} Qe7 {5} 10. Nb1 {2} e4 {4} 11. Na3 {2}
Ng4 {5} 12. Nb1 {2} e3 {1} 13. Na3 {2} exf2# {0} 0-1

*************************************************************

A Distel
Posts: 3618
Joined: Thu Dec 30, 2010 12:33 pm

Re: Extraordinary performance of Strelka 5.

Post by A Distel » Mon Oct 24, 2011 9:22 am

Null Move Test without Book:

Event "6s/Move

Null Move Test
[Black "Strelka 5"]
[Result "0-1"]


1. Na3 {0} e5 {10} 2. Nb1 {2} d5 {5} 3. Na3 {2}
Nc6 {5} 4. Nb1 {3} Nf6 {6} 5. Na3 {2} Bc5 {7} 6. Nb1 {2} O-O {7} 7. Na3 {3} d4
{5} 8. Nb1 {2} e4 {7} 9. Na3 {2} Ng4 {5} 10. Nb1 {2} e3 {8} 11. Na3 {2} exf2# {
0} 0-1

===============================================


Event "6s/Move

Null Move Test
[Black "Houdini 1.5 x64"]
[Result "0-1"]


1. Na3 {0} e5 {10} 2. Nb1 {3} Nf6 {6} 3. Na3 {4}
Nc6 {6} 4. Nb1 {2} Bc5 {7} 5. Na3 {2} O-O {4} 6. Nb1 {2} d5 {8} 7. Na3 {2} e4 {
7} 8. Nb1 {2} Ng4 {4} 9. Na3 {2} Bxf2# {0} 0-1

User avatar
Eelco de Groot
Posts: 4279
Joined: Sun Mar 12, 2006 1:40 am
Location: Groningen

Re: Extraordinary performance of Strelka 5.

Post by Eelco de Groot » Mon Oct 24, 2011 11:48 am

Osipov Jury wrote:
Rebel wrote:
MM wrote: Hi, i watched many games against Houdini. Well, their play is very similar but evaluation is very different, i think some important changes have been made in Strelka 5.
Score is multiplied by 4 or something like that.

Put S5 in analysis mode from the start position, you get something like 0.68
Score is not multiplied or obfuscated. It is the real internal evaluation.
One reason for the difference in evaluations is that Jury (purposely?) forgot to divide the output by 2 to obtain centipawns - Houdini uses 1/200 pawn units internally.
Maybe not multiplied but forgot to divide?
With Strelka we have an engine that is, by its author's admission, the direct result of reverse engineering Houdini 1.5. Unsurprisingly it produces identical results in move choices (see the similarity diagram) and in Elo strength (see the CEGT results).
If the evaluation is different it is impossible to produce identical moves IMO, unless the evaluation is not of the endleaves. Which it isn't according to the description of your work on Strelka 3, 4 and 5 Yuri. Still I find it a bit hard to believe you would not get less similarity (to Houdini) by any system like that if it was often in use. So this is a bit of a mystery... I have not actually tried Strelka 5 so this is not an opinion based on actually seeing the moves or comparing them with Houdini. If it is not a joke, what did you think of Robert Houdart's reaction then? You don't have to reply to this of course Yuri. I just find it a bit mysterious all.

Eelco
Debugging is twice as hard as writing the code in the first
place. Therefore, if you write the code as cleverly as possible, you
are, by definition, not smart enough to debug it.
-- Brian W. Kernighan

User avatar
Rebel
Posts: 5804
Joined: Thu Aug 18, 2011 10:04 am

Re: Extraordinary performance of Strelka 5.

Post by Rebel » Mon Oct 24, 2011 3:19 pm

Osipov Jury wrote:
Rebel wrote:
MM wrote: Hi, i watched many games against Houdini. Well, their play is very similar but evaluation is very different, i think some important changes have been made in Strelka 5.
Score is multiplied by 4 or something like that.

Put S5 in analysis mode from the start position, you get something like 0.68
Score is not multiplied or obfuscated. It is the real internal evaluation.
But then the pawn value isn't 1.00 which is the general accepted way to display the score on the screen. Just saying.

Post Reply