Score is not multiplied or obfuscated. It is the real internal evaluation.Rebel wrote:Score is multiplied by 4 or something like that.MM wrote: Hi, i watched many games against Houdini. Well, their play is very similar but evaluation is very different, i think some important changes have been made in Strelka 5.
Put S5 in analysis mode from the start position, you get something like 0.68
Extraordinary performance of Strelka 5.
Moderators: hgm, Rebel, chrisw
-
- Posts: 186
- Joined: Mon Jan 21, 2008 2:07 pm
- Location: Russia
Re: Extraordinary performance of Strelka 5.
I went through the Rybka code forwards and backwards and took many things.
-
- Posts: 766
- Joined: Sun Oct 16, 2011 11:25 am
Re: Extraordinary performance of Strelka 5.
Hi Jury,Osipov Jury wrote:Score is not multiplied or obfuscated. It is the real internal evaluation.Rebel wrote:Score is multiplied by 4 or something like that.MM wrote: Hi, i watched many games against Houdini. Well, their play is very similar but evaluation is very different, i think some important changes have been made in Strelka 5.
Put S5 in analysis mode from the start position, you get something like 0.68
now your engine seems to be the closest one to houdini, do you think you are going to make some other modifications to improve it?
Thank you
Regards
MM
-
- Posts: 3618
- Joined: Thu Dec 30, 2010 1:33 pm
Re: Extraordinary performance of Strelka 5.
Houdini wrote:One reason for the difference in evaluations is that Jury (purposely?) forgot to divide the output by 2 to obtain centipawns - Houdini uses 1/200 pawn units internally.Laskos wrote:http://www.talkchess.com/forum/viewtopi ... w=&start=0
Look down there on the second graph for Strelka 5. As the evaluation goes, every monotonic function is invertible.
Kai
Seeing the results published on CEGT, I actually feel some sympathy for Norman's complaints about rating lists. If Strelka is readily adopted by the CEGT people, why continue to exclude Ivanhoe or Fire?
With Strelka we have an engine that is, by its author's admission, the direct result of reverse engineering Houdini 1.5. Unsurprisingly it produces identical results in move choices (see the similarity diagram) and in Elo strength (see the CEGT results).
Personally I don't mind Houdini (in whatever form) occupying all the upper ranks of the rating lists, but I'm not sure it adds to the rating list's relevance .
Robert
Null Move Test:
[Event "6s/Move"]
[Black "Houdini 1.5 x64"]
[Result "0-1"]
[PlyCount "24"]
Fritz 13 Uci Book.ctg
1. Na3 {0} e5 {0} 2. Nb1 {2} d5 {7} 3. Na3 {2}
Nc6 {6} 4. Nb1 {2} Nf6 {9} 5. Na3 {2} Be7 {5} 6. Nb1 {3} O-O {5} 7. Na3 {3} d4
{10} 8. Nb1 {4} e4 {9} 9. Na3 {2} Qd5 {16} 10. Nb1 {2} Ng4 {4} 11. Na3 {2} e3 {
1} 12. Nb1 {2} exf2# {0} 0-1
*******************************************************************************************
Null Move Test:
[Event "6s/Move"]
[Black "Strelka 5"]
[Result "0-1"]
[PlyCount "26"]
Fritz 13 Uci Book.ctg
1. Na3 {0} e5 {0} 2. Nb1 {2} d5 {7} 3. Na3 {1}
Nc6 {5} 4. Nb1 {2} Nf6 {8} 5. Na3 {2} Be6 {9} 6. Nb1 {2} Bc5 {11} 7. Na3 {2}
O-O {11} 8. Nb1 {2} d4 {4} 9. Na3 {2} Qe7 {5} 10. Nb1 {2} e4 {4} 11. Na3 {2}
Ng4 {5} 12. Nb1 {2} e3 {1} 13. Na3 {2} exf2# {0} 0-1
*************************************************************
-
- Posts: 3618
- Joined: Thu Dec 30, 2010 1:33 pm
Re: Extraordinary performance of Strelka 5.
Null Move Test without Book:
Event "6s/Move
Null Move Test
[Black "Strelka 5"]
[Result "0-1"]
1. Na3 {0} e5 {10} 2. Nb1 {2} d5 {5} 3. Na3 {2}
Nc6 {5} 4. Nb1 {3} Nf6 {6} 5. Na3 {2} Bc5 {7} 6. Nb1 {2} O-O {7} 7. Na3 {3} d4
{5} 8. Nb1 {2} e4 {7} 9. Na3 {2} Ng4 {5} 10. Nb1 {2} e3 {8} 11. Na3 {2} exf2# {
0} 0-1
===============================================
Event "6s/Move
Null Move Test
[Black "Houdini 1.5 x64"]
[Result "0-1"]
1. Na3 {0} e5 {10} 2. Nb1 {3} Nf6 {6} 3. Na3 {4}
Nc6 {6} 4. Nb1 {2} Bc5 {7} 5. Na3 {2} O-O {4} 6. Nb1 {2} d5 {8} 7. Na3 {2} e4 {
7} 8. Nb1 {2} Ng4 {4} 9. Na3 {2} Bxf2# {0} 0-1
Event "6s/Move
Null Move Test
[Black "Strelka 5"]
[Result "0-1"]
1. Na3 {0} e5 {10} 2. Nb1 {2} d5 {5} 3. Na3 {2}
Nc6 {5} 4. Nb1 {3} Nf6 {6} 5. Na3 {2} Bc5 {7} 6. Nb1 {2} O-O {7} 7. Na3 {3} d4
{5} 8. Nb1 {2} e4 {7} 9. Na3 {2} Ng4 {5} 10. Nb1 {2} e3 {8} 11. Na3 {2} exf2# {
0} 0-1
===============================================
Event "6s/Move
Null Move Test
[Black "Houdini 1.5 x64"]
[Result "0-1"]
1. Na3 {0} e5 {10} 2. Nb1 {3} Nf6 {6} 3. Na3 {4}
Nc6 {6} 4. Nb1 {2} Bc5 {7} 5. Na3 {2} O-O {4} 6. Nb1 {2} d5 {8} 7. Na3 {2} e4 {
7} 8. Nb1 {2} Ng4 {4} 9. Na3 {2} Bxf2# {0} 0-1
-
- Posts: 4567
- Joined: Sun Mar 12, 2006 2:40 am
- Full name:
Re: Extraordinary performance of Strelka 5.
Osipov Jury wrote:Score is not multiplied or obfuscated. It is the real internal evaluation.Rebel wrote:Score is multiplied by 4 or something like that.MM wrote: Hi, i watched many games against Houdini. Well, their play is very similar but evaluation is very different, i think some important changes have been made in Strelka 5.
Put S5 in analysis mode from the start position, you get something like 0.68
Maybe not multiplied but forgot to divide?One reason for the difference in evaluations is that Jury (purposely?) forgot to divide the output by 2 to obtain centipawns - Houdini uses 1/200 pawn units internally.
If the evaluation is different it is impossible to produce identical moves IMO, unless the evaluation is not of the endleaves. Which it isn't according to the description of your work on Strelka 3, 4 and 5 Yuri. Still I find it a bit hard to believe you would not get less similarity (to Houdini) by any system like that if it was often in use. So this is a bit of a mystery... I have not actually tried Strelka 5 so this is not an opinion based on actually seeing the moves or comparing them with Houdini. If it is not a joke, what did you think of Robert Houdart's reaction then? You don't have to reply to this of course Yuri. I just find it a bit mysterious all.With Strelka we have an engine that is, by its author's admission, the direct result of reverse engineering Houdini 1.5. Unsurprisingly it produces identical results in move choices (see the similarity diagram) and in Elo strength (see the CEGT results).
Eelco
Debugging is twice as hard as writing the code in the first
place. Therefore, if you write the code as cleverly as possible, you
are, by definition, not smart enough to debug it.
-- Brian W. Kernighan
place. Therefore, if you write the code as cleverly as possible, you
are, by definition, not smart enough to debug it.
-- Brian W. Kernighan
-
- Posts: 6995
- Joined: Thu Aug 18, 2011 12:04 pm
Re: Extraordinary performance of Strelka 5.
But then the pawn value isn't 1.00 which is the general accepted way to display the score on the screen. Just saying.Osipov Jury wrote:Score is not multiplied or obfuscated. It is the real internal evaluation.Rebel wrote:Score is multiplied by 4 or something like that.MM wrote: Hi, i watched many games against Houdini. Well, their play is very similar but evaluation is very different, i think some important changes have been made in Strelka 5.
Put S5 in analysis mode from the start position, you get something like 0.68