Strelka 5.5 x64- A Double Dose of Trouble!

[geots]

Hello dear George,

Thank you...

Really impressive results by Strelka

I have no patience to test this great engine

Best,
Sedat

I am not impressed.

After reading that strelka cheated in test suites by having some hidden book then I am afraid that strelka is also cheating in games by having a hidden book.

The only reliable test is from positions that are probably not in strelka's secret book and the way to get them is to make some random moves and start the match after the random moves(for example 1.a3 Na6 2.h3 h6 3.f3 b6 4.e4 e5).

I suspect that this is the only way to stop the cheating of strelka because I am sure the strelka team had not enough memory for a book for something like 10^9 balanced positions that can happen after 8 plies.

Maybe strelka is not cheating by a hidden book but
my default assumption is that it is cheating after I read about the cheating in test suites.

Sorry,but i have a different view than you

I don't see anything wrong,in case of storing opening moves in chess engines

In my opinion,(for better engine performance) storing opening moves is quite good idea/method

Actually i can call a real 'cheating' for those new comercial engine releases,when there is no significant Elo improvement

Best,
Sedat

Sedat, I have never heard it said better!! I fully agree with you. Since when, as long as it belongs to him, is it anyone else's damn business what opening- or anything else- that an author puts in his own engine.

All this hullabaloo is strictly because Strelka 5.1 and 5.5 64bit came in here and started kicking butt.

gts

[Dr.Wael Deeb]

Hello dear George,

Thank you...

Really impressive results by Strelka

I have no patience to test this great engine

Best,
Sedat

I am not impressed.

After reading that strelka cheated in test suites by having some hidden book then I am afraid that strelka is also cheating in games by having a hidden book.

The only reliable test is from positions that are probably not in strelka's secret book and the way to get them is to make some random moves and start the match after the random moves(for example 1.a3 Na6 2.h3 h6 3.f3 b6 4.e4 e5).

I suspect that this is the only way to stop the cheating of strelka because I am sure the strelka team had not enough memory for a book for something like 10^9 balanced positions that can happen after 8 plies.

Maybe strelka is not cheating by a hidden book but
my default assumption is that it is cheating after I read about the cheating in test suites.

Sorry,but i have a different view than you

I don't see anything wrong,in case of storing opening moves in chess engines

In my opinion,(for better engine performance) storing opening moves is quite good idea/method

Actually i can call a real 'cheating' for those new comercial engine releases,when there is no significant Elo improvement

Best,
Sedat

100% agreed with you Sedat....
Dr.D

[Dr.Wael Deeb]

Hello dear George,

Thank you...

Really impressive results by Strelka

I have no patience to test this great engine

Best,
Sedat

I am not impressed.

After reading that strelka cheated in test suites by having some hidden book then I am afraid that strelka is also cheating in games by having a hidden book.

The only reliable test is from positions that are probably not in strelka's secret book and the way to get them is to make some random moves and start the match after the random moves(for example 1.a3 Na6 2.h3 h6 3.f3 b6 4.e4 e5).

I suspect that this is the only way to stop the cheating of strelka because I am sure the strelka team had not enough memory for a book for something like 10^9 balanced positions that can happen after 8 plies.

Maybe strelka is not cheating by a hidden book but
my default assumption is that it is cheating after I read about the cheating in test suites.

Sorry,but i have a different view than you

I don't see anything wrong,in case of storing opening moves in chess engines

In my opinion,(for better engine performance) storing opening moves is quite good idea/method

Actually i can call a real 'cheating' for those new comercial engine releases,when there is no significant Elo improvement

Best,
Sedat

Sedat, I have never heard it said better!! I fully agree with you. Since when, as long as it belongs to him, is it anyone else's damn business what opening- or anything else- that an author puts in his own engine.

All this hullabaloo is strictly because Strelka 5.1 and 5.5 64bit came in here and started kicking butt.

gts

Particulary Houdini's butt

[ernest]

Particulary Houdini's butt

According to Robert H., Strelka 5.1 (and thus 5.5) are heavily adapted from Houdini 1.5 (with a bug in evaluation, which makes Strelka ridiculous in game analysis), so Strelka's strength is very good for obvious reasons!

Note that in my tests (matches with Pondering), Strelka 5.5 64-bit only gets 45% against Houdini 2.0c 1Thread 64-bit

[Eelco de Groot]

Hello dear George,

Thank you...

Really impressive results by Strelka

I have no patience to test this great engine

Best,
Sedat

I am not impressed.

After reading that strelka cheated in test suites by having some hidden book then I am afraid that strelka is also cheating in games by having a hidden book.

The only reliable test is from positions that are probably not in strelka's secret book and the way to get them is to make some random moves and start the match after the random moves(for example 1.a3 Na6 2.h3 h6 3.f3 b6 4.e4 e5).

I suspect that this is the only way to stop the cheating of strelka because I am sure the strelka team had not enough memory for a book for something like 10^9 balanced positions that can happen after 8 plies.

Maybe strelka is not cheating by a hidden book but
my default assumption is that it is cheating after I read about the cheating in test suites.

Sorry,but i have a different view than you

I don't see anything wrong,in case of storing opening moves in chess engines

In my opinion,(for better engine performance) storing opening moves is quite good idea/method

Actually i can call a real 'cheating' for those new comercial engine releases,when there is no significant Elo improvement

Best,
Sedat

Sedat, I have never heard it said better!! I fully agree with you. Since when, as long as it belongs to him, is it anyone else's damn business what opening- or anything else- that an author puts in his own engine.

All this hullabaloo is strictly because Strelka 5.1 and 5.5 64bit came in here and started kicking butt.

gts

But George, I thought these were tests for the CCRL? If so I think you will have to use the Richard Vida "amputated" Strelka or somebody will have to check that the newest update does not have this database anymore. Or is this just for your own fun? If Osipov gets away with this 28000 positions in an offical test for the CCRL ratinglist, all engines should be tested again with their own book, if they have it. I would find it very strange if Strelka would be allowed such an exception, where is the logic of equal conditions for everyone then.

From the CCRL 40/40 list homepage:

CCRL 40/40 Testing Conditions

Opening book: Any generic. Examples: remis.ctg, draw.ctg, 5moves.ctg, perfect.ctg etc. Book line length has to be limited to 12 moves per side maximum. The same book should be used for all engines in the same match or tournament.

Engines with their own books should have them disabled (deleted or switched off in parameters). Engines which can't disable their own book can't participate in CCRL 40/40 testing.

Book learning: Off for all engines.

Position learning: Off for all engines.

[geots]

Hello dear George,

Thank you...

Really impressive results by Strelka

I have no patience to test this great engine

Best,
Sedat

I am not impressed.

After reading that strelka cheated in test suites by having some hidden book then I am afraid that strelka is also cheating in games by having a hidden book.

The only reliable test is from positions that are probably not in strelka's secret book and the way to get them is to make some random moves and start the match after the random moves(for example 1.a3 Na6 2.h3 h6 3.f3 b6 4.e4 e5).

I suspect that this is the only way to stop the cheating of strelka because I am sure the strelka team had not enough memory for a book for something like 10^9 balanced positions that can happen after 8 plies.

Maybe strelka is not cheating by a hidden book but
my default assumption is that it is cheating after I read about the cheating in test suites.

Sorry,but i have a different view than you

I don't see anything wrong,in case of storing opening moves in chess engines

In my opinion,(for better engine performance) storing opening moves is quite good idea/method

Actually i can call a real 'cheating' for those new comercial engine releases,when there is no significant Elo improvement

Best,
Sedat

Sedat, I have never heard it said better!! I fully agree with you. Since when, as long as it belongs to him, is it anyone else's damn business what opening- or anything else- that an author puts in his own engine.

All this hullabaloo is strictly because Strelka 5.1 and 5.5 64bit came in here and started kicking butt.

gts

But George, I thought these were tests for the CCRL? If so I think you will have to use the Richard Vida "amputated" Strelka or somebody will have to check that the newest update does not have this database anymore. Or is this just for your own fun? If Osipov gets away with this 28000 positions in an offical test for the CCRL ratinglist, all engines should be tested again with their own book, if they have it. I would find it very strange if Strelka would be allowed such an exception, where is the logic of equal conditions for everyone then.

From the CCRL 40/40 list homepage:

CCRL 40/40 Testing Conditions

Opening book: Any generic. Examples: remis.ctg, draw.ctg, 5moves.ctg, perfect.ctg etc. Book line length has to be limited to 12 moves per side maximum. The same book should be used for all engines in the same match or tournament.

Engines with their own books should have them disabled (deleted or switched off in parameters). Engines which can't disable their own book can't participate in CCRL 40/40 testing.

Book learning: Off for all engines.

Position learning: Off for all engines.

Sorry for the confusion. I was under the impression everyone knew I am no longer associated with CCRL. Yes, this is just my work- I get the pleasure out of trying to give everyone something interesting to look over.

As for an opening book in the engine in the sense you speak of- I can attest that is not possible. It is not there. I don't know what kind of database of moves might be present- but it certainly isn't an opening book.

I load it on the main board in any Fritz gui, and give it a book we cooked up which is called empty.ctg. It is actually empty. Even set at 40 moves in 3 min., you can move p e4 or d4, and it will think 6 seconds before it moves. I don't know what it does have- but it is not an opening book. That is a foolproof test.


All the best Eelco,

george

[Eelco de Groot]

Hello dear George,

Thank you...

Really impressive results by Strelka

I have no patience to test this great engine

Best,
Sedat

I am not impressed.

After reading that strelka cheated in test suites by having some hidden book then I am afraid that strelka is also cheating in games by having a hidden book.

The only reliable test is from positions that are probably not in strelka's secret book and the way to get them is to make some random moves and start the match after the random moves(for example 1.a3 Na6 2.h3 h6 3.f3 b6 4.e4 e5).

I suspect that this is the only way to stop the cheating of strelka because I am sure the strelka team had not enough memory for a book for something like 10^9 balanced positions that can happen after 8 plies.

Maybe strelka is not cheating by a hidden book but
my default assumption is that it is cheating after I read about the cheating in test suites.

Sorry,but i have a different view than you

I don't see anything wrong,in case of storing opening moves in chess engines

In my opinion,(for better engine performance) storing opening moves is quite good idea/method

Actually i can call a real 'cheating' for those new comercial engine releases,when there is no significant Elo improvement

Best,
Sedat

Sedat, I have never heard it said better!! I fully agree with you. Since when, as long as it belongs to him, is it anyone else's damn business what opening- or anything else- that an author puts in his own engine.

All this hullabaloo is strictly because Strelka 5.1 and 5.5 64bit came in here and started kicking butt.

gts

But George, I thought these were tests for the CCRL? If so I think you will have to use the Richard Vida "amputated" Strelka or somebody will have to check that the newest update does not have this database anymore. Or is this just for your own fun? If Osipov gets away with this 28000 positions in an offical test for the CCRL ratinglist, all engines should be tested again with their own book, if they have it. I would find it very strange if Strelka would be allowed such an exception, where is the logic of equal conditions for everyone then.

From the CCRL 40/40 list homepage:

CCRL 40/40 Testing Conditions

Opening book: Any generic. Examples: remis.ctg, draw.ctg, 5moves.ctg, perfect.ctg etc. Book line length has to be limited to 12 moves per side maximum. The same book should be used for all engines in the same match or tournament.

Engines with their own books should have them disabled (deleted or switched off in parameters). Engines which can't disable their own book can't participate in CCRL 40/40 testing.

Book learning: Off for all engines.

Position learning: Off for all engines.

Sorry for the confusion. I was under the impression everyone knew I am no longer associated with CCRL. Yes, this is just my work- I get the pleasure out of trying to give everyone something interesting to look over.


All the Best Eelco,

george

In that case, it is of course your own list George. I didn't know you were making your own list. I just hope in the end it is clear exactly which version has participated, where Maybe this latest version really does not have an opening book anymore. No doubt it is a strong program and as long as you know if it is the version with its own book or not... For my own testing I downloaded the engine too when I did not know yet it was more an April Fool's version I just see it as a Houdini with the same output as Houdini, but scaled a bit differently so the output is just times a certain constant factor which otherwise changes nothing and makes the output more like Stockfish scores again. But I have not really looked at the internals of Strelka like Richard Vida has, but that seems to be the view among the reverse engineers. As long as it doesn't cost as much as Houdini 2.0 and is not based on Houdini 2.0, it is not much of a problem for me.

All the best to you as well George! And thank you. I hope you enjoy your testing of all those very strong programs on your own terms ! And Strelka? At the moment it still has scored 100 % in my testing but that is just one 5' + 5'' game with a fixed Gruenfeld opening... I want my Stockfish clone to learn play the Gruenfeld just like Boris does. But it is not so easy. Even just teaching it to play the fianchetto of both bishops is very difficult. Well without such a database I mean... I have learned from Critter that I should just change the piece square tables to make any program play differently but it still does not play that game like Petrosjan did against Botwinnik

Eelco

[Dr.Wael Deeb]

Particulary Houdini's butt

According to Robert H., Strelka 5.1 (and thus 5.5) are heavily adapted from Houdini 1.5 (with a bug in evaluation, which makes Strelka ridiculous in game analysis), so Strelka's strength is very good for obvious reasons!

Note that in my tests (matches with Pondering), Strelka 5.5 64-bit only gets 45% against Houdini 2.0c 1Thread 64-bit

Robert can say whatever he wants,I don't believe him....

I do respect your observations though....
Dr.D

[geots]

Hello dear George,

Thank you...

Really impressive results by Strelka

I have no patience to test this great engine

Best,
Sedat

I am not impressed.

After reading that strelka cheated in test suites by having some hidden book then I am afraid that strelka is also cheating in games by having a hidden book.

The only reliable test is from positions that are probably not in strelka's secret book and the way to get them is to make some random moves and start the match after the random moves(for example 1.a3 Na6 2.h3 h6 3.f3 b6 4.e4 e5).

I suspect that this is the only way to stop the cheating of strelka because I am sure the strelka team had not enough memory for a book for something like 10^9 balanced positions that can happen after 8 plies.

Maybe strelka is not cheating by a hidden book but
my default assumption is that it is cheating after I read about the cheating in test suites.

Sorry,but i have a different view than you

I don't see anything wrong,in case of storing opening moves in chess engines

In my opinion,(for better engine performance) storing opening moves is quite good idea/method

Actually i can call a real 'cheating' for those new comercial engine releases,when there is no significant Elo improvement

Best,
Sedat

Sedat, I have never heard it said better!! I fully agree with you. Since when, as long as it belongs to him, is it anyone else's damn business what opening- or anything else- that an author puts in his own engine.

All this hullabaloo is strictly because Strelka 5.1 and 5.5 64bit came in here and started kicking butt.

gts

But George, I thought these were tests for the CCRL? If so I think you will have to use the Richard Vida "amputated" Strelka or somebody will have to check that the newest update does not have this database anymore. Or is this just for your own fun? If Osipov gets away with this 28000 positions in an offical test for the CCRL ratinglist, all engines should be tested again with their own book, if they have it. I would find it very strange if Strelka would be allowed such an exception, where is the logic of equal conditions for everyone then.

From the CCRL 40/40 list homepage:

CCRL 40/40 Testing Conditions

Opening book: Any generic. Examples: remis.ctg, draw.ctg, 5moves.ctg, perfect.ctg etc. Book line length has to be limited to 12 moves per side maximum. The same book should be used for all engines in the same match or tournament.

Engines with their own books should have them disabled (deleted or switched off in parameters). Engines which can't disable their own book can't participate in CCRL 40/40 testing.

Book learning: Off for all engines.

Position learning: Off for all engines.

Sorry for the confusion. I was under the impression everyone knew I am no longer associated with CCRL. Yes, this is just my work- I get the pleasure out of trying to give everyone something interesting to look over.


All the Best Eelco,

george

In that case, it is of course your own list George. I didn't know you were making your own list. I just hope in the end it is clear exactly which version has participated, where Maybe this latest version really does not have an opening book anymore. No doubt it is a strong program and as long as you know if it is the version with its own book or not... For my own testing I downloaded the engine too when I did not know yet it was more an April Fool's version I just see it as a Houdini with the same output as Houdini, but scaled a bit differently so the output is just times a certain constant factor which otherwise changes nothing and makes the output more like Stockfish scores again. But I have not really looked at the internals of Strelka like Richard Vida has, but that seems to be the view among the reverse engineers. As long as it doesn't cost as much as Houdini 2.0 and is not based on Houdini 2.0, it is not much of a problem for me.

All the best to you as well George! And thank you. I hope you enjoy your testing of all those very strong programs on your own terms ! And Strelka? At the moment it still has scored 100 % in my testing but that is just one 5' + 5'' game with a fixed Gruenfeld opening... I want my Stockfish clone to learn play the Gruenfeld just like Boris does. But it is not so easy. Even just teaching it to play the fianchetto of both bishops is very difficult. Well without such a database I mean... I have learned from Critter that I should just change the piece square tables to make any program play differently but it still does not play that game like Petrosjan did against Botwinnik

Eelco

I really am not sure exactly what Richard refers to that was evidently in Strelka 5.1. But whatever he says is there- no doubt it is. But as far as an opening book in the engine in the sense that it plays openings just is not what it is. It won't play openings unless you give it a generic book- or some book.

But that wouldn't be what Richard is referring to I would not think. There have been lots of engines with their own opening books in the engine that could not be disabled and there was no one alarmed. The only thing I have ever heard it called is a "database of moves". I'm shortly heading to get about 8 hours of sleep. If you find out any more details, I wish you would pass the info on to me.

gts

[gleperlier]

Thanks George for this test.

Gab