SSDF Rating List 2013-03-23

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Rebel, chrisw

User avatar
icander
Posts: 921
Joined: Fri Mar 10, 2006 11:21 am
Location: Lulea, Sweden

Re: SSDF Rating List 2013-03-23

Post by icander »

Hiarcs 14 have a superior book, specially in Najdorf. Openingrepertoar is part of a chess entity. Human or otherwise.
Tony, SSDF
Modern Times
Posts: 3546
Joined: Thu Jun 07, 2012 11:02 pm

Re: SSDF Rating List 2013-03-23

Post by Modern Times »

icander wrote:Hiarcs 14 have a superior book, specially in Najdorf. Openingrepertoar is part of a chess entity. Human or otherwise.
Very true. But as an analysis tool, the book contributes nothing to the engine.
User avatar
geots
Posts: 4790
Joined: Sat Mar 11, 2006 12:42 am

Re: SSDF Rating List 2013-03-23

Post by geots »

icander wrote:Hiarcs 14 have a superior book, specially in Najdorf. Openingrepertoar is part of a chess entity. Human or otherwise.



I will give you the point there. I considered that, and assumed that Rybka's book, tho older, would be close in quality. Maybe not. I would not be one to argue books, because all my testing is done with each engine having the same generic book. However, stilll, that's asking a lot of a book...............

But if you do read this reply, there is something that bothers me a lot worse than that. You test Fritz 3 & 4 along with Shredder 8, which cannot generate much if any interest- then you neglect to test 4 engines like Houdini, Komodo, Stockfish and Critter. If there was a limit to the number tested, most anyone with knowledge on the subject could have quickly given you a dozen to put in an old folk's home to make room for these 4 gems.

But on reflection, I probably should not have brought it up- because it does put you in a tight spot with really no coherent answer that would make any sense at all.



gts
Modern Times
Posts: 3546
Joined: Thu Jun 07, 2012 11:02 pm

Re: SSDF Rating List 2013-03-23

Post by Modern Times »

Well maybe I'll be the only one to praise the efforts of SSDF.

I like the way they test dedicated units - and that is a huge job and quite manual. And for normal engines, they are the only ones (along with Sedat perhaps) who do it properly, by having just one engine running on each machine, and the machines connected via Auto232 or some other method. That is the ideal way, but it also means you can only run a fraction of the amount of games.

Sure, I don't like the learning for example, but that is up to them.
gerold
Posts: 10121
Joined: Thu Mar 09, 2006 12:57 am
Location: van buren,missouri

Re: SSDF Rating List 2013-03-23

Post by gerold »

Modern Times wrote:Well maybe I'll be the only one to praise the efforts of SSDF.

I like the way they test dedicated units - and that is a huge job and quite manual. And for normal engines, they are the only ones (along with Sedat perhaps) who do it properly, by having just one engine running on each machine, and the machines connected via Auto232 or some other method. That is the ideal way, but it also means you can only run a fraction of the amount of games.

Sure, I don't like the learning for example, but that is up to them.
A big plus Ray.

Most of the chess folks admire the good work SSDF does.
Not saying anything bad about the few in this forum that is off beam a little. :-)
User avatar
icander
Posts: 921
Joined: Fri Mar 10, 2006 11:21 am
Location: Lulea, Sweden

Re: SSDF Rating List 2013-03-23

Post by icander »

gerold wrote:
Modern Times wrote:Well maybe I'll be the only one to praise the efforts of SSDF.

I like the way they test dedicated units - and that is a huge job and quite manual. And for normal engines, they are the only ones (along with Sedat perhaps) who do it properly, by having just one engine running on each machine, and the machines connected via Auto232 or some other method. That is the ideal way, but it also means you can only run a fraction of the amount of games.

Sure, I don't like the learning for example, but that is up to them.
A big plus Ray.

Most of the chess folks admire the good work SSDF does.
Not saying anything bad about the few in this forum that is off beam a little. :-)
Some seem to have missed the point of testing against older engines that have played a lot of games. They have a well established rating.

And that we don't test engines without own books. Our "out of the box" policy.
Tony, SSDF
mike angel
Posts: 50
Joined: Sun Jan 30, 2011 1:08 pm

Re: SSDF Rating List 2013-03-23

Post by mike angel »

Hallo Tony

in former times ...1980 and this years ... this SSDF Rating was the one and only platform which test engines. Yes I remember these times - you were testing them all by hand.

And I am really proud after all these years that I can give a very big thank you to all those who have done that.

But now ... times are changed, the SSDF Rating List is no more what it was in yesterday.

Sorry to say that.

With all the due respect I must say there is really absolutely no sense to publish a list like that what this famous historic name and in the top 100 of the list is nearly no real top engine to find.

Please think it over.

Best to you and the SSDF
Mike
Modern Times
Posts: 3546
Joined: Thu Jun 07, 2012 11:02 pm

Re: SSDF Rating List 2013-03-23

Post by Modern Times »

icander wrote:Some seem to have missed the point of testing against older engines that have played a lot of games. They have a well established rating.

And that we don't test engines without own books. Our "out of the box" policy.
Keep up the good work, some of us appreciate it.

Critter by the way has it's own book, so that is something that could be added to your testing queue if it isn't there already.
IanO
Posts: 496
Joined: Wed Mar 08, 2006 9:45 pm
Location: Portland, OR

Re: SSDF Rating List 2013-03-23

Post by IanO »

icander wrote:
gerold wrote:
Modern Times wrote:Well maybe I'll be the only one to praise the efforts of SSDF.

I like the way they test dedicated units - and that is a huge job and quite manual. And for normal engines, they are the only ones (along with Sedat perhaps) who do it properly, by having just one engine running on each machine, and the machines connected via Auto232 or some other method. That is the ideal way, but it also means you can only run a fraction of the amount of games.

Sure, I don't like the learning for example, but that is up to them.
A big plus Ray.

Most of the chess folks admire the good work SSDF does.
Not saying anything bad about the few in this forum that is off beam a little. :-)
Some seem to have missed the point of testing against older engines that have played a lot of games. They have a well established rating.

And that we don't test engines without own books. Our "out of the box" policy.
I too would like to thank the SSDF for the decades of work that has gone into maintaining their rating list.

I also admire the choice of built-in opening books, tuned to the style of the engine. This is an enormous blind spot in other rating lists, totally at odds with human high-level chess. Rybka used to be on top with a wonderful opening book, yet here comes Hiarcs 14's book, blowing Rybka out of the water 24½-15½! No other rating list reflects this evolution of computer opening theory.

Has your organization considered dropping PC engines altogether? One of the great values of your list is covering the low- and mid-rage of ratings; dedicated computers, cellphones, and PDAs; which are ill-served by any other rating list. Maybe you could make that the focus of your rating list.
Modern Times
Posts: 3546
Joined: Thu Jun 07, 2012 11:02 pm

Re: SSDF Rating List 2013-03-23

Post by Modern Times »

IanO wrote:Rybka used to be on top with a wonderful opening book, yet here comes Hiarcs 14's book, blowing Rybka out of the water 24½-15½! No other rating list reflects this evolution of computer opening theory.
Sure, but it's Elo rating against a wide range of opponents is still below Rybka, despite Rybka being such an old programme now.