SSDF Rating List 2013-03-23

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
icander
Posts: 921
Joined: Fri Mar 10, 2006 10:21 am
Location: Lulea, Sweden
Contact:

Re: SSDF Rating List 2013-03-23

Post by icander » Thu Apr 04, 2013 4:25 pm

Hiarcs 14 have a superior book, specially in Najdorf. Openingrepertoar is part of a chess entity. Human or otherwise.
Tony, SSDF

Modern Times
Posts: 2711
Joined: Thu Jun 07, 2012 9:02 pm

Re: SSDF Rating List 2013-03-23

Post by Modern Times » Thu Apr 04, 2013 6:29 pm

icander wrote:Hiarcs 14 have a superior book, specially in Najdorf. Openingrepertoar is part of a chess entity. Human or otherwise.
Very true. But as an analysis tool, the book contributes nothing to the engine.

User avatar
geots
Posts: 4790
Joined: Fri Mar 10, 2006 11:42 pm

Re: SSDF Rating List 2013-03-23

Post by geots » Thu Apr 04, 2013 6:34 pm

icander wrote:Hiarcs 14 have a superior book, specially in Najdorf. Openingrepertoar is part of a chess entity. Human or otherwise.



I will give you the point there. I considered that, and assumed that Rybka's book, tho older, would be close in quality. Maybe not. I would not be one to argue books, because all my testing is done with each engine having the same generic book. However, stilll, that's asking a lot of a book...............

But if you do read this reply, there is something that bothers me a lot worse than that. You test Fritz 3 & 4 along with Shredder 8, which cannot generate much if any interest- then you neglect to test 4 engines like Houdini, Komodo, Stockfish and Critter. If there was a limit to the number tested, most anyone with knowledge on the subject could have quickly given you a dozen to put in an old folk's home to make room for these 4 gems.

But on reflection, I probably should not have brought it up- because it does put you in a tight spot with really no coherent answer that would make any sense at all.



gts

Modern Times
Posts: 2711
Joined: Thu Jun 07, 2012 9:02 pm

Re: SSDF Rating List 2013-03-23

Post by Modern Times » Thu Apr 04, 2013 7:04 pm

Well maybe I'll be the only one to praise the efforts of SSDF.

I like the way they test dedicated units - and that is a huge job and quite manual. And for normal engines, they are the only ones (along with Sedat perhaps) who do it properly, by having just one engine running on each machine, and the machines connected via Auto232 or some other method. That is the ideal way, but it also means you can only run a fraction of the amount of games.

Sure, I don't like the learning for example, but that is up to them.

gerold
Posts: 10121
Joined: Wed Mar 08, 2006 11:57 pm
Location: van buren,missouri

Re: SSDF Rating List 2013-03-23

Post by gerold » Fri Apr 05, 2013 1:28 am

Modern Times wrote:Well maybe I'll be the only one to praise the efforts of SSDF.

I like the way they test dedicated units - and that is a huge job and quite manual. And for normal engines, they are the only ones (along with Sedat perhaps) who do it properly, by having just one engine running on each machine, and the machines connected via Auto232 or some other method. That is the ideal way, but it also means you can only run a fraction of the amount of games.

Sure, I don't like the learning for example, but that is up to them.
A big plus Ray.

Most of the chess folks admire the good work SSDF does.
Not saying anything bad about the few in this forum that is off beam a little. :-)

User avatar
icander
Posts: 921
Joined: Fri Mar 10, 2006 10:21 am
Location: Lulea, Sweden
Contact:

Re: SSDF Rating List 2013-03-23

Post by icander » Fri Apr 05, 2013 4:32 pm

gerold wrote:
Modern Times wrote:Well maybe I'll be the only one to praise the efforts of SSDF.

I like the way they test dedicated units - and that is a huge job and quite manual. And for normal engines, they are the only ones (along with Sedat perhaps) who do it properly, by having just one engine running on each machine, and the machines connected via Auto232 or some other method. That is the ideal way, but it also means you can only run a fraction of the amount of games.

Sure, I don't like the learning for example, but that is up to them.
A big plus Ray.

Most of the chess folks admire the good work SSDF does.
Not saying anything bad about the few in this forum that is off beam a little. :-)
Some seem to have missed the point of testing against older engines that have played a lot of games. They have a well established rating.

And that we don't test engines without own books. Our "out of the box" policy.
Tony, SSDF

mike angel
Posts: 50
Joined: Sun Jan 30, 2011 12:08 pm

Re: SSDF Rating List 2013-03-23

Post by mike angel » Fri Apr 05, 2013 5:36 pm

Hallo Tony

in former times ...1980 and this years ... this SSDF Rating was the one and only platform which test engines. Yes I remember these times - you were testing them all by hand.

And I am really proud after all these years that I can give a very big thank you to all those who have done that.

But now ... times are changed, the SSDF Rating List is no more what it was in yesterday.

Sorry to say that.

With all the due respect I must say there is really absolutely no sense to publish a list like that what this famous historic name and in the top 100 of the list is nearly no real top engine to find.

Please think it over.

Best to you and the SSDF
Mike

Modern Times
Posts: 2711
Joined: Thu Jun 07, 2012 9:02 pm

Re: SSDF Rating List 2013-03-23

Post by Modern Times » Fri Apr 05, 2013 5:46 pm

icander wrote:Some seem to have missed the point of testing against older engines that have played a lot of games. They have a well established rating.

And that we don't test engines without own books. Our "out of the box" policy.
Keep up the good work, some of us appreciate it.

Critter by the way has it's own book, so that is something that could be added to your testing queue if it isn't there already.

IanO
Posts: 487
Joined: Wed Mar 08, 2006 8:45 pm
Location: Portland, OR
Contact:

Re: SSDF Rating List 2013-03-23

Post by IanO » Fri Apr 05, 2013 6:49 pm

icander wrote:
gerold wrote:
Modern Times wrote:Well maybe I'll be the only one to praise the efforts of SSDF.

I like the way they test dedicated units - and that is a huge job and quite manual. And for normal engines, they are the only ones (along with Sedat perhaps) who do it properly, by having just one engine running on each machine, and the machines connected via Auto232 or some other method. That is the ideal way, but it also means you can only run a fraction of the amount of games.

Sure, I don't like the learning for example, but that is up to them.
A big plus Ray.

Most of the chess folks admire the good work SSDF does.
Not saying anything bad about the few in this forum that is off beam a little. :-)
Some seem to have missed the point of testing against older engines that have played a lot of games. They have a well established rating.

And that we don't test engines without own books. Our "out of the box" policy.
I too would like to thank the SSDF for the decades of work that has gone into maintaining their rating list.

I also admire the choice of built-in opening books, tuned to the style of the engine. This is an enormous blind spot in other rating lists, totally at odds with human high-level chess. Rybka used to be on top with a wonderful opening book, yet here comes Hiarcs 14's book, blowing Rybka out of the water 24½-15½! No other rating list reflects this evolution of computer opening theory.

Has your organization considered dropping PC engines altogether? One of the great values of your list is covering the low- and mid-rage of ratings; dedicated computers, cellphones, and PDAs; which are ill-served by any other rating list. Maybe you could make that the focus of your rating list.

Modern Times
Posts: 2711
Joined: Thu Jun 07, 2012 9:02 pm

Re: SSDF Rating List 2013-03-23

Post by Modern Times » Fri Apr 05, 2013 7:30 pm

IanO wrote:Rybka used to be on top with a wonderful opening book, yet here comes Hiarcs 14's book, blowing Rybka out of the water 24½-15½! No other rating list reflects this evolution of computer opening theory.
Sure, but it's Elo rating against a wide range of opponents is still below Rybka, despite Rybka being such an old programme now.

Post Reply