Home On The Range

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
Graham Banks
Posts: 35064
Joined: Sun Feb 26, 2006 9:52 am
Location: Auckland, NZ

Home On The Range

Post by Graham Banks » Wed Dec 26, 2007 7:36 pm

HOME ON THE RANGE

Dual Athlon XP2000+
Deep Shredder 11 GUI
128mb hash each
3-4-5 piece tablebases
Ponder off
Xmas2640b.bkt
40 moves in 68 minutes repeating (adapted for the CCRL)
4 cycles (44 rounds)


Participants

Parrot 07.07.22
Alf 1.09
Matheus 2.3
Mustang 4.97
Lime 62
Adam 3.1
BigLion 2.23x
OBender 3.1.0
Monarch 1.7
Marvin 1.3.0
Smash 1.0.3
Clueless 1.4

Image

Image

Image

Image

Image

Image

Image

Image

Image

Image

Image

Image


The tournament can be followed and games downloaded after every couple of rounds from here:
http://kirr.homeunix.org/chess/discussi ... f=7&t=2913
Last edited by Graham Banks on Mon Jan 07, 2008 7:46 am, edited 1 time in total.
gbanksnz at gmail.com

User avatar
Graham Banks
Posts: 35064
Joined: Sun Feb 26, 2006 9:52 am
Location: Auckland, NZ

Standings after Round 6 of 44

Post by Graham Banks » Sun Dec 30, 2007 8:44 pm

HOME ON THE RANGE

Dual Athlon XP2000+
Deep Shredder 11 GUI
128mb hash each
3-4-5 piece tablebases
Ponder off
Xmas2640b.bkt
40 moves in 68 minutes repeating (adapted for the CCRL)
4 cycles (44 rounds)


Standings after Round 6

4.5 - Alf 1.09
4.5 - BigLion 2.23x
4.0 - Monarch 1.7
3.5 - Parrot 07.07.22
3.5 - OBender 3.1.0
3.0 - Lime 62
2.5 - Matheus 2.3
2.5 - Adam 3.1
2.5 - Clueless 1.4
2.0 - Marvin 1.3.0
2.0 - Smash 1.0.3
1.5 - Mustang 4.97


The tournament can be followed and games downloaded after every couple of rounds from here:
http://kirr.homeunix.org/chess/discussi ... f=7&t=2913
Last edited by Graham Banks on Mon Jan 07, 2008 7:47 am, edited 1 time in total.
gbanksnz at gmail.com

User avatar
Dr.Wael Deeb
Posts: 9773
Joined: Wed Mar 08, 2006 7:44 pm
Location: Amman,Jordan

Re: Standings after Round 6 of 44

Post by Dr.Wael Deeb » Sun Dec 30, 2007 8:56 pm

So Smash is a weak engine after all playing around 1800 Elo and even a little bit less.... 8-)
_No one can hit as hard as life.But it ain’t about how hard you can hit.It’s about how hard you can get hit and keep moving forward.How much you can take and keep moving forward….

User avatar
Graham Banks
Posts: 35064
Joined: Sun Feb 26, 2006 9:52 am
Location: Auckland, NZ

Re: Standings after Round 6 of 44

Post by Graham Banks » Sun Dec 30, 2007 9:13 pm

Dr.Wael Deeb wrote:So Smash is a weak engine after all playing around 1800 Elo and even a little bit less.... 8-)
Not based on the 180 games or so that we have played! :P
Early days in this tourney of course.
gbanksnz at gmail.com

User avatar
Graham Banks
Posts: 35064
Joined: Sun Feb 26, 2006 9:52 am
Location: Auckland, NZ

Standings after Round 12 of 44

Post by Graham Banks » Mon Jan 07, 2008 7:49 am

HOME ON THE RANGE

Dual Athlon XP2000+
Deep Shredder 11 GUI
128mb hash each
3-4-5 piece tablebases
Ponder off
Xmas2640b.bkt
40 moves in 68 minutes repeating (adapted for the CCRL)
4 cycles (44 rounds)


Standings after Round 12

9.5 - Alf 1.09
8.5 - Monarch 1.7
8.0 - BigLion 2.23x
7.5 - Parrot 07.07.22
6.5 - Lime 62
6.0 - Adam 3.1
5.5 - Matheus 2.3
5.0 - OBender 3.1.0
5.0 - Smash 1.0.3
4.0 - Mustang 4.97
4.0 - Clueless 1.4
2.5 - Marvin 1.3.0


The tournament can be followed and games downloaded after every couple of rounds from here:
http://kirr.homeunix.org/chess/discussi ... f=7&t=2913
gbanksnz at gmail.com

User avatar
hgm
Posts: 25891
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: Standings after Round 6 of 44

Post by hgm » Mon Jan 07, 2008 8:26 am

Dr.Wael Deeb wrote:So Smash is a weak engine after all playing around 1800 Elo and even a little bit less.... 8-)
If you think you can base a meaningful rating on just 3 won games, you just don't know how to determine a rating.

It does not matter if you have a score of 3 out of 60, 3 out of 1000, 3 out of 100,000. In all cases the rating will be crap, and the probability the engine would have scored 2 or 4 points in stead of 3 hardly any lower than that it has 3. In fact the standard error in the rating would be far greater with 3 out of 100,000 games than it would be for 3 out of 10 games, as the same statistical error in score would map to a much larger uncertainty in Elo becuase of the (apparently) increased strength difference between the tested engine and its opponents.

So remember: an Elo based on N points can at most be as accurate as any Elo based on ~2N games. So for 3 points that is only 6 effective games. Most of the tail of your rating list is just based on 6 games, and all the other games were a meaningless exercise. This is why your ratings are crap.

User avatar
Dr.Wael Deeb
Posts: 9773
Joined: Wed Mar 08, 2006 7:44 pm
Location: Amman,Jordan

Re: Standings after Round 6 of 44

Post by Dr.Wael Deeb » Mon Jan 07, 2008 11:14 am

hgm wrote:
Dr.Wael Deeb wrote:So Smash is a weak engine after all playing around 1800 Elo and even a little bit less.... 8-)
If you think you can base a meaningful rating on just 3 won games, you just don't know how to determine a rating.

It does not matter if you have a score of 3 out of 60, 3 out of 1000, 3 out of 100,000. In all cases the rating will be crap, and the probability the engine would have scored 2 or 4 points in stead of 3 hardly any lower than that it has 3. In fact the standard error in the rating would be far greater with 3 out of 100,000 games than it would be for 3 out of 10 games, as the same statistical error in score would map to a much larger uncertainty in Elo becuase of the (apparently) increased strength difference between the tested engine and its opponents.

So remember: an Elo based on N points can at most be as accurate as any Elo based on ~2N games. So for 3 points that is only 6 effective games. Most of the tail of your rating list is just based on 6 games, and all the other games were a meaningless exercise. This is why your ratings are crap.
All the results of the engines under the 2000 barrier are deleted and will be restarted....
And you are right,the tail of my rating list is not accurate,as I pay attention to the higher divisions,now there is another story running for more than 5 yeaars now....
_No one can hit as hard as life.But it ain’t about how hard you can hit.It’s about how hard you can get hit and keep moving forward.How much you can take and keep moving forward….

User avatar
hgm
Posts: 25891
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: Standings after Round 6 of 44

Post by hgm » Mon Jan 07, 2008 11:37 am

Well, there is no reason to delete any results. Just supplement them with more results of the weak engines amongst each other, so that they get more wins and a score closer to 50%. The results you already have will remain helpful to position them, as a group, relative to the stronger engines in an accurate way.

User avatar
Dr.Wael Deeb
Posts: 9773
Joined: Wed Mar 08, 2006 7:44 pm
Location: Amman,Jordan

Re: Standings after Round 6 of 44

Post by Dr.Wael Deeb » Mon Jan 07, 2008 12:40 pm

hgm wrote:Well, there is no reason to delete any results. Just supplement them with more results of the weak engines amongst each other, so that they get more wins and a score closer to 50%. The results you already have will remain helpful to position them, as a group, relative to the stronger engines in an accurate way.
I see,but 6 effective games are way too small and I've started a new series of tournaments on a lower hardware as I won't use my new core 2 duo machine for engines like Cassandre,Soldat,etc....
_No one can hit as hard as life.But it ain’t about how hard you can hit.It’s about how hard you can get hit and keep moving forward.How much you can take and keep moving forward….

User avatar
Graham Banks
Posts: 35064
Joined: Sun Feb 26, 2006 9:52 am
Location: Auckland, NZ

Re: Standings after Round 6 of 44

Post by Graham Banks » Mon Jan 07, 2008 6:52 pm

hgm wrote:This is why your ratings are crap.
That's a bit harsh on Wael. :(
gbanksnz at gmail.com

Post Reply