ChessUSA.com TalkChess.com
Hosted by Your Move Chess & Games
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Observator bias or...
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Programming and Technical Discussions Flat
View previous topic :: View next topic  
Author Message
Uri Blass



Joined: 08 Mar 2006
Posts: 5958
Location: Tel-Aviv Israel
PostPost subject: Re: Observator bias or...    Posted: Wed May 30, 2007 10:17 am Reply to topic Reply with quote
Alessandro Scotti wrote:
I seem to have reached a plateau with Hamsters where new features and even bug fixes hardly contribute any elo to it.
This would bother me little if not for the fact that each and every test starts by feeding me the illusion of improvement, and only later falls to the same old stats.
A 700 games test tournament might go like this:
- games 0-100: 53% (ehi, not bad)
- games 101-200: 54% (great!)
- games 201-300: 54% (yo-hoo!... goes out to buy champagne!)
- games 301-400: 53% (just a little glitch!)
- games 401-500: 52% (some bad luck here...)
- games 501-600: 51% (you son of a...)
- games 601-700: 50% (nooooooooooooooo!!!)
So it's just my imagination or this happen to you too?!? One would think that 400 games already provide a good approximation, yet... Sad


I do not play so many games but one note is that I test only with fixed number of nodes because my changes do not cause significant changes in nodes per second.

If I test only with fixed number of nodes I do not need to care about problem like one engine that is slowed down by a significant factor and I can safely do other things at the same time on the same computer when I know they will not influence the result(I already saw games from people when one engine lost because of the simple fact that it was slowed down by a significant factor).

unfortunately unlike you I have not enough time to run 700 games and I am often happy with 200 games to decide which version to accept when hopefully I will accept more often the better version so I will have more productive changes than counterproductive changes and I will get an improvement.

Uri
Back to top
View user's profile Send private message
Display posts from previous:   
Subject Author Date/Time
Observator bias or... Alessandro Scotti Tue May 29, 2007 6:25 pm
      Re: Observator bias or... Dann Corbit Tue May 29, 2007 6:33 pm
      Re: Observator bias or... ed Tue May 29, 2007 6:59 pm
            Re: Observator bias or... H.G.Muller Wed May 30, 2007 9:43 am
                  Re: Observator bias or... ed Wed May 30, 2007 3:02 pm
                        Re: Observator bias or... cwb Wed May 30, 2007 4:48 pm
                        Re: Observator bias or... Peter Fendrich Wed May 30, 2007 6:36 pm
                        Re: Observator bias or... Robert Hyatt Sun Jun 10, 2007 3:30 pm
      Re: Observator bias or... Uri Blass Wed May 30, 2007 10:17 am
            Re: Observator bias or... Alessandro Scotti Wed May 30, 2007 12:35 pm
                  Re: Observator bias or... Robert Hyatt Sat Jun 02, 2007 6:41 am
                        Re: Observator bias or... H.G.Muller Sat Jun 02, 2007 10:18 am
                              Re: Observator bias or... Robert Hyatt Sun Jun 03, 2007 12:54 am
                                    Re: Observator bias or... Uri Blass Sun Jun 03, 2007 5:44 am
                                          Re: Observator bias or... Robert Hyatt Sun Jun 03, 2007 8:34 pm
                                                Re: Observator bias or... H.G.Muller Mon Jun 04, 2007 9:52 am
                                                      Re: Observator bias or... Robert Hyatt Mon Jun 04, 2007 6:58 pm
                                                            Re: Observator bias or... H.G.Muller Tue Jun 05, 2007 2:24 pm
                                                                  Re: Observator bias or... Robert Hyatt Wed Jun 06, 2007 1:31 am
                                                                        Re: Observator bias or... Uri Blass Wed Jun 06, 2007 6:42 am
                                                                              Re: Observator bias or... Robert Hyatt Thu Jun 07, 2007 2:18 am
                                                                        Re: Observator bias or... H.G.Muller Thu Jun 07, 2007 2:20 pm
                                                                              Re: Observator bias or... Robert Hyatt Fri Jun 08, 2007 3:31 am
                                                                                    time for some real data Robert Hyatt Fri Jun 08, 2007 4:02 pm
                                                                                          Re: time for some real data H.G.Muller Fri Jun 08, 2007 4:51 pm
                                                                                          Re: time for some real data Robert Hyatt Sat Jun 09, 2007 1:43 am
                                    Re: Observator bias or... H.G.Muller Fri Jun 08, 2007 9:40 am
                                          Re: Observator bias or... Robert Hyatt Sun Jun 10, 2007 2:24 am
                              Re: Observator bias or... Charles Roberson Wed Jun 06, 2007 2:44 am
                                    Re: Observator bias or... Uri Blass Wed Jun 06, 2007 6:46 am
      Re: Observator bias or... Ron Murawski Wed May 30, 2007 8:26 pm
            Re: Observator bias or... Alessandro Scotti Wed May 30, 2007 8:31 pm
                  Re: Observator bias or... ed Wed May 30, 2007 11:50 pm
                        Re: Observator bias or... Dann Corbit Thu May 31, 2007 12:19 am
                              Re: Observator bias or... Dann Corbit Thu May 31, 2007 12:33 am
                                    Re: Observator bias or... Dann Corbit Thu May 31, 2007 12:40 am
                                          Re: Observator bias or... ed Thu May 31, 2007 9:40 am
                  Re: Observator bias or... H.G.Muller Thu May 31, 2007 11:02 am
                        Re: Observator bias or... Tony Thu May 31, 2007 12:04 pm
                              Re: Observator bias or... Uri Blass Thu May 31, 2007 12:51 pm
                                    Re: Observator bias or... Tony Thu May 31, 2007 12:55 pm
                        Re: Observator bias or... Alessandro Scotti Thu May 31, 2007 12:56 pm
                        Re: Observator bias or... Robert Hyatt Sat Jun 02, 2007 6:37 am
                        Re: Observator bias or... Eelco de Groot Sat Jun 02, 2007 11:15 pm
      Re: Observator bias or... Michael Sherwin Sun Jun 03, 2007 6:29 am
            Re: Observator bias or... Uri Blass Sun Jun 03, 2007 8:11 am
                  Re: Observator bias or... Eelco de Groot Sun Jun 03, 2007 9:07 am
                        Re: Observator bias or... Uri Blass Sun Jun 03, 2007 9:39 am
                        Re: Observator bias or... H.G.Muller Sun Jun 03, 2007 9:47 am
            Re: Observator bias or... Alessandro Scotti Sun Jun 03, 2007 8:36 am
                  Re: Observator bias or... Ron Murawski Sun Jun 03, 2007 5:50 pm
      Re: Observator bias or... MartinBryant Sun Jun 03, 2007 9:07 am
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Programming and Technical Discussions

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum




Powered by phpBB © 2001, 2005 phpBB Group
Enhanced with Moby Threads