ChessUSA.com TalkChess.com
Hosted by Your Move Chess & Games
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Comparative nodes per second
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Programming and Technical Discussions Flat
View previous topic :: View next topic  
Author Message
Don Dailey



Joined: 29 Apr 2008
Posts: 4323
PostPost subject: Re: Comparative nodes per second    Posted: Fri Apr 20, 2012 12:31 pm Reply to topic Reply with quote
mcostalba wrote:
bob wrote:
Someone has previously suggested, although I have not given it much thought, that I could produce a pretty good eval -> winning percentage formula on my cluster stuff.


This eval <-> winning percentage stuff is really misleading. I don't know from where it came out but a lot of people (normally chess players more then programmers) blindly believes in this fetish. Eval score has no meaning taken as an absolute single number. It has a sense only considered relative to another eval score from another position.



P.S. It's my opinion that the biggest problem in computer chess is the fact that even though programs are good at comparing similar positions, they are much less capable of comparing positions that are significantly different and even worse at comparing positions that are unbalanced. This is another way of saying that they do not have very good evaluation functions. So really a single number should be all that is required because an evaluation function should be transitive. In chess programs they are not, simply because chess programs are broken with respect to evaluation.

With a properly transitive evaluation function you should be able to compare any 2 positions, even if they are totally different, and determine which one is "best." But in practice that doesn't work very well. If you have a choice of two ways to proceed that lead to 2 completely different kinds of positions your chess program probably doesn't have a clue which is better - unless one is significantly better than the other.

Part of the reason I like the logistic function is that it imposes a definite meaning to a score, or at least it attempts to. What does a pawn up really mean? You are never really a pawn up, just giving up a pawn usually gives you some advantage, even if it's not enough. If you watch 2 master player and one is pawn up or down after just a few moves, he probably has (at least some) compensation. Thus you sometimes hear the phrase, "a pawn down but with compensation" or "equal chances." But as chess programs have improved over the years you will notice that they are not quite as materialistic as they used to be. Komodo and most other good programs when given a gambit position to analyse will return a score fairly close to zero. So at least we are starting to think more in terms of positional chess and not head count.

Very often you will find that 2 chess programs have different "scales" when it comes to scoring positions, one tends to be more aggressive about scoring than the other. Where Komodo thinks 25 centipawns, Stockfish thinks 40 or 50 centipawns for example. We could standardize the meaning of 1 centi-pawn for each program by applying a simple calibration function - a few hundred test games could easily do this.

What I suggest however is not just something superficial, it should really be the way we think about an evaluation score. For example, if your program says you are up 100 centi-pawns does that mean the same thing in the ending as in the opening? If not, you have a source of error and your program will be lousy at comparing these two positions, thinking they are they equally good when they are not.

Quote:



So it is just a part of a pair use to compare positions but what counts for the engine search is the pair.

Don said instead an interesting thing, that the same eval is not the same if returned upon searching the leafs or high in the tree.

_________________
"Your superior intellect is no match for our puny weapons." -Kang and Kodos
Back to top
View user's profile Send private message Send e-mail
Display posts from previous:   
Subject Author Date/Time
Comparative nodes per second Larry Kaufman Tue Apr 10, 2012 7:49 pm
      Re: Comparative nodes per second Vincent Diepeveen Tue Apr 10, 2012 8:16 pm
      Re: Comparative nodes per second Marco Costalba Tue Apr 10, 2012 8:41 pm
            Re: Comparative nodes per second Larry Kaufman Tue Apr 10, 2012 9:02 pm
                  Re: Comparative nodes per second Thomas Petzke Tue Apr 10, 2012 9:30 pm
            Re: Comparative nodes per second Richard Vida Tue Apr 10, 2012 10:15 pm
                  Re: Comparative nodes per second Larry Kaufman Tue Apr 10, 2012 10:18 pm
                        Re: Comparative nodes per second Sam Hamilton Tue Apr 10, 2012 10:54 pm
                        Re: Comparative nodes per second Richard Vida Tue Apr 10, 2012 11:28 pm
                              Re: Comparative nodes per second Larry Kaufman Wed Apr 11, 2012 4:52 am
            Re: Comparative nodes per second Don Dailey Wed Apr 11, 2012 11:23 pm
                  Re: Comparative nodes per second Don Dailey Wed Apr 11, 2012 11:50 pm
                        Re: Comparative nodes per second Sam Hamilton Thu Apr 12, 2012 12:07 am
      Re: Comparative nodes per second Robert Hyatt Tue Apr 10, 2012 9:31 pm
            Re: Comparative nodes per second Larry Kaufman Tue Apr 10, 2012 9:35 pm
                  Re: Comparative nodes per second Robert Hyatt Thu Apr 12, 2012 1:10 am
      Re: Comparative nodes per second Daniel Shawul Tue Apr 10, 2012 11:41 pm
            Re: Comparative nodes per second Larry Kaufman Wed Apr 11, 2012 5:05 am
                  Re: Comparative nodes per second Sam Hamilton Wed Apr 11, 2012 5:28 am
                        Re: Comparative nodes per second Marco Costalba Wed Apr 11, 2012 5:58 am
                              Re: Comparative nodes per second Larry Kaufman Wed Apr 11, 2012 1:04 pm
                  Re: Comparative nodes per second Daniel Shawul Wed Apr 11, 2012 12:27 pm
      Re: Comparative nodes per second Jon Dart Wed Apr 11, 2012 1:59 pm
            Re: Comparative nodes per second Sam Hamilton Wed Apr 11, 2012 2:25 pm
                  Re: Comparative nodes per second Larry Kaufman Wed Apr 11, 2012 2:36 pm
                        Re: Comparative nodes per second Jon Dart Wed Apr 11, 2012 3:06 pm
            Re: Comparative nodes per second Larry Kaufman Wed Apr 11, 2012 2:28 pm
                  Re: Comparative nodes per second Jon Dart Wed Apr 11, 2012 3:10 pm
                        Re: Comparative nodes per second Larry Kaufman Wed Apr 11, 2012 3:59 pm
                              Re: Comparative nodes per second Robert Hyatt Thu Apr 12, 2012 1:19 am
                  Re: Comparative nodes per second Robert Hyatt Thu Apr 12, 2012 1:15 am
                        Re: Comparative nodes per second Larry Kaufman Thu Apr 12, 2012 1:35 am
                              Re: Comparative nodes per second Sam Hamilton Thu Apr 12, 2012 2:32 am
                                    Re: Comparative nodes per second Larry Kaufman Thu Apr 12, 2012 2:50 am
                                          Re: Comparative nodes per second Sam Hamilton Thu Apr 12, 2012 3:10 am
                                                Re: Comparative nodes per second Larry Kaufman Thu Apr 12, 2012 3:35 am
                              Re: Comparative nodes per second Jon Dart Thu Apr 12, 2012 2:37 pm
                                    Re: Comparative nodes per second Larry Kaufman Thu Apr 12, 2012 3:24 pm
                                          Re: Comparative nodes per second Jon Dart Thu Apr 12, 2012 4:25 pm
                                                Re: Comparative nodes per second Don Dailey Thu Apr 12, 2012 4:33 pm
                                                      Re: Comparative nodes per second Jon Dart Thu Apr 12, 2012 4:41 pm
                                                      Re: Comparative nodes per second Robert Hyatt Thu Apr 12, 2012 8:13 pm
                                                            Re: Comparative nodes per second Robert Hyatt Thu Apr 12, 2012 9:00 pm
                                                            Re: Comparative nodes per second Daniel Homan Thu Apr 12, 2012 9:52 pm
                                                                  Re: Comparative nodes per second Robert Hyatt Thu Apr 12, 2012 11:26 pm
                                                                        Re: Comparative nodes per second Larry Kaufman Thu Apr 12, 2012 11:40 pm
                                                                              Re: Comparative nodes per second Sam Hamilton Thu Apr 12, 2012 11:51 pm
                                                                              Re: Comparative nodes per second Robert Hyatt Sat Apr 14, 2012 4:23 am
                                                                        Re: Comparative nodes per second Uri Blass Sat Apr 14, 2012 5:05 pm
                                                                              Re: Comparative nodes per second Robert Hyatt Sat Apr 14, 2012 6:46 pm
                                                                                    Re: Comparative nodes per second Don Dailey Sat Apr 14, 2012 6:56 pm
                                                                                          Re: Comparative nodes per second Robert Hyatt Sun Apr 15, 2012 4:31 am
                                                                                    Re: Comparative nodes per second Larry Kaufman Sat Apr 14, 2012 7:47 pm
                                                                                          Re: Comparative nodes per second Robert Hyatt Sun Apr 15, 2012 4:36 am
                                                                                          Re: Comparative nodes per second Larry Kaufman Sun Apr 15, 2012 4:47 am
                                                                                          Re: Comparative nodes per second Robert Hyatt Mon Apr 16, 2012 7:14 pm
                                                                                          Re: Comparative nodes per second Robert Hyatt Tue Apr 17, 2012 3:02 pm
                                          Re: Comparative nodes per second Robert Hyatt Thu Apr 12, 2012 6:50 pm
                                                Re: Comparative nodes per second Larry Kaufman Thu Apr 12, 2012 8:12 pm
                                                      Re: Comparative nodes per second Robert Hyatt Sat Apr 14, 2012 4:28 am
                                                            Re: Comparative nodes per second Larry Kaufman Sat Apr 14, 2012 4:45 am
                                                                  Re: Comparative nodes per second Robert Hyatt Sat Apr 14, 2012 4:46 pm
                                                                        Re: Comparative nodes per second Larry Kaufman Sat Apr 14, 2012 5:10 pm
                                                                              Re: Comparative nodes per second Sam Hamilton Sat Apr 14, 2012 5:35 pm
                                                                              Re: Comparative nodes per second Robert Hyatt Sun Apr 15, 2012 4:38 am
                                                                                    Re: Comparative nodes per second Larry Kaufman Sun Apr 15, 2012 4:51 am
                                                                                          Re: Comparative nodes per second Robert Hyatt Mon Apr 16, 2012 7:20 pm
                                                                                    Re: Comparative nodes per second Don Dailey Sun Apr 15, 2012 12:14 pm
                                                                                          Re: Comparative nodes per second Robert Hyatt Mon Apr 16, 2012 7:21 pm
                                                                  Re: Comparative nodes per second Uri Blass Sat Apr 14, 2012 5:17 pm
                                                                        Re: Comparative nodes per second Larry Kaufman Sat Apr 14, 2012 5:22 pm
                                                                              Re: Comparative nodes per second Uri Blass Sat Apr 14, 2012 5:36 pm
                                                                                    Re: Comparative nodes per second Larry Kaufman Sat Apr 14, 2012 5:47 pm
                                    Re: Comparative nodes per second Sam Hamilton Thu Apr 12, 2012 4:49 pm
                                          Re: Comparative nodes per second Larry Kaufman Thu Apr 12, 2012 5:14 pm
                                                Re: Comparative nodes per second Jon Dart Thu Apr 12, 2012 6:00 pm
                                                      Re: Comparative nodes per second Uri Blass Thu Apr 12, 2012 6:37 pm
                                                            Re: Comparative nodes per second Jon Dart Thu Apr 12, 2012 6:50 pm
                                                            Re: Comparative nodes per second Uri Blass Thu Apr 12, 2012 6:53 pm
                                                                  Re: Comparative nodes per second Lucas Braesch Sun Apr 15, 2012 4:56 am
                                                                        Re: Comparative nodes per second Uri Blass Sun Apr 15, 2012 5:16 am
                                                                              Re: Comparative nodes per second Lucas Braesch Sun Apr 15, 2012 7:58 am
                                                                        Re: Comparative nodes per second Don Dailey Sun Apr 15, 2012 1:10 pm
                                                                              Re: Comparative nodes per second Uri Blass Sun Apr 15, 2012 2:48 pm
                                                                                    Re: Comparative nodes per second Don Dailey Sun Apr 15, 2012 3:26 pm
                                                                                    Re: Comparative nodes per second Robert Hyatt Tue Apr 17, 2012 5:17 pm
                                                                                          Re: Comparative nodes per second Sam Hamilton Tue Apr 17, 2012 6:05 pm
                                                                                          Re: Comparative nodes per second Robert Hyatt Tue Apr 17, 2012 7:11 pm
                                                                                          Re: Comparative nodes per second Don Dailey Tue Apr 17, 2012 6:06 pm
                                                                                          Re: Comparative nodes per second Sam Hamilton Tue Apr 17, 2012 6:16 pm
                                                                                          Re: Comparative nodes per second Don Dailey Tue Apr 17, 2012 6:45 pm
                                                                                          Re: Comparative nodes per second Sam Hamilton Tue Apr 17, 2012 6:55 pm
                                                                                          Re: Comparative nodes per second Robert Hyatt Tue Apr 17, 2012 7:16 pm
                                                                                          Re: Comparative nodes per second Sam Hamilton Tue Apr 17, 2012 7:25 pm
                                                                                          Re: Comparative nodes per second Robert Hyatt Tue Apr 17, 2012 7:59 pm
                                                                                          Re: Comparative nodes per second Sam Hamilton Tue Apr 17, 2012 8:14 pm
                                                                                          Re: Comparative nodes per second Miguel A. Ballicora Tue Apr 17, 2012 10:33 pm
                                                                                          Re: Comparative nodes per second Sam Hamilton Tue Apr 17, 2012 10:51 pm
                                                                                          Re: Comparative nodes per second Vincent Diepeveen Wed Apr 18, 2012 12:12 am
                                                                                          Re: Comparative nodes per second Sam Hamilton Wed Apr 18, 2012 12:28 am
                                                                                          Re: Comparative nodes per second Miguel A. Ballicora Wed Apr 18, 2012 2:28 am
                                                                                          Re: Comparative nodes per second Sam Hamilton Wed Apr 18, 2012 4:32 am
                                                                                          Re: Comparative nodes per second Miguel A. Ballicora Wed Apr 18, 2012 4:44 am
                                                                                          Re: Comparative nodes per second Miguel A. Ballicora Tue Apr 17, 2012 10:28 pm
                                                                              Re: Comparative nodes per second José C. Martínez Galán Wed Apr 18, 2012 6:16 am
                                                                                    Re: Comparative nodes per second Robert Hyatt Thu Apr 19, 2012 9:53 pm
                                                                                    Re: Comparative nodes per second Don Dailey Thu Apr 19, 2012 10:22 pm
                                                                                          Re: Comparative nodes per second José C. Martínez Galán Thu Apr 19, 2012 10:49 pm
                                                                                          Re: Comparative nodes per second Robert Hyatt Thu Apr 19, 2012 11:21 pm
                                                                                          Re: Comparative nodes per second Marco Costalba Fri Apr 20, 2012 5:18 am
                                                                                          Re: Comparative nodes per second Michel Van den Bergh Fri Apr 20, 2012 6:10 am
                                                                                          Re: Comparative nodes per second Miguel A. Ballicora Fri Apr 20, 2012 11:35 am
                                                                                          Re: Comparative nodes per second Don Dailey Fri Apr 20, 2012 10:57 am
                                                                                          Re: Comparative nodes per second Marco Costalba Fri Apr 20, 2012 1:58 pm
                                                                                          Re: Comparative nodes per second Sam Hamilton Fri Apr 20, 2012 2:19 pm
                                                                                          Re: Comparative nodes per second Miguel A. Ballicora Fri Apr 20, 2012 2:29 pm
                                                                                          Re: Comparative nodes per second Don Dailey Fri Apr 20, 2012 3:13 pm
                                                                                          Re: Comparative nodes per second Don Dailey Fri Apr 20, 2012 4:52 pm
                                                                                          Re: Comparative nodes per second Uri Blass Sat Apr 21, 2012 6:03 am
                                                                                          Re: Comparative nodes per second Marco Costalba Sat Apr 21, 2012 8:14 am
                                                                                          Re: Comparative nodes per second Ed Schroder Sat Apr 21, 2012 10:50 am
                                                                                          Re: Comparative nodes per second Marco Costalba Sat Apr 21, 2012 11:13 am
                                                                                          Re: Comparative nodes per second Don Dailey Fri Apr 20, 2012 12:31 pm
                                                                                          Re: Comparative nodes per second Dan Andersson Fri Apr 20, 2012 4:14 pm
                                                                                          Re: Comparative nodes per second Don Dailey Fri Apr 20, 2012 4:38 pm
                                                            Re: Comparative nodes per second Robert Hyatt Thu Apr 12, 2012 7:10 pm
                                                Re: Comparative nodes per second Robert Hyatt Thu Apr 12, 2012 6:55 pm
                                                      Re: Comparative nodes per second (early results) Robert Hyatt Thu Apr 12, 2012 7:36 pm
                              Re: Comparative nodes per second Robert Hyatt Thu Apr 12, 2012 7:13 pm
                                    Re: Comparative nodes per second Larry Kaufman Thu Apr 12, 2012 8:18 pm
                                          Re: Comparative nodes per second Robert Hyatt Thu Apr 12, 2012 8:55 pm
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Programming and Technical Discussions

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum




Powered by phpBB © 2001, 2005 phpBB Group
Enhanced with Moby Threads