ChessUSA.com TalkChess.com
Hosted by Your Move Chess & Games
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

When will we see HOUDINI in official tournaments?
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: General Topics Flat
View previous topic :: View next topic  
Author Message
Don Dailey



Joined: 29 Apr 2008
Posts: 4507
PostPost subject: Re: When will we see HOUDINI in official tournaments?    Posted: Fri May 11, 2012 4:11 am Reply to topic Reply with quote
Mike S. wrote:
Quote:
The test measures the evaluation function more than anything else (...)

That is exactly what I have pointed to, earlier. While I do not doubt that your similarity testing tool is a very good achievement and a step forward, I DO doubt that the exclusion of any engine, based on similarity percentage, should be based on evaluation similartiy (almost) only. What about differences of the search code?

If someone heavily modifies the search, the similarity detector will NOT detect that the program has changed substantially.

However that is one of those "bugs" that I consider a feature. It's easy to make substantial search changes - even small changes that can have a big impact on the search. I can make Komodo search a ply deeper for example without it hurting the ELO much.

By the way, I am not in favor of the test being used to exclude programs from tournaments. However I am in favor of it being used to clear a program or as an investigative tool but not to be taken too seriously until it's better understood.

However if it were used like that you are obviously bothered by the fact that a programmer could modify the evaluation function significantly and get away with it since it does not catch search changes. I would not worry too much about that - the real grunt work is in the evaluation function and the cheaters are not usually going to put a lot of time into evaluation - they took someone else's code in order to not have to deal with writing an evaluation function from scratch and they are lazy.

The good news is that a plagiarist is likely to make substantial search modifications and even improve the program significantly - and yet will still get caught by the test because writing an evaluation from scratch is almost the same as writing a new chess program from scratch. I can build a search function in a day or two that is reasonably good, but it takes months to write a reasonably good evaluation function from scratch.

Quote:


Not being a programmer, I still think that the set of useful evaluation terms is limited and it's basically "predefined" (e.g. as described in chess literature), differences of implementation are more or less small details. But I can imagine that for the search code(s) - a "highly technical" thing as viewn from my limited horizon of these things - there are endless opportunities and variations of how to do it.


Wow! You have this exactly backwards. It's true the search can be highly technical but as I just said you can build a reasonable search very quickly, but you cannot build a reasonable evaluation function without taking a few months.

Quote:


In other words, I critizise that it may be said an engine is "65% similar" just because the eval is maybe 61% similar, but the search much different. We are on a wrong path.


I give the evaluation function at least 80% of the weight and the search is almost an insignificant detail in comparison. Now please understand that I'm not saying the search is not critical - it's extremely important and a huge contributor to the strength of a program. But from the perspective of the originality of the program the evaluation is much more important. Remember, we are talking about a CHESS program and the domain specific part of the program is the evaluation function - not the search (except for the move generator itself which is the most mundane part of a program.)
_________________
The Optimist thinks this is the best of all possible worlds, the Pessimist fears it is true.
Back to top
View user's profile Send private message Send e-mail
Display posts from previous:   
Subject Author Date/Time
When will we see HOUDINI in official tournaments? Teodoriu Catalin Tue May 01, 2012 5:46 pm
      Re: When will we see HOUDINI in official tournaments? Robert Houdart Tue May 01, 2012 6:21 pm
            Re: When will we see HOUDINI in official tournaments? Teodoriu Catalin Tue May 01, 2012 6:56 pm
            Re: When will we see HOUDINI in official tournaments? S.Taylor Fri May 04, 2012 9:44 am
            Re: When will we see HOUDINI in official tournaments? Ed Schroder Fri May 04, 2012 12:34 pm
                  Re: When will we see HOUDINI in official tournaments? Robert Houdart Fri May 04, 2012 6:20 pm
                        Re: When will we see HOUDINI in official tournaments? Graham Banks Fri May 04, 2012 6:58 pm
                              Re: When will we see HOUDINI in official tournaments? Dr.Wael Deeb Fri May 04, 2012 7:25 pm
                              Re: When will we see HOUDINI in official tournaments? Robert Houdart Fri May 04, 2012 7:38 pm
                        Re: When will we see HOUDINI in official tournaments? Alexander Schmidt Fri May 04, 2012 7:48 pm
                              Re: When will we see HOUDINI in official tournaments? Robert Houdart Fri May 04, 2012 8:10 pm
                                    Re: When will we see HOUDINI in official tournaments? Alexander Schmidt Fri May 04, 2012 8:31 pm
                              Re: When will we see HOUDINI in official tournaments? Dr.Wael Deeb Fri May 04, 2012 8:48 pm
                        Re: When will we see HOUDINI in official tournaments? Miguel A. Ballicora Fri May 04, 2012 8:20 pm
                              Re: When will we see HOUDINI in official tournaments? Robert Houdart Fri May 04, 2012 8:43 pm
                                    Re: When will we see HOUDINI in official tournaments? Ed Schroder Sat May 05, 2012 9:47 am
                                          Re: When will we see HOUDINI in official tournaments? Robert Houdart Sat May 05, 2012 9:57 am
                                                Re: When will we see HOUDINI in official tournaments? Graham Banks Sat May 05, 2012 10:03 am
                                                Re: When will we see HOUDINI in official tournaments? Ed Schroder Sat May 05, 2012 10:27 am
                                                      Re: When will we see HOUDINI in official tournaments? Mike Scheidl Sat May 05, 2012 10:42 am
                                                            Re: When will we see HOUDINI in official tournaments? Ed Schroder Sat May 05, 2012 11:41 am
                                                                  Re: When will we see HOUDINI in official tournaments? Mike Scheidl Sat May 05, 2012 12:16 pm
                                                                        Re: When will we see HOUDINI in official tournaments? Ed Schroder Sat May 05, 2012 6:45 pm
                                                                              Re: When will we see HOUDINI in official tournaments? Mike Scheidl Sun May 06, 2012 4:23 am
                                                                                    Re: When will we see HOUDINI in official tournaments? Uri Blass Sun May 06, 2012 5:46 am
                                                                                    Re: When will we see HOUDINI in official tournaments? Ed Schroder Sun May 06, 2012 2:27 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Johan Havegheer Sun May 06, 2012 7:57 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Graham Banks Sun May 06, 2012 8:24 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Ed Schroder Sun May 06, 2012 9:30 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Richard Vida Sun May 06, 2012 11:21 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Neel Basant Mon May 07, 2012 3:40 am
                                                                                          Re: When will we see HOUDINI in official tournaments? Dr.Wael Deeb Mon May 07, 2012 7:48 am
                                                                                          Re: When will we see HOUDINI in official tournaments? Neel Basant Mon May 07, 2012 9:02 am
                                                                                          Re: When will we see HOUDINI in official tournaments? Dr.Wael Deeb Mon May 07, 2012 2:24 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Sedat Canbaz Mon May 07, 2012 4:50 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? S.Taylor Mon May 07, 2012 9:27 am
                                                                                          Re: When will we see HOUDINI in official tournaments? Dr.Wael Deeb Mon May 07, 2012 2:23 pm
                                                      Re: When will we see HOUDINI in official tournaments? Ingo Bauer Sat May 05, 2012 11:42 am
                                                            Re: When will we see HOUDINI in official tournaments? Ed Schroder Sat May 05, 2012 12:08 pm
                                                                  Re: When will we see HOUDINI in official tournaments? Ingo Bauer Sat May 05, 2012 12:20 pm
                                                                        Re: When will we see HOUDINI in official tournaments? Ed Schroder Sat May 05, 2012 3:41 pm
                                                                              Re: When will we see HOUDINI in official tournaments? Ingo Bauer Sat May 05, 2012 4:55 pm
                                                                                    Re: When will we see HOUDINI in official tournaments? Miguel A. Ballicora Sat May 05, 2012 5:06 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Ingo Bauer Sat May 05, 2012 5:14 pm
                                                                                    Re: When will we see HOUDINI in official tournaments? Harvey Williamson Sat May 05, 2012 5:10 pm
                                                                                    Re: When will we see HOUDINI in official tournaments? Uri Blass Sat May 05, 2012 5:16 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Ingo Bauer Sat May 05, 2012 5:24 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Miguel A. Ballicora Sat May 05, 2012 5:55 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Harvey Williamson Sat May 05, 2012 6:00 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Miguel A. Ballicora Sat May 05, 2012 6:14 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Harvey Williamson Sat May 05, 2012 6:16 pm
                                                                                    Re: When will we see HOUDINI in official tournaments? Sam Hamilton Sat May 05, 2012 6:07 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Ingo Bauer Sat May 05, 2012 6:25 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Miguel A. Ballicora Sat May 05, 2012 6:28 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Ingo Bauer Sat May 05, 2012 6:46 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Miguel A. Ballicora Sat May 05, 2012 7:10 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Ingo Bauer Sat May 05, 2012 7:22 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Miguel A. Ballicora Sat May 05, 2012 8:42 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Ingo Bauer Sat May 05, 2012 8:48 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Graham Banks Sat May 05, 2012 7:47 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Ingo Bauer Sat May 05, 2012 7:58 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Damir Desevac Sat May 05, 2012 8:01 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Gabriel LEPERLIER Sat May 05, 2012 8:02 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Graham Banks Sat May 05, 2012 8:03 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Ingo Bauer Sat May 05, 2012 8:18 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Graham Banks Sat May 05, 2012 8:21 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Ingo Bauer Sat May 05, 2012 8:23 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Harvey Williamson Sat May 05, 2012 8:26 pm
                                                                  Re: When will we see HOUDINI in official tournaments? Sam Hamilton Sat May 05, 2012 2:08 pm
                                                                        Re: When will we see HOUDINI in official tournaments? Graham Banks Sat May 05, 2012 2:35 pm
                                                                              Re: When will we see HOUDINI in official tournaments? Chris Whittington Sat May 05, 2012 2:46 pm
                                                                                    Re: When will we see HOUDINI in official tournaments? George Speight Sun May 06, 2012 12:23 am
                                                                        Re: When will we see HOUDINI in official tournaments? Ed Schroder Sat May 05, 2012 3:53 pm
                                                                              Re: When will we see HOUDINI in official tournaments? Albert Silver Sat May 05, 2012 4:02 pm
                                                                  Re: When will we see HOUDINI in official tournaments? Uri Blass Sat May 05, 2012 2:55 pm
                                                                        Re: When will we see HOUDINI in official tournaments? Uri Blass Sat May 05, 2012 3:04 pm
                                                                              Re: When will we see HOUDINI in official tournaments? Ed Schroder Sat May 05, 2012 4:08 pm
                                                                        Re: When will we see HOUDINI in official tournaments? Miguel A. Ballicora Sat May 05, 2012 3:04 pm
                                                                              Re: When will we see HOUDINI in official tournaments? Harvey Williamson Sat May 05, 2012 3:05 pm
                                                                                    Re: When will we see HOUDINI in official tournaments? Ed Schroder Sat May 05, 2012 4:20 pm
                                                                                    Re: When will we see HOUDINI in official tournaments? Alex Newman Sat May 05, 2012 5:16 pm
                                                                  Re: When will we see HOUDINI in official tournaments? H.G.Muller Sat May 05, 2012 9:28 pm
                        Re: When will we see HOUDINI in official tournaments? Ed Schroder Fri May 04, 2012 8:38 pm
                              Re: When will we see HOUDINI in official tournaments? Robert Houdart Fri May 04, 2012 8:44 pm
                              Re: When will we see HOUDINI in official tournaments? Dr. Peter Martan Sat May 05, 2012 3:24 am
                                    Re: When will we see HOUDINI in official tournaments? Robert Houdart Sat May 05, 2012 9:12 am
                                          Re: When will we see HOUDINI in official tournaments? Teodoriu Catalin Sat May 05, 2012 10:27 am
                                          Re: When will we see HOUDINI in official tournaments? Engin Üstün Thu May 10, 2012 9:13 pm
                                                Re: When will we see HOUDINI in official tournaments? Engin Üstün Thu May 10, 2012 9:15 pm
                                                      Re: When will we see HOUDINI in official tournaments? Albert Silver Thu May 10, 2012 9:38 pm
                                                            Re: When will we see HOUDINI in official tournaments? Don Dailey Thu May 10, 2012 9:48 pm
                                                                  Re: When will we see HOUDINI in official tournaments? Mike Scheidl Fri May 11, 2012 3:35 am
                                                                        Re: When will we see HOUDINI in official tournaments? Don Dailey Fri May 11, 2012 4:11 am
                                                                              Re: When will we see HOUDINI in official tournaments? Mike Scheidl Fri May 11, 2012 4:23 am
                                                                                    Re: When will we see HOUDINI in official tournaments? Don Dailey Fri May 11, 2012 4:27 am
                                                                              Re: When will we see HOUDINI in official tournaments? Alex Newman Fri May 11, 2012 5:11 pm
                                                                                    Re: When will we see HOUDINI in official tournaments? Don Dailey Fri May 11, 2012 6:25 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Graham Banks Fri May 11, 2012 6:40 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Don Dailey Fri May 11, 2012 6:46 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Alex Newman Fri May 11, 2012 9:01 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Don Dailey Fri May 11, 2012 10:25 pm
                                                                                    Re: When will we see HOUDINI in official tournaments? Uri Blass Fri May 11, 2012 7:39 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Don Dailey Fri May 11, 2012 7:44 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Uri Blass Fri May 11, 2012 7:52 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Don Dailey Fri May 11, 2012 8:33 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Uri Blass Sat May 12, 2012 1:41 am
                                                                                          Re: When will we see HOUDINI in official tournaments? Miguel A. Ballicora Sat May 12, 2012 6:33 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Dan Honeycutt Sat May 12, 2012 6:53 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Uri Blass Sat May 12, 2012 10:18 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Dan Honeycutt Sun May 13, 2012 12:39 am
                                                                                    Re: When will we see HOUDINI in official tournaments? Charles Roberson Sat May 12, 2012 2:02 am
                                                                                          Re: When will we see HOUDINI in official tournaments? Alex Newman Sat May 12, 2012 5:25 pm
                                                                                          Re: When will we see HOUDINI in official tournaments? Uri Blass Sat May 12, 2012 6:14 pm
                        Re: When will we see HOUDINI in official tournaments? Neel Basant Sat May 05, 2012 5:55 am
                              Re: When will we see HOUDINI in official tournaments? Robert Houdart Sat May 05, 2012 9:18 am
                              Re: When will we see HOUDINI in official tournaments? George Speight Mon May 07, 2012 5:21 pm
                  Re: When will we see HOUDINI in official tournaments? Sedat Canbaz Sun May 06, 2012 4:22 am
      Re: When will we see HOUDINI in official tournaments? S.Taylor Sat May 05, 2012 11:08 pm
            Re: When will we see HOUDINI in official tournaments? Alex Brown Sat May 05, 2012 11:35 pm
      CSVN rules? Emil Vlasak Mon May 07, 2012 3:51 am
            Re: CSVN rules? Ed Schroder Mon May 07, 2012 4:22 am
                  Re: CSVN rules? Emil Vlasak Mon May 07, 2012 12:53 pm
                        Re: CSVN rules? Alex Newman Mon May 07, 2012 5:16 pm
                              Re: CSVN rules? Miguel A. Ballicora Mon May 07, 2012 5:44 pm
                              Re: CSVN rules? H.G.Muller Mon May 07, 2012 5:44 pm
                                    Re: CSVN rules? Alex Newman Mon May 07, 2012 5:52 pm
                                          Re: CSVN rules? H.G.Muller Mon May 07, 2012 5:56 pm
                                                Re: CSVN rules? Alex Newman Mon May 07, 2012 6:22 pm
                                                      Re: CSVN rules? H.G.Muller Mon May 07, 2012 6:27 pm
                        Re: CSVN rules? Ed Schroder Mon May 07, 2012 8:18 pm
                              Re: CSVN rules? Emil Vlasak Tue May 08, 2012 6:20 am
                                    Re: CSVN rules? H.G.Muller Tue May 08, 2012 7:13 am
                                          Re: CSVN rules? Emil Vlasak Tue May 08, 2012 2:47 pm
                                                Re: CSVN rules? H.G.Muller Tue May 08, 2012 4:03 pm
                                                      Re: CSVN rules? Alex Newman Tue May 08, 2012 5:36 pm
                                                            Re: CSVN rules? H.G.Muller Tue May 08, 2012 5:54 pm
                                                                  Re: CSVN rules? Harvey Williamson Tue May 08, 2012 6:19 pm
                                                                        Re: CSVN rules? Mark Lefler Tue May 08, 2012 6:20 pm
                                                                        Re: CSVN rules? Don Dailey Wed May 09, 2012 11:27 am
                                                                              Re: CSVN rules? Harvey Williamson Wed May 09, 2012 11:31 am
                                                                                    Re: CSVN rules? Don Dailey Wed May 09, 2012 12:11 pm
                                                                                          Re: CSVN rules? Harvey Williamson Wed May 09, 2012 12:13 pm
                                                                                    Re: CSVN rules? Thorsten Czub Wed May 09, 2012 5:26 pm
                                                                              Re: CSVN rules? Thorsten Czub Wed May 09, 2012 5:24 pm
                                                                                    Re: CSVN rules? Don Dailey Wed May 09, 2012 5:29 pm
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: General Topics

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum




Powered by phpBB © 2001, 2005 phpBB Group
Enhanced with Moby Threads