ChessUSA.com TalkChess.com
Hosted by Your Move Chess & Games
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

tuning for the uninformed
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Programming and Technical Discussions Flat
View previous topic :: View next topic  
Author Message
Álvaro Begué



Joined: 09 Mar 2010
Posts: 758
Location: New York

PostPost subject: Re: tuning for the uninformed    Posted: Mon Nov 27, 2017 5:52 pm Reply to topic Reply with quote

Robert Pope wrote:
Henk wrote:
CheckersGuy wrote:
Henk wrote:
All does not work if search space has a great many local optima and only very few global optima that you are interested in. But simulated annealing taking too long.


Local search and any other practical algorithm to minimize the error will end up in a local optimum.


Wasn't it that if it optimizes enough parameters you won't get trapped in a local optimum. I can't remember.


I don't think there is ever a guarantee of that without additional information about the domain. There's always the chance of a global minimum that is far from the rest of the "good" solutions that you will never hit except by very good chance.

Consider trying to find the minimum of this function (without actually knowing the function ahead of time):

y=-x, for 4999<x<=5000
y=x^2, for all other x

There is basically no chance someone is going to find the global minimum at x=5000. Then add another 100 dimensions for chess tuning.


That function is not qualitatively similar to the loss function being minimized in chess tuning. Adding many more dimensions actually ameliorates the problem of getting stuck in local minima.

The fear of getting stuck in a local minimum is likely overblown. If your evaluation function is linear, the corresponding optimization problem is convex, which implies there is only one critical point, which is the global minimum. If your evaluation function is something like a deep neural network with ReLU activations, the minimization problem is not convex and there are gazillions of critical points, but because of the high dimensionality most of them are saddle points and not minima. There are results from solid-state physics (something about randomized polynomials) that indicate that all the local minima have values contained in a narrow region above the true minimum, so it doesn't really matter which one you find.
Back to top
View user's profile Send private message
Display posts from previous:   
Subject Author Date/Time
tuning for the uninformed Folkert van Heusden Thu Nov 23, 2017 10:37 am
      Re: tuning for the uninformed Henk van den Belt Thu Nov 23, 2017 11:49 am
      Re: tuning for the uninformed Mehdi Amini Thu Nov 23, 2017 11:49 am
            Re: tuning for the uninformed Henk van den Belt Thu Nov 23, 2017 11:53 am
                  Re: tuning for the uninformed Folkert van Heusden Thu Nov 23, 2017 12:26 pm
                        Re: tuning for the uninformed Henk van den Belt Thu Nov 23, 2017 12:44 pm
      Re: tuning for the uninformed Alexandru Mosoi Thu Nov 23, 2017 2:07 pm
            Re: tuning for the uninformed Folkert van Heusden Thu Nov 23, 2017 2:16 pm
                  Re: tuning for the uninformed Folkert van Heusden Thu Nov 23, 2017 5:34 pm
                        Re: tuning for the uninformed Sander Maassen vd Brink Thu Nov 23, 2017 6:47 pm
                              Re: tuning for the uninformed Henk van den Belt Thu Nov 23, 2017 7:21 pm
                                    Re: tuning for the uninformed Sander Maassen vd Brink Thu Nov 23, 2017 7:26 pm
                                          Re: tuning for the uninformed Henk van den Belt Thu Nov 23, 2017 7:36 pm
                                                Re: tuning for the uninformed Sander Maassen vd Brink Thu Nov 23, 2017 7:43 pm
                                          Re: tuning for the uninformed Martin Fierz Tue Dec 12, 2017 9:17 pm
                                                Re: tuning for the uninformed Álvaro Begué Tue Dec 12, 2017 10:47 pm
                              Re: tuning for the uninformed Folkert van Heusden Fri Nov 24, 2017 9:15 am
                                    Re: tuning for the uninformed Sander Maassen vd Brink Fri Nov 24, 2017 9:30 am
                              Re: tuning for the uninformed Robin Messemer Fri Nov 24, 2017 11:04 pm
                                    Re: tuning for the uninformed Henk van den Belt Sat Nov 25, 2017 11:30 am
                                          Re: tuning for the uninformed Robin Messemer Sat Nov 25, 2017 1:58 pm
                                                Re: tuning for the uninformed Henk van den Belt Sat Nov 25, 2017 2:48 pm
                                                      Re: tuning for the uninformed Robert Pope Mon Nov 27, 2017 5:38 pm
                                                            Re: tuning for the uninformed Álvaro Begué Mon Nov 27, 2017 5:52 pm
                                                                  Re: tuning for the uninformed Alexandru Mosoi Mon Nov 27, 2017 9:56 pm
                                                                        Re: tuning for the uninformed Robert Pope Tue Nov 28, 2017 4:41 pm
                                                                              Re: tuning for the uninformed Robin Messemer Tue Nov 28, 2017 7:23 pm
                                                                                    Re: tuning for the uninformed Álvaro Begué Tue Nov 28, 2017 7:37 pm
                                                            Re: tuning for the uninformed Jon Dart Sat Dec 02, 2017 3:02 pm
                                          Re: tuning for the uninformed Jon Dart Sat Dec 02, 2017 2:58 pm
                                    Re: tuning for the uninformed Erin Dame Sat Dec 02, 2017 10:50 am
                        Re: tuning for the uninformed Jon Dart Sat Dec 02, 2017 2:54 pm
                              Re: tuning for the uninformed Erin Dame Sat Dec 02, 2017 5:14 pm
                                    Re: tuning for the uninformed Jon Dart Sat Dec 02, 2017 5:56 pm
                                          Re: tuning for the uninformed - Poisson? Jon Dart Sat Dec 02, 2017 8:09 pm
                                                Re: tuning for the uninformed - Poisson? Matthew R. Brades Sun Dec 03, 2017 10:09 am
                                                      Re: tuning for the uninformed - Poisson? Álvaro Begué Sun Dec 03, 2017 1:03 pm
                                                      Re: derivatives, scaling Jon Dart Sun Dec 03, 2017 3:47 pm
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Programming and Technical Discussions

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum




Powered by phpBB © 2001, 2005 phpBB Group
Enhanced with Moby Threads