TalkChess.com
Hosted by Your Move Chess & Games

Author Message
Álvaro Begué

Joined: 09 Mar 2010
Posts: 804
Location: New York

Post subject: Re: tuning for the uninformed    Posted: Mon Nov 27, 2017 5:52 pm

Robert Pope wrote:
Henk wrote:
CheckersGuy wrote:
 Henk wrote: All does not work if search space has a great many local optima and only very few global optima that you are interested in. But simulated annealing taking too long.

Local search and any other practical algorithm to minimize the error will end up in a local optimum.

Wasn't it that if it optimizes enough parameters you won't get trapped in a local optimum. I can't remember.

I don't think there is ever a guarantee of that without additional information about the domain. There's always the chance of a global minimum that is far from the rest of the "good" solutions that you will never hit except by very good chance.

Consider trying to find the minimum of this function (without actually knowing the function ahead of time):

y=-x, for 4999<x<=5000
y=x^2, for all other x

There is basically no chance someone is going to find the global minimum at x=5000. Then add another 100 dimensions for chess tuning.

That function is not qualitatively similar to the loss function being minimized in chess tuning. Adding many more dimensions actually ameliorates the problem of getting stuck in local minima.

The fear of getting stuck in a local minimum is likely overblown. If your evaluation function is linear, the corresponding optimization problem is convex, which implies there is only one critical point, which is the global minimum. If your evaluation function is something like a deep neural network with ReLU activations, the minimization problem is not convex and there are gazillions of critical points, but because of the high dimensionality most of them are saddle points and not minima. There are results from solid-state physics (something about randomized polynomials) that indicate that all the local minima have values contained in a narrow region above the true minimum, so it doesn't really matter which one you find.
 Display posts from previous: All Posts1 Day7 Days2 Weeks1 Month3 Months6 Months1 Year Oldest FirstNewest First
Subject Author Date/Time
Folkert van Heusden Thu Nov 23, 2017 10:37 am
Henk van den Belt Thu Nov 23, 2017 11:49 am
Mehdi Amini Thu Nov 23, 2017 11:49 am
Henk van den Belt Thu Nov 23, 2017 11:53 am
Folkert van Heusden Thu Nov 23, 2017 12:26 pm
Henk van den Belt Thu Nov 23, 2017 12:44 pm
Alexandru Mosoi Thu Nov 23, 2017 2:07 pm
Folkert van Heusden Thu Nov 23, 2017 2:16 pm
Folkert van Heusden Thu Nov 23, 2017 5:34 pm
Sander Maassen vd Brink Thu Nov 23, 2017 6:47 pm
Henk van den Belt Thu Nov 23, 2017 7:21 pm
Sander Maassen vd Brink Thu Nov 23, 2017 7:26 pm
Henk van den Belt Thu Nov 23, 2017 7:36 pm
Sander Maassen vd Brink Thu Nov 23, 2017 7:43 pm
Martin Fierz Tue Dec 12, 2017 9:17 pm
Álvaro Begué Tue Dec 12, 2017 10:47 pm
Folkert van Heusden Fri Nov 24, 2017 9:15 am
Sander Maassen vd Brink Fri Nov 24, 2017 9:30 am
Robin Messemer Fri Nov 24, 2017 11:04 pm
Henk van den Belt Sat Nov 25, 2017 11:30 am
Robin Messemer Sat Nov 25, 2017 1:58 pm
Henk van den Belt Sat Nov 25, 2017 2:48 pm
Robert Pope Mon Nov 27, 2017 5:38 pm
Re: tuning for the uninformed Álvaro Begué Mon Nov 27, 2017 5:52 pm
Alexandru Mosoi Mon Nov 27, 2017 9:56 pm
Robert Pope Tue Nov 28, 2017 4:41 pm
Robin Messemer Tue Nov 28, 2017 7:23 pm
Álvaro Begué Tue Nov 28, 2017 7:37 pm
Jon Dart Thu Dec 14, 2017 3:09 pm
Álvaro Begué Thu Dec 14, 2017 3:15 pm
Jon Dart Sat Dec 02, 2017 3:02 pm
Jon Dart Sat Dec 02, 2017 2:58 pm
Erin Dame Sat Dec 02, 2017 10:50 am
Jon Dart Sat Dec 02, 2017 2:54 pm
Erin Dame Sat Dec 02, 2017 5:14 pm
Jon Dart Sat Dec 02, 2017 5:56 pm
Jon Dart Sat Dec 02, 2017 8:09 pm
Matthew R. Brades Sun Dec 03, 2017 10:09 am
Álvaro Begué Sun Dec 03, 2017 1:03 pm
Jon Dart Sun Dec 03, 2017 3:47 pm

 Jump to: Select a forum Computer Chess Club Forums----------------Computer Chess Club: General TopicsComputer Chess Club: Tournaments and MatchesComputer Chess Club: Programming and Technical DiscussionsComputer Chess Club: Engine Origins Other Forums----------------Chess Thinkers ForumChess Players ForumForum Help and Suggestions
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum