ChessUSA.com TalkChess.com
Hosted by Your Move Chess & Games
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Deep misery
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Programming and Technical Discussions Flat
View previous topic :: View next topic  
Author Message
Henk van den Belt



Joined: 27 May 2013
Posts: 4895

PostPost subject: Re: Deep misery    Posted: Tue Feb 13, 2018 10:45 pm Reply to topic Reply with quote

trulses wrote:
Henk wrote:
I looked at batch normalization. But my network does not use mini-batches and computes gradient for loss over one training example.

I read that probably ELU or SELU might help and is faster than using batch normalization. If not then I switch over to mini batches.


I logged on for the sole purpose of saying this, but implement mini-batching as your top priority. It's going to stabilize everything you're doing by some massive amount.


Using SELU now. Some say it's the best others are not so enthusiastic about it. I think I have to calculate variance and mean of the input values and see if it works or not.
Back to top
View user's profile Send private message
Display posts from previous:   
Subject Author Date/Time
Deep misery Henk van den Belt Wed Feb 07, 2018 8:34 pm
      Re: Deep misery Álvaro Begué Wed Feb 07, 2018 8:48 pm
            Re: Deep misery Henk van den Belt Wed Feb 07, 2018 8:55 pm
                  Re: Deep misery Álvaro Begué Wed Feb 07, 2018 9:08 pm
                        Re: Deep misery Henk van den Belt Thu Feb 08, 2018 4:42 pm
                              Re: Deep misery Álvaro Begué Thu Feb 08, 2018 4:56 pm
                                    Re: Deep misery Henk van den Belt Thu Feb 08, 2018 5:17 pm
                                          Re: Deep misery Álvaro Begué Thu Feb 08, 2018 6:03 pm
                                          Re: Deep misery Henk van den Belt Fri Feb 09, 2018 11:19 am
                                                Re: Deep misery Álvaro Begué Fri Feb 09, 2018 12:38 pm
                                                      Re: Deep misery Henk van den Belt Fri Feb 09, 2018 12:50 pm
                                                            Re: Deep misery Henk van den Belt Sat Feb 10, 2018 2:30 pm
                  Re: Deep misery Truls Edvard Stokke Tue Feb 13, 2018 8:38 pm
                        Re: Deep misery Henk van den Belt Tue Feb 13, 2018 10:45 pm
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Programming and Technical Discussions

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum




Powered by phpBB © 2001, 2005 phpBB Group
Enhanced with Moby Threads