Tag: theano

Tips and Tricks for training Neural Network in Theano

Theano is a popular Python’s meta programming framework used for Deep Learning on top of either CPU or GPU. Purpose of this blog is to suggest some tips which you can incorporate if you are getting trouble while performing Deep Learning on your problem.

  • Constant Validation Error– If you have just started with Theano and are applying logistic regression model to your problem (MNIST’s Digit recognition is not considered as problem here), then you are likely to get constant validation error while training. If that happens you need to fix your learning rate by determining the optimal one. Start with 0.1 and keep reducing it by a factor of 10 after every epoch until you see a fall in validation error and then use that learning rate for training. Tip- Whenever you initiate training always start with a smaller dataset, say 500-1000 samples, and try to overfit your model. Give same Dataset to training, validation and test. You should get a 100% test error. Your network should have more number of nodes compared to your input so that it can fit. If this is not happening certainly there’s some bug in your implementation.
  • Gaussian Initialization– By default Theano developers have set Initialization of weights to random uniform distribution. Change it to Gaussian(normal) Distribution, you are then likely to get improved results.

Continue reading “Tips and Tricks for training Neural Network in Theano”

Installing Theano and integrating it with GPU on Ubuntu.

If you have a working experience on theano, you probably wouldn’t have forgot that such a pain in the ass task it was. So I felt it really worth to blog about it for people aspiring to get in Deep Learning. Installation instructions given on the official website are capable enough to break down the morale of any newbie who wants to get started.  Follow the instructions
mentioned below to setup theano on Ubuntu.

Continue reading “Installing Theano and integrating it with GPU on Ubuntu.”