Funciones de activacion

Avatar de Usuario
Dalamar
Site Admin
Mensajes: 8349
Registrado: 09 May 2012 01:38
Ubicación: En Communera! Creando comunidades!
Contactar:

Funciones de activacion

Mensajepor Dalamar » 04 Abr 2017 16:44

*********************************
***** EN PROGRESO ***************
*********************************


logictic sigmoid

relu

Dying Relu problem: Relu neurons can sometimes be pushed into states in which they become inactive for essentially all inputs. In this state, no gradients flow backward through the neuron, and so the neuron becomes stuck in a perpetually inactive state and "dies." In some cases, large numbers of neurons in a network can become stuck in dead states, effectively decreasing the model capacity. This problem typically arises when the learning rate is set too high.


tanh

softmax

There are plenty more activation functions available and different researchers keep on looking for new ones (e.g. Exponential Linear Units (ELUs), Gaussian Error Linear Units (GELUs), ...) with different/better properties


Regression: linear (because values are unbounded)

Classification: softmax (simple sigmoid works too but softmax works better)

Use simple sigmoid only if your output admits multiple "true" answers, for instance, a network that checks for the presence of various objects in an image. In other words, the output is not a probability distribution (does not need to sum to 1).

Sigmoid and tanh should not be used as activation function for the hidden layer. This is because of the vanishing gradient problem, i.e., if your input is on a higher side (where sigmoid goes flat) then the gradient will be near zero. This will cause very slow or no learning during backpropagation as weights will be updated with really small values.

Detailed explanation here: http://cs231n.github.io/neural-networks-1/#actfun

The best function for hidden layers is thus ReLu.
Adjuntos
SoftMax.JPG
¿Te ha gustado este hilo? Retweetealo para que se sume mas gente a la conversacion!

Volver a “Inteligencia Artificial”

cron

Ingresar