Logistic function
lastmod 2018-08-22created 2015-12-2313 edits
The logistic equation is a differential equation, or difference equation, devised in ecology as a simple model of population growth. It was studied as one of the early subjects of chaos theory and is now applied not only in ecology but in many fields.
Derivative
sigmoid function
- How to Compute the Derivative of a Sigmoid Function (fully worked example)
- Derivative of the Logistic Sigmoid (Activation) Function
The standard logistic function (k = 1, x0 = 0, L = 1) has an easily calculated derivative:
\[ \frac{d}{dx}f(x) = f(x)\cdot(1-f(x)) \]
See Sigmoid function.
\[ s(x) = \frac{1}{1+e^{-x}} \]
Use the quotient rule of differentiation (Derivative). (The quotient rule makes use of the product rule)
\[ \begin{align} \frac{d}{dx}s(x) &= \frac{d}{dx}(\frac{1}{1+e^{-x}}) \\ &= \frac{ \frac{d}{dx}(1) * (1+e^{-x}) - (1 * \frac{d}{dx}(1+e^{-x})) }{ (1+e^{-x})^2 } \\ &= \frac{ -\frac{d}{dx}(1+e^{-x}) }{ (1+e^{-x})^2 } \\ &= \frac{ -( \frac{d}{dx}(1) + \frac{d}{dx}(e^{-x}) ) }{ (1+e^{-x})^2 } \\ &= \frac{ -(\frac{d}{dx}(\frac{1}{e^x})) }{ (1+e^{-x})^2 } \end{align} \]
Among the differentiation properties of Euler's number (e) is the following.
\[ \frac{d}{dx}e^{x}=e^{x} \]
Using the above and the quotient rule, for convenience of calculation, we get the following derivative.
\[ \begin{align} \frac{d}{dx}(\frac{1}{e^x}) &= \frac{ \frac{d}{dx}(1)*(e^x) - (1)*(\frac{d}{dx}(e^x)) }{ (e^x)^2 } \\ &= \frac{ -(e^x) }{ (e^x)(e^x) } \\ &= \frac{ -1 }{ e^x } \\ &= -e^{-x} \end{align} \]
Substituting this result back gives the following.
\[ \begin{align} \frac{d}{dx}s(x) &= \frac{ -(\frac{d}{dx}(\frac{1}{e^x})) }{ (1+e^{-x})^2 } \\ &= \frac{ -(-e^{-x}) }{ (1+e^{-x})^2 } \\ &= \frac{ e^{-x} }{ (1+e^{-x})^2 } \end{align} \]
How does the derivative of a sigmoid s(x) equal s(x)(1-(s(x))?
It can be computed as follows.
\[ \begin{align} \frac{d}{dx}s(x) &= \frac{ e^{-x} }{ (1+e^{-x})^2 } \\ &= \frac{(e^{-x} + 1 -1)}{(1+e^{-x})^{2}} \\ &= \frac{(1 + e^{-x} -1)}{(1+e^{-x})^{2}} \\ &= \frac{(1 + e^{-x})}{(1+e^{-x})^{2}} - \frac{1}{(1+e^{-x})^{2}} \\ &= \frac{1}{(1+e^{-x})} - \frac{1}{(1+e^{-x})^{2}} \\ &= \frac{1}{(1+e^{-x})} (1 - \frac{1}{(1+e^{-x})}) \end{align} \]
Therefore the following holds.
\[ s'(x) = \frac{d}{dx}s(x) = s(x) (1 - s(x)) \]
Population growth model
The study of the number of individuals of organisms belongs to the field of population ecology. It has long been studied because it is useful, for example for population estimates and for forecasting pest outbreaks. Since many organisms produce more offspring than actually survive, the population would grow exponentially if all of them survived. Reality, however, is different.
In general, a population count takes integer values, and since reproduction often happens at particular times, population growth takes a stepwise rather than continuous form. However, to simplify the mathematics, both the growth and the population are often treated as continuous values.
Since the number of offspring produced by one parent is usually roughly constant, if the growth rate is r, the rate of increase of the population N can be written as
\[ \frac{dN}{dt} = rN \]
The solution of this equation is an exponential curve, causing a population explosion even in a short time. Because it was Thomas Malthus who pointed out that the growth of organisms in such a population growth model is exponential, it is also called Malthusian growth. Real organisms live in a particular environment, and it is natural to assume that there is an upper limit to the population that can live there. That is, one can imagine that as the population grows, its growth rate decreases. So, to describe such realistic population changes, an equation with the following properties is needed.
- At a population of 0, the growth rate is 0.
- As the population increases, the growth rate decreases.
- If K is the carrying capacity of the environment, the growth rate becomes 0 when N=K.