This page includes{{< mnote “in the Hugo sense of the word” >}} my Chapter notes for the book by Michael Nielsen.
Regularisation
This page is for the closed form solutions (where they exist) and approximation solutions to Regularised Regressions.
We will also understand that regularisation is sensible artifact once we consider its MAP (maximum a posteriori) derivation.
- given the findings of the previous chapter (universality), why would we concern ourselves with learning deep neural nets?
- especially given that we are guaranteed to be able to approximate any function with just a single layer of hidden neurons?
well, just because something is possible, it doesn’t mean it’s a good idea!
considering that we are using computers, it’s usually a good idea to break the problem down into smaller sub-problems, solve those, and then come back to solve the main problem.
{{< tikz >}}
\begin{tikzpicture}[x=1.6cm,y=1.1cm] \large \message{^^JDeep convolution neural network}
% Define colors \colorlet{myred}{red!80!black} \colorlet{myblue}{blue!80!black} \colorlet{mygreen}{green!60!black} \colorlet{myorange}{orange!70!red!60!black} \colorlet{mydarkred}{red!30!black} \colorlet{mydarkblue}{blue!40!black} \colorlet{mydarkgreen}{green!30!black}
% Define TikZ styles \tikzset{ >=latex, % for default LaTeX arrow head node/.style={thick,circle,draw=myblue,minimum size=22,inner sep=0.5,outer sep=0.6}, node in/.style={node,green!20!black,draw=mygreen!30!black,fill=mygreen!25}, node hidden/.style={node,blue!20!black,draw=myblue!30!black,fill=myblue!20}, node convol/.style={node,orange!20!black,draw=myorange!30!black,fill=myorange!20}, node out/.style={node,red!20!black,draw=myred!30!black,fill=myred!20}, connect/.style={thick,mydarkblue}, %,line cap=round connect arrow/.style={-{Latex[length=4,width=3.5]},thick,mydarkblue,shorten <=0.5,shorten >=1} }
% Define layers and nodes \def\layerNodes{{5,5,4,3,2,4,4,3}} % Number of nodes in each layer \def\NC{6} % number of convolutional layers \def\totalLayers{8} % total number of layers