In short, Neural network is a function: input data, output result.
Let's take MNIST handwritten digit image recognition as an example to define the corresponding function form:
This is an entry-level application of Neural Network. The input is a low-resolution (28 x 28) grayscale image, corresponding to a function with 784 input variables. If it is a megapixel color image, the corresponding input variables will reach 3 million.
It can be seen that the neural network is used to solve some complex problems, and the corresponding functions are also complex. The realization of the algorithm is to construct the corresponding function.
How to construct such a complicated function? We can start with simple functions. The simplest and most successful example is digital circuits.
Digital circuits are the cornerstones of computers, and have built our huge digital world. But its core is composed of AND, OR, and NOT logic gates.
What is a logic gate? It's actually a function. And they are the simplest functions.
Logic gate | expression | function form |
---|---|---|
AND gate | ||
OR gate | ||
NOT gate |
NOT gate image (use 0 for and 1 for )
AND gate image
OR gate image
Combine simple logic gates to get more powerful functions.
Construct a new binary function:
Construct a multivariate function:
By combining simple logic gate functions, new logic functions can be constructed. Including operations such as addition, subtraction, multiplication, and division of 32-bit integers, and operations of 32-bit single-precision floating-point types, etc.
Let's look at programming languages again. Take Python as an example, look at the elements in Python.
Name | Symbol | Function |
---|---|---|
Logical operators | and or not | Binary and unary logic functions |
Arithmetic Operators | +, -, *, /, %, **, // | Binary Functions |
Comparison operators | ==, !=, >, <, >=, <= | Binary functions |
... |
Taking the floating-point number addition operator (+) as an example, the function image is as follows:
You can define your own functions in Python:
def f(x, y): return max(0, 2*x + 3*y - 3)
A new function is defined here. It uses "+, -, *, max" and other functions to construct the new function. The method of construction is also through the composition of functions.
Functions are everywhere in programming languages. By combining basic functions, new functions can be constructed and new algorithms can be obtained.
Neural network is also function. Like digital circuits and programming languages, it is also composed of simple functions. The basic units of digital circuits are logical functions such as AND, OR, and NOT. The basic units in programming languages are functions such as various operators, while the basic unit of neural networks is neurons.
So what is a neuron? A biological neuron is a cell with input dendrites and output axons. And the neuron on the neural network is an artificial neuron, it is also a function, more precisely, it is a kind of function.
The number of inputs of neurons can be changed, which means that it represents a -element function , and can be different for different neurons.
Neurons combine with each other to form a neural network. As shown below:
A neural network is a function composed of neuron functions. A neuron function is a function composed of simple functions, such as AND, OR, and NOT.
The neural network contains three neurons(do not count input neurons):
The function represented by the neural network is:
Just knowing that it is a -element function is not enough. The basic units AND, OR, and NOT gates in digital circuits all list the truth table and draw the figure, but what about the neuron?