Beginning of Artificial Neural Network: Perceptron

Beginning of Artificial Neural Network: Perceptron

Frank Rosenblatt invented perceptron in 1958 at Cornell University. The perceptron was an attempt to recognize characters using neural network procedures. The perceptron was a linear method that was useful for resolving problems in which the input groups could be separated linearly in the input space. It is one of the first and most basic forms of artificial neural networks. It isn’t “deep” learning, but it is a crucial starting point.

A mathematical representation of a biological neuron is the perceptron. Although human neurons’ dendrite accepts electrical signals from other neurons’ axons, these electrical signals are expressed numerically in the perceptron. Electrical signals are modified in varying quantities at synapses between dendrites and axons. In the perceptron, this is also represented by multiplying each input value by a weight value. Only when the input signals’ cumulative intensity reaches a certain specific does a real neuron fire an output signal. Calculating the weighted sum of the inputs to show the input signals’ cumulative intensity and assigning a phase function to the sum to calculate its output is how this phenomenon in perception is modeled. This performance is passed to other perceptrons, much as in biological neural networks.

What can a perceptron do?

A perceptron, as previously said, measures the weighted sum of the input values. For the sake of convenience, suppose that a perceptron P has two input values, x1 and x2. Let’s assume the weights for x1 and x2 are w1 and w2, respectively, and the weighted sum is w1 x1 + w2 x2. So w1 x1 is input one, and w2x2 is input two.

Since the perceptron only produces a non-zero value when the weighted sum reaches a certain threshold y, the performance of this perception can be written down to the following:

It determines a weighted sum of its inputs and thresholds it with a phase function, as we can see. In terms of geometry, this implies that the perceptron will use a hyperplane to divide its input space. That’s where the idea that a perceptron can only isolate issues that are linearly separable came from. Since the XOR equation is not linearly separable, separating it with a single hyperplane is impractical. Because of the lacking ability to solve non-linear problems of perceptron, new approaches have emerged, and the foundations of the complex deep learning models we use today have begun to be laid.

Source

Jain, A. K., Jianchang Mao, & Mohiuddin, K. M. (1996). Artificial neural networks: a tutorial. Computer, 29(3), 31–44. https://doi.org/10.1109/2.485891 

Basheer, I. A., & Hajmeer, M. (2000). Artificial neural networks: fundamentals, computing, design, and application. Journal of Microbiological Methods, 43(1), 3–31. https://doi.org/10.1016/s0167-7012(00)00201-3 

Widrow, B., & Lehr, M. A. (1990). 30 years of adaptive neural networks: perceptron, madaline, and backpropagation. Proceedings of the IEEE, 78(9), 1415-1442.