Module 1: Neural Network Foundations

From Neuron to Network

Neural networks power image recognition, speech transcription, translation and many recommendation systems. Behind the impressive results is a surprisingly simple building block: an artificial neuron that multiplies inputs by weights, adds them up and passes the result through a function. In this lesson you will understand that building block precisely, then see how stacking many of them produces a network that can learn complex patterns.

The artificial neuron

A single neuron takes several numeric inputs, for example three facts about a loan applicant: monthly income, existing debt and years at current job. Each input x has a weight w that says how much it matters and in which direction. The neuron also has a bias b, a constant that shifts the result. It computes a weighted sum:

z = w1*x1 + w2*x2 + w3*x3 + b

It then applies an activation function to z, giving the neuron's output. A common choice for a final yes or no decision is the sigmoid function, which squeezes any number into the range 0 to 1 so it can be read as a probability.

import numpy as np

def sigmoid(z):
    return 1 / (1 + np.exp(-z))

x = np.array([0.8, 0.3, 0.5])      # scaled inputs: income, debt, tenure
w = np.array([1.5, -2.0, 0.7])     # weights
b = -0.2                           # bias

z = np.dot(w, x) + b               # 1.2 - 0.6 + 0.35 - 0.2 = 0.75
output = sigmoid(z)                # about 0.68

A positive weight pushes the output up when the input grows; a negative weight pushes it down. Learning, which you will study shortly, simply means finding weights and biases that make the outputs useful.

From one neuron to a layer

One neuron can only draw a straight-line boundary between classes. To capture richer patterns you place many neurons side by side in a layer. Each neuron in the layer sees the same inputs but has its own weights, so each can detect a different combination of features. In matrix form, a whole layer is one multiplication:

X = np.random.rand(4, 3)        # 4 examples, 3 features each
W = np.random.randn(3, 5)       # 3 inputs to 5 neurons
b = np.zeros(5)

Z = X @ W + b                   # shape (4, 5): 5 outputs for each example
A = np.maximum(0, Z)            # ReLU activation, explained in the next lesson

This is why deep learning relies on fast matrix arithmetic, and why graphics processors, which are built for exactly that, speed it up so much.

Stacking layers into a network

A feed-forward network, also called a multilayer perceptron, chains layers together. The outputs of one layer become the inputs of the next:

• The input layer is simply your features, such as pixel values or customer attributes.
• One or more hidden layers transform the data step by step into more useful representations.
• The output layer produces the prediction: one number for a price, a probability for yes or no, or a probability for each of several classes.

"Deep" learning simply means using several hidden layers. Early layers tend to learn simple features, and later layers combine them into more abstract ones. In an image model, early layers respond to edges, middle layers to textures and shapes, and later layers to whole objects.

Worked example: counting parameters

Suppose you build a network to classify 28 by 28 pixel greyscale images of handwritten digits into 10 classes. You flatten each image into 784 inputs, use one hidden layer of 128 neurons and an output layer of 10 neurons. How many numbers must the network learn?

Over a hundred thousand parameters for a small network shows why neural networks need plenty of data: every parameter must be estimated from examples. Keras reports this same count when you call model.summary(), which you will do in Module 2.

What a neural network is not

Despite the name, artificial neurons are a loose mathematical inspiration from biology, not a model of the brain. A network does not understand its task; it finds numerical patterns that map inputs to outputs in its training data. That is why networks can be confidently wrong on inputs unlike anything they were trained on.

Common mistakes to avoid

• Feeding unscaled inputs. An income in tens of thousands next to a ratio between 0 and 1 makes learning slow and unstable. Scale features to similar ranges.
• Assuming bigger is always better. More layers and neurons mean more parameters to learn and more risk of memorising the training data.
• Ignoring shapes. Most beginner errors are shape mismatches. Write down the shape of the data at every layer.
• Using deep learning for small tabular problems where simpler models such as gradient boosting often perform as well and are easier to explain.