Neural Networks Explained from Scratch using Python

This video provides a complete walkthrough of building a fully connected neural network from scratch to classify handwritten digits from the MNIST dataset. It explains key concepts such as weight matrix conventions, the addition of a dedicated bias neuron, and one-hot encoding labels for multi-class output. The training process is detailed through nested loops over epochs and image-label pairs, using forward propagation with a sigmoid activation and backpropagation to adjust weights based on mean-squared error. A notable simplification in the output-layer delta calculation is highlighted, while hidden layers require the sigmoid derivative for error propagation. The final trained network achieves over 93% accuracy and correctly identifies a handwritten "three," with the full code made publicly available for viewers.

• ▶ 0:18 A neural network is built from neurons organized into layers—input, hidden, and output—and when every neuron in one layer connects to every neuron in the next, the layers are fully connected.
• ▶ 2:14 Weight matrices are shaped from the right layer to the left layer (e.g., 4×5 for connecting 5 inputs to 4 hidden neurons), a convention that allows cleaner and faster computations later.
• ▶ 2:55 A dedicated bias neuron, always set to 1 and initialized to 0, is added so the network can shift functions up or down and avoid starting with an unintentional bias.
• ▶ 4:48 The images and labels come from the MNIST dataset: 60,000 handwritten 28×28 grayscale images, known as the “Hello World” dataset for machine learning.
• ▶ 5:32 Loading the data in Python gives the images a shape of 60000×784, while the labels are shaped 60000×10 rather than 60000×1.
• ▶ 6:00 Because there are more than two possible outputs, labels are one-hot encoded into binary vectors—e.g., an image labeled 3 has a vector where only the fourth output neuron should be 1.
• ▶ 6:57 Training runs in nested loops: an inner loop over all image-label pairs and an outer loop over epochs, so with 3 epochs each step is repeated 3 times for all 60,000 images.
• ▶ 8:15 Forward propagation reshapes input vectors into matrices, multiplies inputs by weights, adds bias weights, and applies a sigmoid activation to normalize hidden layer values between 0 and 1.
• ▶ 10:51 Outputs are compared to the label using mean-squared error, and accuracy is checked by seeing which output neuron has the highest value — though this accuracy check does not affect training.
• ▶ 12:00 Backpropagation is the core algorithm for learning: after computing error, it propagates that error backward from the output layer to determine how much each weight contributed and how to adjust them.
• ▶ 12:49 For the mean squared error cost function, the output-layer delta simplifies to just output – label, so the error value is not needed at this step (though it becomes necessary for other cost functions).
• ▶ 14:26 For hidden layers, the delta calculation requires the derivative of the sigmoid activation (h * (1 - h)), obtained by transposing the updated weight matrix and multiplying with output deltas; these steps repeat across all hidden layers until every weight is updated.
• ▶ 15:31 The trained neural network is run and achieves over 93% accuracy, described as “quite good!”
• ▶ 16:38 The network is used in action and correctly identifies the digit “three,” demonstrating successful inference.
• ▶ 16:56 The full code from the video is made available to everyone via a link in the description.

• ▶ 0:00 Neural Network Architecture and Weights (0:00 - 3:58) - - Explains neurons, fully connected layers, weight matrices, and the bias neuron.
• ▶ 3:58 Data Preparation and Encoding (3:58 - 6:57) - - Covers training images, the MNIST dataset, loading data in Python, and one-hot encoding.
• ▶ 6:57 Training Loops and Forward Propagation (6:57 - 12:06) - - Shows training loops, reshaping vectors, forward propagation, and the cost/error function.
• ▶ 12:06 Backpropagation (12:06 - 15:27) - - Details output-layer deltas, weight updates, bias weights, and hidden-layer updates.
• ▶ 15:21 Running and Using the Network (15:21 - 17:18) - - Runs the trained network, shows accuracy, and demonstrates using it in action.