Tommaso Vaccari
Back to blog
Series

Neural Networks

6 published parts, in reading order.

Read the series introduction

  1. 1
    Neural Networks: Introduction

    Introduction to Neural Networks.

  2. 2
    Neural Networks: Linear Regression

    From math to code: build linear regression from scratch and see how it connects to neural networks.

  3. 3
    From Bigrams to Neural Networks: The First Step in Language Modeling

    A hands-on walkthrough from count-based bigrams to a simple neural model.

  4. 4
    Neural Network Training: Statistics, Initialization and Diagnostics

    A structured study of neural network training dynamics, covering statistical preliminaries, loss interpretation, saturation, gradient flow, initialization schemes, diagnostic tools, and stabilization mechanisms.

  5. 5
    From Self-Attention to a Small GPT

    A personal from-scratch derivation of causal self-attention, starting from a character language model and ending with the core pieces of a small GPT.

  6. 6
    Deep Dive into a Custom GPT Architecture

    A from-scratch GPT implementation, its computational cost, and experiments on arithmetic and character-level language modeling.