# Ai Foundations
All Ai Foundations notes →1. The Evolution of Artificial Intelligence
Traces the arc from symbolic AI and expert systems through statistical ML, deep learning, and the scaling-law-driven emergence of foundation models, framing why agentic AI is the current inflection point rather than a fresh discipline.
2. Machine Learning Fundamentals
Covers supervised vs. unsupervised vs. reinforcement learning, the bias-variance tradeoff, loss functions, and gradient descent as the fundamentals that still govern how modern LLMs are trained and fine-tuned.
3. Deep Learning Essentials
Covers neural network building blocks — layers, activation functions, backpropagation, regularization, and optimizers — as the substrate transformers are built on, explained from first principles for a staff-level interview bar.
4. Transformer Architecture
Breaks down the encoder-decoder transformer — self-attention, multi-head attention, positional encoding, and feed-forward blocks — and why this architecture displaced RNNs and LSTMs as the default for sequence modeling at scale.
5. Tokens, Embeddings & Attention
Explains how raw text becomes tokens, how tokens become dense embedding vectors, and how the attention mechanism computes contextual relevance between them — the three concepts most commonly conflated in interview answers.
6. Context Windows & Tokenization
Covers tokenizer algorithms (BPE, WordPiece, SentencePiece), context window sizing and its quadratic attention-cost tradeoff, and practical strategies — chunking, sliding windows, summarization — for working within a fixed context budget.
7. Foundation Models
Defines what makes a model foundational — pretraining scale, transfer learning, and emergent capabilities — and surveys major foundation model families and the positioning tradeoffs between them.
8. Large Language Models
Covers the LLM training pipeline end to end — pretraining, supervised fine-tuning, and RLHF/DPO alignment — and the resulting capability, cost, and latency tradeoffs an architect weighs when picking a model for production.
9. Reasoning Models
Covers chain-of-thought and inference-time compute scaling in reasoning models, how they differ architecturally and operationally from standard next-token LLMs, and when the added latency and cost is actually justified.
10. The AI Ecosystem
Maps the current AI ecosystem — model providers, orchestration frameworks, vector databases, evaluation tooling, and inference infrastructure — as the landscape an agentic system architect has to navigate.
11. Probability, Sampling & Decoding
The math intuition underneath every model call — how a raw logit vector becomes a probability distribution, why temperature and top-p reshape that distribution differently, why beam search lost to sampling for chat and agent models, and how entropy and KL divergence turn 'the model is uncertain' and 'alignment training' into something you can actually reason about.
12. Vector Geometry & Similarity
The geometric intuition behind embeddings -- cosine similarity as angle versus Euclidean distance as magnitude, why unnormalized dot products silently bias search, and why high-dimensional spaces make naive nearest-neighbor search break down.