here is a collection of resources that were particularly helpful for me when i was learning

• 3b1b

  • essence of linear algebra
  • essence of calculus
  • neural networks

• andrej karpathy

  • The spelled-out intro to neural networks and backpropagation: building micrograd
  • The spelled-out intro to language modeling: building makemore
  • Building makemore Part 2: MLP
  • Building makemore Part 3: Activations & Gradients, BatchNorm
  • Building makemore Part 4: Becoming a Backprop Ninja
  • Let's build GPT: from scratch, in code, spelled out.
  • Let's build the GPT Tokenizer
  • Let's reproduce GPT-2 (124M)

• nick

  • cuda crash course

• probability theory

  • Are you Bayesian or Frequentist?

• books

  • mathematics for machine learning

• podcasts

  • lex fridman
    • george hotz
      • George Hotz: Comma.ai, OpenPilot, and Autonomous Vehicles | Lex Fridman Podcast #31
      • George Hotz: Hacking the Simulation & Learning to Drive with Neural Nets | Lex Fridman Podcast #132
      • George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God | Lex Fridman Podcast #387
    • chris lattner
      • Chris Lattner: Future of Programming and AI | Lex Fridman Podcast #381

• blog posts

  • How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance: a Worklog

• semianalytics

• ring reduce algorithm

• beam search

• mpi course

• brief history of intelligence (book)