Off-Ramp

Beyond The Academy

The academy is a floor, not a ceiling. When the six-phase ladder feels mechanical and the exit gates are boring, your next edges are outside this repo: harder math, real papers, modern model families, and actual research or production work. This page is the off-ramp.

Study Plan Tracks Decision Clinics Back To Home

Math Refresh

Linear Algebra, Probability, Optimization

The academy teaches judgment. Research demands the math underneath. Short pointers into canonical sources, not re-teaching.

Paper Skill

Read, Reproduce, Extract

A trained researcher reads a paper, pulls out the one workflow decision the authors made, and reproduces the result. Practice the skill, not just the reading.

Modern Surface

What The Academy Skipped

Retrieval, vision-language models, reinforcement learning, diffusion, large-scale training. Survey-level pointers into each area.

You Should Be On This Page If

• every exit gate in the Study Plan feels easy
• you can pass a timed checkpoint sheet cold
• you can finish the mock task drill in IOAI Competition Surface without panic
• your real question is "what do I do that is still hard?"

If any of those feel wobbly, go back to the matching phase. The off-ramp is not a reward for reading the academy — it is the next place to go after the academy has stopped stretching you.

Math Refresh Pointers

The academy deliberately teaches workflow judgment over math. At some point, the math becomes load-bearing again — when you read papers, when you have to reason about why a method works, when you design something new.

Treat these as the canonical starting points, not the only ones:

• Linear algebra. Gilbert Strang's MIT 18.06 lectures remain the best introduction. For a faster refresh aimed at ML readers, chapter 2 of the Deep Learning book by Goodfellow, Bengio, Courville is compact and sufficient.
• Probability and statistics. MIT 6.041 (Bertsekas) for a solid foundation, or Blitzstein's Stat 110 at Harvard if you prefer the dense textbook route.
• Optimization. Boyd & Vandenberghe's Convex Optimization is the reference. You do not need all of it — chapters 1–3 and 9–10 cover what you will actually see in ML papers.
• Matrix calculus for ML. Parr & Howard's "The Matrix Calculus You Need For Deep Learning" is a short paper that covers the gradient manipulations that confuse most self-taught readers.

Do not try to read all four at once. Pick the one whose weakness just bit you on a paper or a project, and work through only the sections that unstick you. Math you read without a problem to apply it to evaporates.

Paper-Reading Skill

Reading ML papers well is its own practice. The academy's philosophy applies directly — decide first, inspect second, defend third.

A first pass on any paper should answer:

1. What problem are they solving, and what was the previous best approach?
2. What is the single workflow decision they changed from the baseline?
3. What evidence do they present that the change mattered? Is the evidence honest — do they compare on the same held-out set, control for compute, report variance?
4. What is the one failure mode the paper admits?
5. What would you need to reproduce the result?

If you cannot answer (2) in one sentence, you have not understood the paper. Authors are usually proposing one idea. Everything else is scaffolding.

Practice routine. Pick one paper a week from a credible venue — NeurIPS, ICML, ICLR, ACL, CVPR. Aim for one that has a public code release. Read it end to end once in under 40 minutes, then answer the five questions above in writing. Only then clone the repo and try to run the main result on a small dataset.

Starting venues:

• Papers With Code — browse by benchmark, see papers with runnable code
• arXiv sanity — aggregated view of recent arXiv submissions
• Andrej Karpathy's reading list and lectures — a good calibration for what "a strong person reads" looks like
• Distill (archived but intact) — for beautifully explained foundational ideas

Reproduction targets. Once you have done a handful of paper-reads, pick one and reproduce the headline result on a small slice. This is the single fastest way to discover how much of a paper is scaffolding you can safely ignore and how much was doing actual work.

Modern Surface — Now Covered Inside The Academy

Many areas that used to live only as pointers on this page are now first-class academy topics. Start inside the academy and use the external links below only as deeper dives.

Large Language Models, Retrieval, And Prompting

• Retrieval-Augmented Generation — the academy's treatment of RAG: chunking, retrieval families, two-loop evaluation
• Text Generation and Language Models — decoding, CoT, function-calling, evaluation stack
• Prompting and Tool Use — the prompt contract, few-shot discipline, agent loops, prompt injection

Deeper dives:

• Jay Alammar's "Illustrated Transformer" — the standard visual explanation
• HuggingFace's LLM course — practical, runnable
• Lewis et al. "Retrieval-Augmented Generation" — the original RAG paper
• Anthropic's "Contextual Retrieval" — a modern production RAG baseline

Vision-Language Models And Self-Supervised Learning

• Self-Supervised and Representation Learning — contrastive, masked modeling, CLIP as cross-modal contrastive, linear-probe evaluation
• Vision and Text Encoders — using pretrained encoders downstream

Deeper dives:

• CLIP — the paper that made joint image-text training normal
• Flamingo — the paper that made instruction-following VLMs normal
• HuggingFace's multimodal tutorials — runnable entry points

Reinforcement Learning

• Reinforcement Learning Foundations — vocabulary, REINFORCE, PPO, reward hacking, RLHF

Deeper dives:

• Sutton & Barto's Reinforcement Learning: An Introduction — free PDF, the canonical textbook
• Spinning Up in Deep RL (OpenAI) — practical, code-first
• RLHF specifically: Ouyang et al. "Training language models to follow instructions"

Diffusion And Generative Models

• Diffusion Models — forward / reverse process, DDPM training, DDIM sampling, classifier-free guidance
• Autoencoders and VAEs — the generative-modeling bridge; the first stage of latent diffusion
• Generative Adversarial Networks — the fast-sampling alternative and its characteristic failures

Deeper dives:

• Ho et al. "Denoising Diffusion Probabilistic Models"
• Karras et al. "Elucidating the Design Space of Diffusion-Based Generative Models" — cleaned-up design space
• Lilian Weng's diffusion post — the best explainer

Scaling, Efficient Training, And Deployment

• PEFT and LoRA — LoRA math, target-layer choice, QLoRA, adapter merging
• Mixture-of-Experts and Scaling — MoE routing, load balance, Chinchilla scaling
• Quantization, Distillation, and Deployment — PTQ/QAT, distillation, pruning, deployment pipelines

Deeper dives:

• Chinchilla — scaling laws for LLMs
• LoRA — the low-rank fine-tuning paper that made PEFT practical
• EleutherAI's training cookbook — if you are going to train something non-trivial

Graphs

• Graph Neural Networks — message passing, GCN/GAT/SAGE, over-smoothing, inductive vs. transductive splits

Off-Ramps That Are Not Reading

Reading more is not always the next step. At this level, the single biggest accelerator is doing real work with feedback from people better than you.

• Research internship or research assistantship. Email researchers whose papers you reproduced. Send the reproduction, not a cold CV. This works more often than students expect.
• Kaggle competitions. Pick a real competition (not a playground) and commit to finishing it. Read the top-5 write-ups afterwards — that is where the real teaching happens.
• Open-source contribution. HuggingFace, PyTorch Lightning, and scikit-learn all have tagged "good first issue" labels. Fixing a real bug in a serious library will teach you more about the stack than any course.
• Teach something. Run a study group for the phase you just finished. Nothing exposes shallow understanding like explaining to someone who will ask exactly the question you cannot answer.

Maintenance Back At The Academy

Even after you move past the phases, the academy stays useful as a judgment gym:

• one Decision Clinic a week keeps the commit-before-reveal muscle alive
• one timed checkpoint sheet a month catches decay in specific reasoning
• one mock task from IOAI Competition Surface every six to eight weeks recalibrates your speed

If those three routines feel like overkill, you are probably ready to stop using the academy at all. Good — go make something.