Skip to content
Road to Intelligence

Papers

The family tree

Research is a conversation across decades. Each line joins a paper to one it built on. Pick a paper to light up its ancestors (blue) and descendants (orange). Rows follow the chapter where each paper is discussed.
1. What Is Artificial Intelligence?2. Math Toolkit3. Machine Learning4. Neural Networks5. Vision, Speech & Reinforcement Learning6. Language Before Transformers7. Transformers8. Rise of Large Language Models9. How an LLM Is Actually Built10. From Base Model to Assistant11. Inside Modern LLMs12. Embeddings, RAG & the LLM Application Stack13. Agents14. Reasoning Models15. Multimodal AI16. Evaluation, Reliability, Safety & InterpretabilityBeyond (later chapters)19011943194819501955195819621966196819721976198019821986198919921994199520012003200620092010201520202025Bengio et al. 2003: A Neural Probabilistic Language ModelMikolov et al. 2013: Efficient Estimation of Word Representations in Vector SpaceMikolov et al. 2013: Distributed Representations of Words and Phrases and their CompositionalityPennington et al. 2014: GloVe: Global Vectors for Word RepresentationCho et al. 2014: Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine TranslationSutskever et al. 2014: Sequence to Sequence Learning with Neural NetworksSutskever et al. 2014Bahdanau et al. 2014: Neural Machine Translation by Jointly Learning to Align and TranslateBahdanau et al. 2014Vaswani et al. 2017: Attention Is All You NeedVaswani et al. 2017He et al. 2015: Deep Residual Learning for Image RecognitionHe et al. 2015Ba et al. 2016: Layer NormalizationBa et al. 2016Devlin et al. 2018: BERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingDevlin et al. 2018Raffel et al. 2019: Exploring the Limits of Transfer Learning with a Unified Text-to-Text TransformerRaffel et al. 2019Xiong et al. 2020: On Layer Normalization in the Transformer ArchitectureXiong et al. 2020Sennrich et al. 2015: Neural Machine Translation of Rare Words with Subword UnitsBrown et al. 2020: Language Models are Few-Shot LearnersBrown et al. 2020Kaplan et al. 2020: Scaling Laws for Neural Language ModelsKaplan et al. 2020Hoffmann et al. 2022: Training Compute-Optimal Large Language ModelsKingma et al. 2014: Adam: A Method for Stochastic OptimizationShannon 1948: A Mathematical Theory of CommunicationRobbins et al. 1951: A Stochastic Approximation MethodKullback et al. 1951: On Information and SufficiencyRuder 2016: An overview of gradient descent optimization algorithmsLoshchilov et al. 2017: Decoupled Weight Decay RegularizationMcCulloch et al. 1943: A logical calculus of the ideas immanent in nervous activityTuring 1950: Computing Machinery and IntelligenceMcCarthy et al. 1955: A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence, August 31, 1955Rosenblatt 1958: The perceptron: A probabilistic model for information storage and organization in the brain.Samuel 1959: Some Studies in Machine Learning Using the Game of CheckersWeizenbaum 1966: ELIZA—a computer program for the study of natural language communication between man and machineHart et al. 1968: A Formal Basis for the Heuristic Determination of Minimum Cost PathsNewell et al. 1976: Computer science as empirical inquiryRumelhart et al. 1986: Learning representations by back-propagating errorsKrizhevsky et al. 2012: ImageNet Classification with Deep Convolutional Neural NetworksMnih et al. 2015: Human-level control through deep reinforcement learningSilver et al. 2016: Mastering the game of Go with deep neural networks and tree searchChristiano et al. 2017: Deep reinforcement learning from human preferencesRadford et al. 2021: Learning Transferable Visual Models From Natural Language SupervisionRadford et al. 2021Wei et al. 2022: Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsOuyang et al. 2022: Training language models to follow instructions with human feedbackDeepSeek-AI et al. 2025: DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement LearningDomingos 2012: A few useful things to know about machine learningKaufman et al. 2012: Leakage in data miningCortes et al. 1995: Support-vector networksBreiman 2001: Random ForestsTibshirani 1996: Regression Shrinkage and Selection Via the LassoPearson 1901: On lines and planes of closest fit to systems of points in spaceLloyd 1982: Least squares quantization in PCMHornik et al. 1989: Multilayer feedforward networks are universal approximatorsCybenko 1989: Approximation by superpositions of a sigmoidal functionBengio et al. 1994: Learning long-term dependencies with gradient descent is difficultHochreiter et al. 1997: Long Short-Term MemoryLeCun et al. 1998: Gradient-based learning applied to document recognitionGlorot et al. 2010: Understanding the difficulty of training deep feedforward neural networksSrivastava et al. 2014: Dropout: A Simple Way to Prevent Neural Networks from OverfittingIoffe et al. 2015: Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftHe et al. 2015: Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet ClassificationHendrycks et al. 2016: Gaussian Error Linear Units (GELUs)Hubel et al. 1962: Receptive fields, binocular interaction and functional architecture in the cat's visual cortexFukushima 1980: Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in positionLeCun et al. 1989: Backpropagation Applied to Handwritten Zip Code RecognitionDeng et al. 2009: ImageNet: A large-scale hierarchical image databaseRen et al. 2015: Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksRonneberger et al. 2015: U-Net: Convolutional Networks for Biomedical Image SegmentationDosovitskiy et al. 2020: An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleDosovitskiy et al. 2020Graves et al. 2006: Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networksHinton et al. 2012: Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research GroupsOord et al. 2016: WaveNet: A Generative Model for Raw AudioRadford et al. 2022: Robust Speech Recognition via Large-Scale Weak SupervisionRadford et al. 2022Sutton et al. 2018: Reinforcement Learning: An Introduction (2nd edition)Watkins et al. 1992: Q-learningWilliams 1992: Simple statistical gradient-following algorithms for connectionist reinforcement learningMnih et al. 2016: Asynchronous Methods for Deep Reinforcement LearningSchulman et al. 2017: Proximal Policy Optimization AlgorithmsPeters et al. 2018: Deep contextualized word representationsHoward et al. 2018: Universal Language Model Fine-tuning for Text ClassificationRadford et al. 2018: Improving Language Understanding by Generative Pre-TrainingRadford et al. 2018Radford et al. 2019: Language Models are Unsupervised Multitask LearnersFan et al. 2018: Hierarchical Neural Story GenerationHoltzman et al. 2019: The Curious Case of Neural Text DegenerationBommasani et al. 2021: On the Opportunities and Risks of Foundation ModelsWei et al. 2021: Finetuned Language Models Are Zero-Shot LearnersMin et al. 2022: Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?Olsson et al. 2022: In-context Learning and Induction HeadsWei et al. 2022: Emergent Abilities of Large Language ModelsSchaeffer et al. 2023: Are Emergent Abilities of Large Language Models a Mirage?Delétang et al. 2023: Language Modeling Is CompressionTouvron et al. 2023: LLaMA: Open and Efficient Foundation Language ModelsOpenAI et al. 2023: GPT-4 Technical ReportBroder 1997: On the resemblance and containment of documentsSchuster et al. 2012: Japanese and Korean voice searchChen et al. 2016: Training Deep Nets with Sublinear Memory CostLoshchilov et al. 2016: SGDR: Stochastic Gradient Descent with Warm RestartsGoyal et al. 2017: Accurate, Large Minibatch SGD: Training ImageNet in 1 HourMicikevicius et al. 2017: Mixed Precision TrainingKudo et al. 2018: SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text ProcessingHuang et al. 2018: GPipe: Efficient Training of Giant Neural Networks using Pipeline ParallelismKalamkar et al. 2019: A Study of BFLOAT16 for Deep Learning TrainingShoeybi et al. 2019: Megatron-LM: Training Multi-Billion Parameter Language Models Using Model ParallelismShoeybi et al. 2019Rajbhandari et al. 2019: ZeRO: Memory Optimizations Toward Training Trillion Parameter ModelsWenzek et al. 2019: CCNet: Extracting High Quality Monolingual Datasets from Web Crawl DataGao et al. 2020: The Pile: An 800GB Dataset of Diverse Text for Language ModelingNarayanan et al. 2021: Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMDodge et al. 2021: Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled CorpusLee et al. 2021: Deduplicating Training Data Makes Language Models BetterChowdhery et al. 2022: PaLM: Scaling Language Modeling with PathwaysZhang et al. 2022: OPT: Open Pre-trained Transformer Language ModelsKorthikanti et al. 2022: Reducing Activation Recomputation in Large Transformer ModelsZhao et al. 2023: PyTorch FSDP: Experiences on Scaling Fully Sharded Data ParallelMuennighoff et al. 2023: Scaling Data-Constrained Language ModelsGunasekar et al. 2023: Textbooks Are All You NeedSardana et al. 2023: Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling LawsBesiroglu et al. 2024: Chinchilla Scaling: A replication attemptPenedo et al. 2024: The FineWeb Datasets: Decanting the Web for the Finest Text Data at ScaleGrattafiori et al. 2024: The Llama 3 Herd of ModelsStiennon et al. 2020: Learning to summarize from human feedbackSanh et al. 2021: Multitask Prompted Training Enables Zero-Shot Task GeneralizationBai et al. 2022: Training a Helpful and Harmless Assistant with Reinforcement Learning from Human FeedbackGao et al. 2022: Scaling Laws for Reward Model OveroptimizationBai et al. 2022: Constitutional AI: Harmlessness from AI FeedbackWang et al. 2022: Self-Instruct: Aligning Language Models with Self-Generated InstructionsZhou et al. 2023: LIMA: Less Is More for AlignmentRafailov et al. 2023: Direct Preference Optimization: Your Language Model is Secretly a Reward ModelCui et al. 2023: UltraFeedback: Boosting Language Models with Scaled AI FeedbackSharma et al. 2023: Towards Understanding Sycophancy in Language ModelsTunstall et al. 2023: Zephyr: Direct Distillation of LM AlignmentLambert et al. 2024: Tulu 3: Pushing Frontiers in Open Language Model Post-TrainingPope et al. 2022: Efficiently Scaling Transformer InferenceShazeer 2019: Fast Transformer Decoding: One Write-Head is All You NeedShazeer 2019Ainslie et al. 2023: GQA: Training Generalized Multi-Query Transformer Models from Multi-Head CheckpointsDao et al. 2022: FlashAttention: Fast and Memory-Efficient Exact Attention with IO-AwarenessDao et al. 2022Dao 2023: FlashAttention-2: Faster Attention with Better Parallelism and Work PartitioningSu et al. 2021: RoFormer: Enhanced Transformer with Rotary Position EmbeddingSu et al. 2021Zhang et al. 2019: Root Mean Square Layer NormalizationShazeer 2020: GLU Variants Improve TransformerShazeer 2020Shazeer et al. 2017: Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts LayerFedus et al. 2021: Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient SparsityJiang et al. 2024: Mixtral of ExpertsDeepSeek-AI et al. 2024: DeepSeek-V3 Technical ReportDettmers et al. 2022: LLM.int8(): 8-bit Matrix Multiplication for Transformers at ScaleFrantar et al. 2022: GPTQ: Accurate Post-Training Quantization for Generative Pre-trained TransformersLin et al. 2023: AWQ: Activation-aware Weight Quantization for LLM Compression and AccelerationDettmers et al. 2023: QLoRA: Efficient Finetuning of Quantized LLMsHu et al. 2021: LoRA: Low-Rank Adaptation of Large Language ModelsHoulsby et al. 2019: Parameter-Efficient Transfer Learning for NLPHinton et al. 2015: Distilling the Knowledge in a Neural NetworkSanh et al. 2019: DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighterHan et al. 2015: Learning both Weights and Connections for Efficient Neural NetworksFrankle et al. 2018: The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural NetworksFrantar et al. 2023: SparseGPT: Massive Language Models Can Be Accurately Pruned in One-ShotLeviathan et al. 2022: Fast Inference from Transformers via Speculative DecodingChen et al. 2023: Accelerating Large Language Model Decoding with Speculative SamplingKwon et al. 2023: Efficient Memory Management for Large Language Model Serving with PagedAttentionTouvron et al. 2023: Llama 2: Open Foundation and Fine-Tuned Chat ModelsChen et al. 2023: Extending Context Window of Large Language Models via Positional InterpolationMicikevicius et al. 2022: FP8 Formats for Deep LearningJones 1972: A Statistical Interpretation of Term Specificity and Its Application in RetrievalRobertson et al. 2009: The Probabilistic Relevance Framework: BM25 and BeyondCormack et al. 2009: Reciprocal Rank Fusion Outperforms Condorcet and Individual Rank Learning MethodsJégou et al. 2011: Product Quantization for Nearest Neighbor SearchMalkov et al. 2016: Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphsJohnson et al. 2017: Billion-scale similarity search with GPUsChen et al. 2017: Reading Wikipedia to Answer Open-Domain QuestionsOord et al. 2018: Representation Learning with Contrastive Predictive CodingNogueira et al. 2019: Passage Re-ranking with BERTReimers et al. 2019: Sentence-BERT: Sentence Embeddings using Siamese BERT-NetworksPetroni et al. 2019: Language Models as Knowledge Bases?Guu et al. 2020: REALM: Retrieval-Augmented Language Model Pre-TrainingKarpukhin et al. 2020: Dense Passage Retrieval for Open-Domain Question AnsweringKhattab et al. 2020: ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTLewis et al. 2020: Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksThakur et al. 2021: BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval ModelsBorgeaud et al. 2021: Improving language models by retrieving from trillions of tokensMuennighoff et al. 2022: MTEB: Massive Text Embedding BenchmarkKandpal et al. 2022: Large Language Models Struggle to Learn Long-Tail KnowledgeWang et al. 2022: Text Embeddings by Weakly-Supervised Contrastive Pre-trainingGao et al. 2022: Precise Zero-Shot Dense Retrieval without Relevance LabelsMallen et al. 2022: When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric MemoriesRam et al. 2023: In-Context Retrieval-Augmented Language ModelsShi et al. 2023: Large Language Models Can Be Easily Distracted by Irrelevant ContextSchick et al. 2023: Toolformer: Language Models Can Teach Themselves to Use ToolsGreshake et al. 2023: Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt InjectionGao et al. 2023: Enabling Large Language Models to Generate Text with CitationsLiu et al. 2023: Lost in the Middle: How Language Models Use Long ContextsWillard et al. 2023: Efficient Guided Generation for Large Language ModelsEs et al. 2023: Ragas: Automated Evaluation of Retrieval Augmented GenerationXu et al. 2023: Retrieval meets Long Context Large Language ModelsJiang et al. 2023: LLMLingua: Compressing Prompts for Accelerated Inference of Large Language ModelsAsai et al. 2023: Self-RAG: Learning to Retrieve, Generate, and Critique through Self-ReflectionHsieh et al. 2024: RULER: What's the Real Context Size of Your Long-Context Language Models?Edge et al. 2024: From Local to Global: A Graph RAG Approach to Query-Focused SummarizationYao et al. 2022: ReAct: Synergizing Reasoning and Acting in Language ModelsShinn et al. 2023: Reflexion: Language Agents with Verbal Reinforcement LearningJimenez et al. 2023: SWE-bench: Can Language Models Resolve Real-World GitHub Issues?Yang et al. 2024: SWE-agent: Agent-Computer Interfaces Enable Automated Software EngineeringYao et al. 2024: $τ$-bench: A Benchmark for Tool-Agent-User Interaction in Real-World DomainsWu et al. 2023: AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent ConversationZhou et al. 2023: WebArena: A Realistic Web Environment for Building Autonomous AgentsCobbe et al. 2021: Training Verifiers to Solve Math Word ProblemsKojima et al. 2022: Large Language Models are Zero-Shot ReasonersWang et al. 2022: Self-Consistency Improves Chain of Thought Reasoning in Language ModelsZelikman et al. 2022: STaR: Bootstrapping Reasoning With ReasoningYao et al. 2023: Tree of Thoughts: Deliberate Problem Solving with Large Language ModelsLightman et al. 2023: Let's Verify Step by StepShao et al. 2024: DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language ModelsSnell et al. 2024: Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model ParametersTurpin et al. 2023: Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought PromptingGoodfellow et al. 2014: Generative Adversarial NetworksKingma et al. 2013: Auto-Encoding Variational BayesSohl-Dickstein et al. 2015: Deep Unsupervised Learning using Nonequilibrium ThermodynamicsOord et al. 2017: Neural Discrete Representation LearningHeusel et al. 2017: GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash EquilibriumYuksekgonul et al. 2022: When and why vision-language models behave like bags-of-words, and what to do about it?Ha et al. 2018: World ModelsHo et al. 2020: Denoising Diffusion Probabilistic ModelsSong et al. 2020: Denoising Diffusion Implicit ModelsSong et al. 2020: Score-Based Generative Modeling through Stochastic Differential EquationsRamesh et al. 2021: Zero-Shot Text-to-Image GenerationHessel et al. 2021: CLIPScore: A Reference-free Evaluation Metric for Image CaptioningDhariwal et al. 2021: Diffusion Models Beat GANs on Image SynthesisMeng et al. 2021: SDEdit: Guided Image Synthesis and Editing with Stochastic Differential EquationsKim et al. 2021: OCR-free Document Understanding TransformerZeghidour et al. 2021: SoundStream: An End-to-End Neural Audio CodecRombach et al. 2021: High-Resolution Image Synthesis with Latent Diffusion ModelsAlayrac et al. 2022: Flamingo: a Visual Language Model for Few-Shot LearningRamesh et al. 2022: Hierarchical Text-Conditional Image Generation with CLIP LatentsSaharia et al. 2022: Photorealistic Text-to-Image Diffusion Models with Deep Language UnderstandingHo et al. 2022: Classifier-Free Diffusion GuidanceHo et al. 2022: Video Diffusion ModelsReed et al. 2022: A Generalist AgentReed et al. 2022Borsos et al. 2022: AudioLM: a Language Modeling Approach to Audio GenerationLipman et al. 2022: Flow Matching for Generative ModelingDéfossez et al. 2022: High Fidelity Neural Audio CompressionBrooks et al. 2022: InstructPix2Pix: Learning to Follow Image Editing InstructionsPeebles et al. 2022: Scalable Diffusion Models with TransformersLi et al. 2023: BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsCarlini et al. 2023: Extracting Training Data from Diffusion ModelsHafner et al. 2023: Mastering Diverse Domains through World ModelsLiu et al. 2023: Visual Instruction TuningLi et al. 2023: Evaluating Object Hallucination in Large Vision-Language ModelsBrohan et al. 2023: RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic ControlLiu et al. 2023: Improved Baselines with Visual Instruction TuningYue et al. 2023: MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIBruce et al. 2024: Genie: Generative Interactive EnvironmentsTeam 2024: Chameleon: Mixed-Modal Early-Fusion Foundation ModelsKim et al. 2024: OpenVLA: An Open-Source Vision-Language-Action ModelOpenAI et al. 2024: GPT-4o System CardHendrycks et al. 2020: Measuring Massive Multitask Language UnderstandingLiang et al. 2022: Holistic Evaluation of Language ModelsZheng et al. 2023: Judging LLM-as-a-Judge with MT-Bench and Chatbot ArenaChiang et al. 2024: Chatbot Arena: An Open Platform for Evaluating LLMs by Human PreferenceMiller 2024: Adding Error Bars to Evals: A Statistical Approach to Language Model EvaluationsDubois et al. 2024: Length-Controlled AlpacaEval: A Simple Way to Debias Automatic EvaluatorsWang et al. 2023: Large Language Models are not Fair EvaluatorsSclar et al. 2023: Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingRein et al. 2023: GPQA: A Graduate-Level Google-Proof Q&A BenchmarkKiela et al. 2021: Dynabench: Rethinking Benchmarking in NLPPhan et al. 2025: Humanity's Last ExamGuo et al. 2017: On Calibration of Modern Neural NetworksKadavath et al. 2022: Language Models (Mostly) Know What They KnowLin et al. 2021: TruthfulQA: Measuring How Models Mimic Human FalsehoodsKalai et al. 2025: Why Language Models HallucinateMin et al. 2023: FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text GenerationGeifman et al. 2017: Selective Classification for Deep Neural NetworksGekhman et al. 2024: Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?Wei et al. 2023: Jailbroken: How Does LLM Safety Training Fail?Zou et al. 2023: Universal and Transferable Adversarial Attacks on Aligned Language ModelsPerez et al. 2022: Red Teaming Language Models with Language ModelsGanguli et al. 2022: Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons LearnedCarlini et al. 2020: Extracting Training Data from Large Language ModelsCarlini et al. 2022: Quantifying Memorization Across Neural Language ModelsNasr et al. 2023: Scalable Extraction of Training Data from (Production) Language ModelsAbadi et al. 2016: Deep Learning with Differential PrivacySzegedy et al. 2013: Intriguing properties of neural networksGoodfellow et al. 2014: Explaining and Harnessing Adversarial ExamplesShokri et al. 2016: Membership Inference Attacks against Machine Learning ModelsAlain et al. 2016: Understanding intermediate layers using linear classifier probesHewitt et al. 2019: Designing and Interpreting Probes with Control TasksElhage et al. 2022: Toy Models of SuperpositionMeng et al. 2022: Locating and Editing Factual Associations in GPTWang et al. 2022: Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallHase et al. 2023: Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language ModelsCunningham et al. 2023: Sparse Autoencoders Find Highly Interpretable Features in Language ModelsMarks et al. 2023: The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False DatasetsBurns et al. 2022: Discovering Latent Knowledge in Language Models Without SupervisionNanda et al. 2023: Progress measures for grokking via mechanistic interpretabilityElhage et al. 2021: A Mathematical Framework for Transformer CircuitsElhage et al. 2021Bricken et al. 2023: Towards Monosemanticity: Decomposing Language Models With Dictionary LearningTempleton et al. 2024: Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet

built on (ancestors) built on it (descendants)Larger dots: essential papers. Click a dot, or pick from the list.

2017 · essential

Attention Is All You Need

Introduced the Transformer — the architecture behind BERT, GPT and nearly every modern large language model, and later adapted to vision, audio and more.

Built directly on

Directly influenced

Links record direct influence as described in each paper and in the chapters; they are not a complete citation graph. Browse every paper with its summary in the paper library.