✏ Welcome to Yuanpang Blog

LLM Alignment - GRPO Implementation The blog transitions to a practical walkthrough illustrating policy gradient mechanics through the lens of GRPO (Group Relative Policy Optimization). GRPO simplifies PPO by removing the Value function (critic) and instead leveraging the group structure of LLM rollouts — i.e., multiple responses per prompt — t... Read More

LLM Alignment - Reinforcement Learning This post continues the exploration of Reinforcement Learning (RL) techniques for aligning Large Language Models (LLMs) — tracing the evolution from Direct Preference Optimization (DPO) to Proximal Policy Optimization (PPO) and Generalized Reinforcement Learning with Policy Optimization (GRPO), culminating... Read More

Reinforcement Learning (RL) — From Fundamentals to PPO & GRPO in LLMs (II) This blog continues to provide the advanced policy optimization techniques like PPO and TRPO, and concluding with the application of RL in Large Language Models (LLMs) via PPO and GRPO. 🚀 Section 2: Policy Gradient and Trust Region Optimization … In the last blog, ... Read More

Reinforcement Learning (RL) — From Fundamentals to PPO & GRPO in LLMs (I) This blog provides a detailed study of Reinforcement Learning (RL), starting from fundamental concepts and algorithms, moving through advanced policy optimization techniques like PPO and TRPO, and concluding with the application of RL in Large Language Models (LLMs) v... Read More

🎓 RLHF & Alignment: Making LLMs Useful and Safe This lecture, CS336 Lecture 15, dives into Reinforcement Learning from Human Feedback (RLHF) and alignment — the crucial post-training step that makes large pre-trained models like GPT-3 become helpful and safe assistants (like InstructGPT and ChatGPT). It follows the classic three-step proces... Read More

Filtering and Deduplication Algorithms for LLM Data Processing This lecture dives deeply into how raw web data is transformed into clean, usable training data for large language models (LLMs), focusing specifically on filtering and deduplication algorithms. Raw data moves from live services to dumps or crawls and must undergo HTML-to-text conve... Read More

The Crucial Role of Data in Training Language Models This lecture highlights the central role of data in the development of language models, following previous discussions about architectures and training strategies. It dissects the data pipeline, explores historical datasets, and addresses legal and ethical issues surrounding data use. Course... Read More

Evaluating Language Models — Beyond the Numbers 💻 This lecture provides a deep dive into the evaluation of language models, showing that while it seems simple, it’s actually a complex and profound discipline that shapes AI’s progress. It’s structured around key concepts and modern benchmark categories that define how we measure and compare inte... Read More

🚀 Modern LLM Inference — Workloads, Bottlenecks, and Optimization Techniques 🧠 From defining inference to lossy/lossless acceleration and dynamic serving — a full-stack view of how modern LLM inference actually works. Course link 1. 🏁 Introduction: What Is Inference and Why It’s Hard ✅ What Is Inference? Inference answers one fundamental que... Read More

⚖️ Scaling — Case Study and Details This lecture, “Scaling – Case Study and Details,” dives into best practices for scaling and hyperparameter tuning in large language models (LLMs). It revisits whether the Chinchilla-derived scaling methodologies still hold in modern model development and explores recent case studies (CerebrasGPT, MiniCPM, Dee... Read More

⚙️ The Predictable World of Scaling Laws in Language Models Scaling laws provide simple, predictive rules 📈 that govern the performance of Language Models (LMs), offering a pathway to optimize large-scale model design without relying on expensive, full-scale experimentation. They enable developers to tune hyperparameters on small models and con... Read More

Triton Introduction Here we use Triton to implement the weighted sum kernel (both forward and backward pass) as an example. The implementation is taking from the assignment 2 of the Stanford CS336 lecture. import triton import triton.language as tl import torch from einops import rearrange from triton import cdiv import time @triton.jit def ... Read More

LLM Training Parallelism Basics “Parallelism Basics” focuses on the system complexities behind training massive language models (LMs) that exceed a single GPU’s capacity. Goals: Understand different parallelization paradigms. Learn why multiple methods are combined. See how large-scale training is organized. Course link Code Link in th... Read More

GPU Kernels & Triton Programming 💻 This lecture dives into writing high-performance GPU code, which is essential for accelerating language models. The Challenge: Bridging the gap between high-level frameworks like PyTorch and the underlying GPU hardware, which often leads to “performance mysteries.” The Goal: To effectively optimize c... Read More

GPUs for Deep Learning 🚀 This lecture synthesizes key insights on GPUs, focusing on their architecture, performance bottlenecks, and advanced optimization techniques crucial for scaling large language models (LLMs). 🔥 Core Message: While GPU computational power (especially for matrix multiplications) has scaled exponentially, memory access ... Read More

Mixture of Experts 🤖 Mixture of Experts (MoE) architectures have rapidly become a cornerstone in developing high-performance, large-scale language models (LLMs). Once a “bonus lecture” topic, MoEs are now fundamental to state-of-the-art systems. 🚀 High Performance: MoEs offer significant advantages over traditional dense models in terms of ... Read More

LLM Architectures and Hyperparameters 🧠 This lecture summarizes key architectural trends, hyperparameter choices, and stability tricks observed in modern Large Language Models (LLMs). 📈 Architectural Trends: While the field is rapidly evolving, a “convergent evolution” towards “LLaMA-like” architectures is evident. 🔑 Key Consensus: Widesp... Read More

Language Model Training - PyTorch Primitives & Resource Accounting This blog summarizes key concepts from Stanford CS336 Lecture 2, focusing on PyTorch primitives, efficient resource accounting (memory and compute), and foundational elements of training deep learning models from scratch. Memory # Parameters num_parameters = (D * D *... Read More

Language Modeling from Scratch Overview and Tokenization Course link 1. Course Philosophy and Motivation: The “Build It from Scratch” Ethos Philosophy: “To understand it, you have to build it.” Problem: Increasing abstraction in AI research disconnects researchers from the underlying systems. Goal: Combat abstraction crisis by re-engag... Read More

🧠 Imitation Learning via Privileged Teachers and Generative Models like Diffusions This lecture builds upon previous discussions on Imitation Learning (IL) and delves into advanced techniques and current research areas. Course link 🔁 1. Recap of Previous Lecture (Imitation Learning Part 1) Lecture 6 begins with a brief review of key concepts... Read More

🧠 Markov Decision Processes (MDP) Basics and Imitation Learning This lecture provides a review of key themes and concepts related to Imitation Learning (IL) in the context of Robot Learning Course link 🔑 Key Themes and Concepts 1. Markov Decision Processes (MDPs) and Partially Observed MDPs (POMDPs) Sequential decision-making in robotics is... Read More

🧠 Robot Learning Overview This lecture provides a comprehensive introduction to Robot Learning, a field focused on achieving embodied intelligence in the physical world. The key challenge lies in the integration of algorithms, data, computation, and hardware to allow robots to perform tasks that are easy for humans but hard for machines — a phe... Read More

🤖 Introduction to Robot Learning (CMU 16-831) I. 📘 Course Overview and Core Concepts The “16-831: Introduction to Robot Learning” course, taught by Professor Guanya Shi at Carnegie Mellon University (CMU), focuses on the fundamental principles and applications of robot learning. Theme: “Learning to make sequential decisions in the physical... Read More

DeepSeek Reasoning Models Series 📌 In Part Two, we focus on DeepSeek’s Reasoning Models. DeepSeek-Coder: When the Large Language Model Meets Programming - The Rise of Code Intelligence DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence MATH-SHEPHERD: VERIFY AND REINFORCE LLMS STEP-BY-STEP WITHOUT HUMAN A... Read More

🧠 DeepSeek Base Models Series The blog is structured around two primary categories of DeepSeek’s work: Base Models and Reasoning Models. 📌 In Part One, we focus on DeepSeek’s Base Models. We’ll walk through the core ideas of four foundational DeepSeek papers, along with example PyTorch code to illustrate key components like Multi-head Latent A... Read More

Quantum Machine Learning Modern AI models are becoming increasingly large, demanding substantial computational resources and memory. This creates a gap between the computational demands of these models and the available hardware capabilities. Pruning addresses this gap by reducing model size, memory footprint, and ultimately, energy consumption... Read More

TinyML On-device Training Modern AI models are becoming increasingly large, demanding substantial computational resources and memory. This creates a gap between the computational demands of these models and the available hardware capabilities. Pruning addresses this gap by reducing model size, memory footprint, and ultimately, energy consumptio... Read More

TinyML Distributed Training Part 2 Modern AI models are becoming increasingly large, demanding substantial computational resources and memory. This creates a gap between the computational demands of these models and the available hardware capabilities. Pruning addresses this gap by reducing model size, memory footprint, and ultimately, energy c... Read More

TinyML Distributed Training Part 1 Modern AI models are becoming increasingly large, demanding substantial computational resources and memory. This creates a gap between the computational demands of these models and the available hardware capabilities. Pruning addresses this gap by reducing model size, memory footprint, and ultimately, energy c... Read More

TinyML Diffsion Models Modern AI models are becoming increasingly large, demanding substantial computational resources and memory. This creates a gap between the computational demands of these models and the available hardware capabilities. Pruning addresses this gap by reducing model size, memory footprint, and ultimately, energy consumption. ... Read More

TinyML GAN, Video, Point Cloud Modern AI models are becoming increasingly large, demanding substantial computational resources and memory. This creates a gap between the computational demands of these models and the available hardware capabilities. Pruning addresses this gap by reducing model size, memory footprint, and ultimately, energy consu... Read More

Vision Transformer Modern AI models are becoming increasingly large, demanding substantial computational resources and memory. This creates a gap between the computational demands of these models and the available hardware capabilities. Pruning addresses this gap by reducing model size, memory footprint, and ultimately, energy consumption. Cou... Read More

Long-Context LLM Modern AI models are becoming increasingly large, demanding substantial computational resources and memory. This creates a gap between the computational demands of these models and the available hardware capabilities. Pruning addresses this gap by reducing model size, memory footprint, and ultimately, energy consumption. Cours... Read More

LLM agents: brief history and overview Great talk from Shunyu Yao. Course Link What is an Agent? 🤖 In the realm of Artificial Intelligence, an “agent” is defined as an intelligent system capable of interacting with its environment. This interaction involves perceiving the environment through observations and acting upon it through actions. The... Read More

TinyML LLM Post-Training Modern AI models are becoming increasingly large, demanding substantial computational resources and memory. This creates a gap between the computational demands of these models and the available hardware capabilities. Pruning addresses this gap by reducing model size, memory footprint, and ultimately, energy consumption... Read More

TinyML LLM Deployment Techniques Modern AI models are becoming increasingly large, demanding substantial computational resources and memory. This creates a gap between the computational demands of these models and the available hardware capabilities. Pruning addresses this gap by reducing model size, memory footprint, and ultimately, energy con... Read More

Transformer and LLM Modern AI models are becoming increasingly large, demanding substantial computational resources and memory. This creates a gap between the computational demands of these models and the available hardware capabilities. Pruning addresses this gap by reducing model size, memory footprint, and ultimately, energy consumption. Co... Read More

TinyML TinyEngine Modern AI models are becoming increasingly large, demanding substantial computational resources and memory. This creates a gap between the computational demands of these models and the available hardware capabilities. Pruning addresses this gap by reducing model size, memory footprint, and ultimately, energy consumption. Cour... Read More

TinyML MCUNet Modern AI models are becoming increasingly large, demanding substantial computational resources and memory. This creates a gap between the computational demands of these models and the available hardware capabilities. Pruning addresses this gap by reducing model size, memory footprint, and ultimately, energy consumption. Course l... Read More

TinyML Distillation Modern AI models are becoming increasingly large, demanding substantial computational resources and memory. This creates a gap between the computational demands of these models and the available hardware capabilities. Pruning addresses this gap by reducing model size, memory footprint, and ultimately, energy consumption. Co... Read More

Neural Architecture Search Modern AI models are becoming increasingly large, demanding substantial computational resources and memory. This creates a gap between the computational demands of these models and the available hardware capabilities. Pruning addresses this gap by reducing model size, memory footprint, and ultimately, energy consumpti... Read More

TinyML Quantization II Modern AI models are becoming increasingly large, demanding substantial computational resources and memory. This creates a gap between the computational demands of these models and the available hardware capabilities. Pruning addresses this gap by reducing model size, memory footprint, and ultimately, energy consumption. ... Read More

TinyML Quantization I Modern AI models are becoming increasingly large, demanding substantial computational resources and memory. This creates a gap between the computational demands of these models and the available hardware capabilities. Pruning addresses this gap by reducing model size, memory footprint, and ultimately, energy consumption. ... Read More

Pruning and Sparsity Modern AI models are becoming increasingly large, demanding substantial computational resources and memory. This creates a gap between the computational demands of these models and the available hardware capabilities. Pruning addresses this gap by reducing model size, memory footprint, and ultimately, energy consumption. C... Read More

Basics of Neural Networks Course link 🌟 Main Themes The growing computational demand of deep learning models is outpacing hardware advancements, creating a strong need for efficient deep learning techniques. Understanding the basic building blocks of neural networks and their associated efficiency metrics is essential for designing optim... Read More

TinyML 📚 Introduction MIT’s TinyML and Efficient Deep Learning Computing course, taught by Professor Song Han, kicks off with an introduction to optimizing and speeding up deep learning models. As models grow in complexity, hardware constraints create a gap between model needs and deployment capabilities, driving up costs and emphasizing the ne... Read More

Predicting Customer Churn: Regression vs. Survival Analysis 🚀 When it comes to predicting customer churn, the choice between regression and survival analysis depends on your data and objectives. While regression models may seem simpler, survival analysis is often better suited for time-to-event problems, especially when dealing with censored da... Read More

Overview of Quantization Motivation Data Types Quantization Basics Quantization Target Post-Training Quantization (PTQ) Quantization-Aware Training (QAT) Quantization Introduction K-means-based Weight Quantization Linear Quantization Binary and Ternary Quantization Automatic... Read More

Traditional NLP Metrics The direct use of metrics such as perplexity and BLEU score has declined in popularity, largely due to their inherent flaws in many contexts. However, it remains crucial to comprehend these metrics and discern their appropriate applications. BLEU Paper: BLEU: a Method for Automatic Evaluation of Machine Translation Ori... Read More

This guide dives into the essentials of recommendation systems, with a focus on key metrics, experimentation methods, and the underlying architecture. Whether you’re a beginner or looking to deepen your understanding of recommendation strategies, this will give you a structured breakdown of critical components. Follow this awesome tutorial by ... Read More

This guide dives into the essentials of recommendation systems, with a focus on key metrics, experimentation methods, and the underlying architecture. Whether you’re a beginner or looking to deepen your understanding of recommendation strategies, this will give you a structured breakdown of critical components. Follow this awesome tutorial by ... Read More

📊 Recommendation System Overview This guide dives into the essentials of recommendation systems, with a focus on key metrics, experimentation methods, and the underlying architecture. Whether you’re a beginner or looking to deepen your understanding of recommendation strategies, this will give you a structured breakdown of critical components. ... Read More

Q-Functions in Reinforcement Learning Lecture 8 is about making Q-learning work in practice. Lecture 7 gave the value-function theory. Lecture 8 focuses on practical stability tricks used in deep RL systems: replay buffers target networks double Q-learning multi-step targets handling continuous actions Course Link 1. Recap: Fitt... Read More

Value Function Methods in Reinforcement Learning This lecture is about one core idea: Learn a value function that scores actions. Act by choosing the action with the highest score. If the lecture feels heavy, use this reading order: Section 2 (tiny tabular example) Sections 6 and 7 (FQI and online Q-learning) Section 8 (why conv... Read More

Actor-Critic Algorithms in Reinforcement Learning This lecture focuses on Actor-Critic algorithms in Deep Reinforcement Learning. It covers the evolution from basic policy gradients, the role of value functions, various policy evaluation techniques, practical implementation considerations, and advanced variance reduction methods. Course Link ... Read More

Policy Gradients in Reinforcement Learning This lecture summarizes the core concepts, derivations, practical considerations, and advanced topics related to Policy Gradients, a fundamental algorithm in reinforcement learning (RL). Course Link 1. Introduction to Policy Gradients Policy gradients are a foundational RL algorithm that directly opt... Read More

Introduction to Reinforcement Learning This lecture covers fundamental definitions, the objective of RL, the anatomy of RL algorithms, and a categorization of various algorithm types along with their trade-offs and assumptions. Course Link 1. Core Terminology and Concepts The fundamental difference between reinforcement learning (RL) and imit... Read More

🤖 Basic Pick and Place - Differential kinematics via optimization Course Link Optimization-Based Differential Inverse Kinematics (DIK-QP) Problem with Pseudo-Inverse Limitation: Does not handle joint limits or other real-world constraints. Consequence: Can lead to clipped velocities and off-course end-effector trajectories. Pseudo-Inve... Read More

🤖 Basic Pick and Place - Differential kinematics Course Link The fundamental goal is to move a “red brick from one bin to the second bin” using a robot. This requires: Defining target poses: Ideal gripper positions/orientations for picking and placing Generating trajectories: Compose keyframes into smooth paths Robot control: Convert d... Read More

🤖 Basic Pick and Place - Kinematics and Spatial Algebra The lecture covers foundational techniques for robot manipulation, especially pick-and-place tasks, with a focus on kinematics and spatial algebra. Course Link I. Introduction to Robotic Manipulation and the Pick and Place Problem A core challenge in robotic manipulation is commanding a... Read More

🤖 Robot Hardware This blog summarizes key concepts from 6.4210 Fall 2023 Lecture 2: Let’s get you a robot! and supporting materials, focusing on robot arm hardware, the intricacies of simulation, and the evolving landscape of robot hands. Course Link 1. Robot Arm Hardware: Evolution and Key Characteristics 🏭 Industrial Robots (Traditional) ... Read More

1. Defining Robotic Manipulation The lecture delves into a nuanced definition of robotic manipulation, moving beyond simple object handling to encompass complex interactions within dynamic and unstructured environments. Course Link A. Core Definition Matt Mason’s Definition: “Manipulation refers to the agent’s control of the en... Read More

Imitation Learning: Challenges and Solutions This lecture reviews supervised Learning of Behaviors / Imitation Learning Course Link 1. Introduction to Imitation Learning and Behavioral Cloning Imitation Learning (IL), specifically Behavioral Cloning (BC), involves training a policy (a model that maps observations to actions) using a dataset o... Read More

🤖 Deep Reinforcement Learning This briefing document reviews the main themes and key takeaways from a collection of sources focused on deep reinforcement learning (Deep RL), including insights from CS 285 lectures and supplementary materials. Course Link 📉 The Limitations of Data-Driven AI Data-driven AI has achieved impressive results, p... Read More

♟️ AlphaGo, AlphaGo Zero, and AlphaZero - Deep Reinforcement Learning Meets Search This blog explores the structure and training process of AlphaGo and its successors, AlphaGo Zero and AlphaZero, illustrating how deep reinforcement learning and search are combined to achieve superhuman performance in complex board games. 🧩 1. Introduction and ... Read More

🎭 The Actor–Critic Method: Bridging Policy-Based and Value-Based Reinforcement Learning The Actor–Critic (AC) method is a foundational algorithm in Reinforcement Learning that elegantly combines the strengths of Policy-Based and Value-Based methods. It uses two neural networks — the Policy Network (Actor) and the Value Network (Critic) — traine... Read More

🧭 Policy-Based Reinforcement Learning — Directly Learning to Act Policy-Based Reinforcement Learning (RL), also known as Policy Learning, focuses on directly modeling and optimizing the agent’s policy $( \pi )$, i.e., the agent’s behavior function. This contrasts with Value-Based RL (like DQN), which indirectly learns the policy by estimating t... Read More

Value-Based Reinforcement Learning Foundations Value-based reinforcement learning (RL) focuses on estimating how valuable it is to take a particular action in a given state — quantified as the expected discounted future reward. This approach underpins algorithms like Q-learning and Deep Q-Networks (DQN). 1️. Value-Based Reinforcement Learning ... Read More

Reinforcement Learning Basics Follow this awesome tutorial by Shusen Wang, which provides a foundational overview of reinforcement learning (RL) — starting from probability theory and building up to key RL concepts such as states, actions, policies, and value functions. 1. 🎲 A Little Bit of Probability Theory The lecture begins with essentia... Read More

Course Link 🌐 What is a Distributed System? A distributed system is a collection of computers that communicate over a network to perform a task together. Examples 📱 Popular app backends (e.g., for messaging) 🌐 Large websites 🖧 Domain Name System (DNS) 📞 Phone systems These systems often use services that are themselves ... Read More

📚 Generative Adversarial Networks Course Link This document reviews the main themes and key takeaways from Deep Learning Systems: Algorithms and Implementation** at Carnegie Mellon University, taught by J. Zico Kolter and Tianqi Chen. 🧾 Briefing Document: Generative Adversarial Networks (GANs) 🚀 Introduction: This document provides a detai... Read More

📚 Transformer Implementation Course Link This document reviews the main themes and key takeaways from Deep Learning Systems: Algorithms and Implementation** at Carnegie Mellon University, taught by J. Zico Kolter and Tianqi Chen. This document details the implementation of a Transformer model using NumPy, comparing it to PyTorch’s implement... Read More

📚 Transformers and Autoregressive Models Course Link This document reviews the main themes and key takeaways from Deep Learning Systems: Algorithms and Implementation** at Carnegie Mellon University, taught by J. Zico Kolter and Tianqi Chen. This document summarizes key concepts from the lecture on Transformers and attention mechanisms. The... Read More

📚 LSTM Implementation Course Link This document reviews the main themes and key takeaways from Deep Learning Systems: Algorithms and Implementation** at Carnegie Mellon University, taught by J. Zico Kolter and Tianqi Chen. Lecture Overview The lecture focuses on implementing a Long Short-Term Memory (LSTM) network, starting with a single c... Read More

📚 Sequence Modeling and Recurrent Networks Course Link This document reviews the main themes and key takeaways from Deep Learning Systems: Algorithms and Implementation** at Carnegie Mellon University, taught by J. Zico Kolter and Tianqi Chen. This lecture covers sequence modeling, RNNs, LSTMs, and their applications in complex prediction t... Read More

📚 Convolutional Networks Implementation Course Link This document reviews the main themes and key takeaways from Deep Learning Systems: Algorithms and Implementation** at Carnegie Mellon University, taught by J. Zico Kolter and Tianqi Chen. 🖥️ Implementing Convolutions in Code The lecture focuses on how to implement convolutions in code, m... Read More

📚 GPU Acceleration Course Link This document reviews the main themes and key takeaways from Deep Learning Systems: Algorithms and Implementation** at Carnegie Mellon University, taught by J. Zico Kolter and Tianqi Chen. This lecture summarizes key concepts and techniques related to GPU (Graphics Processing Unit) acceleration, particularly w... Read More

📚 Hardware Acceleration Course Link This document reviews the main themes and key takeaways from Deep Learning Systems: Algorithms and Implementation** at Carnegie Mellon University, taught by J. Zico Kolter and Tianqi Chen. I. Introduction and Motivation ⚡ Necessity of Acceleration The increasing computational demands of large models and... Read More

📚 Convolutional Networks Course Link This document reviews the main themes and key takeaways from Deep Learning Systems: Algorithms and Implementation** at Carnegie Mellon University, taught by J. Zico Kolter and Tianqi Chen. This document summarizes key concepts and practical considerations related to Convolutional Networks (CNNs) based on... Read More

📚 Neural Network Library Implementation Course Link This document reviews the main themes and key takeaways from Deep Learning Systems: Algorithms and Implementation** at Carnegie Mellon University, taught by J. Zico Kolter and Tianqi Chen. 🛠️ I. Introduction & Setup 🎉 Welcoming participants back to the Deep Learning Systems: Algorit... Read More

📚 Normalization and Regularization Course Link This document reviews the main themes and key takeaways from Deep Learning Systems: Algorithms and Implementation** at Carnegie Mellon University, taught by J. Zico Kolter and Tianqi Chen. 🏁 Initialization and Optimization Weight Initialization Initializing weights is critical for... Read More

📚 Common Abstractions for Neural Network Computations Course Link This document reviews the main themes and key takeaways from Deep Learning Systems: Algorithms and Implementation** at Carnegie Mellon University, taught by J. Zico Kolter and Tianqi Chen. 📚 Introduction to Neural Network Library Abstractions I. Introduction to Neural Networ... Read More

📚 Fully Connected Networks, Optimization, Initialization and Activations Course Link This document reviews the main themes and key takeaways from Deep Learning Systems: Algorithms and Implementation** at Carnegie Mellon University, taught by J. Zico Kolter and Tianqi Chen. 🧠 Fully Connected Networks In a fully connected network, each neuro... Read More

📚 Automatic Differentiation Lab Course Link This document reviews the main themes and key takeaways from Deep Learning Systems: Algorithms and Implementation** at Carnegie Mellon University, taught by J. Zico Kolter and Tianqi Chen. 🔍 The “Needle” Package: A Deep Dive into Automatic Differentiation Implementation The “Needle” package is a co... Read More

📚 Introduction of Automatic Differentiation Course Link This document reviews the main themes and key takeaways from Deep Learning Systems: Algorithms and Implementation** at Carnegie Mellon University, taught by J. Zico Kolter and Tianqi Chen. 1. Machine Learning Components Every machine learning algorithm has three fundamental elements: ... Read More

📚 Manual Neural Networks Course Link This document reviews the main themes and key takeaways from Deep Learning Systems: Algorithms and Implementation** at Carnegie Mellon University, taught by J. Zico Kolter and Tianqi Chen. 1. Limitations of Linear Classifiers 🧮 Linear classifiers divide the input space into linear regions, limiting thei... Read More

📚 Softmax Regression Course Link This document reviews the main themes and key takeaways from Deep Learning Systems: Algorithms and Implementation** at Carnegie Mellon University, taught by J. Zico Kolter and Tianqi Chen. 1. 🛠️ Ingredients of a Machine Learning Algorithm: Hypothesis Function: Maps input features to output predictions. Lo... Read More

📚 Deep Learning Systems Introduction Course Link This document reviews the main themes and key takeaways from Deep Learning Systems: Algorithms and Implementation** at Carnegie Mellon University, taught by J. Zico Kolter and Tianqi Chen. 1. 🚀 Why Study Deep Learning (DL) and DL Systems? The lecture emphasizes the powerful capabilities of mod... Read More