# Karthik Balasubramanian > ML Inference & GPU Systems Engineer — Karthik Balasubramanian writes about LLM inference engineering and GPU kernels: Triton and CUDA, attention, KV cache, vLLM and SGLang serving, benchmarking and observability. Software Engineer at Adobe (since 2017, starting at Marketo) transitioning from site reliability, Kubernetes infrastructure, observability, and agentic systems into **ML Inference Engineering** and **GPU Kernel Engineering**. ## Focus & Target Roles - **Target roles:** ML Inference Engineer, GPU Systems Engineer, ML Infrastructure Engineer (LLM serving, performance benchmarking, reliability). - **Current focus:** Triton & CUDA GPU kernels (Vizuara AI Kernel Engineering), building LLMs from scratch in PyTorch (Sebastian Raschka), and LLM serving/benchmarking with vLLM and SGLang (p50/p99 latency, TTFT, tokens/s, MFU, KV-cache management). ## Background & Skills - **Languages:** Python, Java - **ML & LLMs:** PyTorch, Transformers, LLM evaluation (*analyze → measure → improve*, Hamel Husain & Shreya Shankar), agentic frameworks (LangGraph), agent harnesses & isolated sandboxes (Agent Sandbox), A2A protocol - **Inference & Kernels (in progress):** Triton, CUDA, vLLM, SGLang, roofline/MFU analysis - **Infrastructure:** Kubernetes (CKA), Docker, Helm, Terraform, Airflow, FastAPI, microservices, GitOps CI/CD - **Observability:** Prometheus, Grafana, OpenTelemetry, Splunk - **Data:** PostgreSQL, MySQL, MS SQL Server, Azure Data Explorer, Redis, Databricks - **Education:** Master of Information Systems Management, Carnegie Mellon University (Heinz College, 2017); B.Tech in Information Technology, SRM University (2013) - **Certifications:** Certified Kubernetes Administrator (CKA, Linux Foundation, 2024); AI Evals for Engineers & PMs (Maven, 2026); Udacity Nanodegrees in Cloud DevOps (2020), Data Engineering (2020), and Machine Learning (2019) ## Core Pages - [Home](https://karthikbalasubramanian.github.io/): Positioning, featured inference/LLM work, and latest posts - [Blog](https://karthikbalasubramanian.github.io/blog/): Full archive of technical writing - [Projects](https://karthikbalasubramanian.github.io/projects/): Public GitHub repositories and ML/infrastructure projects - [About](https://karthikbalasubramanian.github.io/about/): Detailed background, skills, education, and certifications - [Tools](https://karthikbalasubramanian.github.io/tools/): Client-side personal finance calculators built in React and TypeScript - [RSS Feed](https://karthikbalasubramanian.github.io/feed.xml): Atom/RSS feed of all blog posts ## Key Repositories - [pytorch_tutorials](https://github.com/karthikBalasubramanian/pytorch_tutorials): Building an LLM from scratch in PyTorch (BPE tokenization, embeddings, causal/multi-head attention, GPTModel, MPS vs CPU benchmarks, DDP) - [jev-tutorials](https://github.com/karthikBalasubramanian/jev-tutorials): Calibrated non-generative decision models for support triage in TypeScript/Node.js (Vercel AI SDK) - [project-ml-microservice-kubernetes](https://github.com/karthikBalasubramanian/project-ml-microservice-kubernetes): Containerized ML inference API deployed on Kubernetes with CI/CD and structured logging - [FER](https://github.com/karthikBalasubramanian/FER): Facial emotion recognition using heuristic and transfer learning ## Writing - [The Flashlight and the Shadow: How Dot Products and Normalization Power AI Attention](https://karthikbalasubramanian.github.io/ml/2026/09/02/utility_of_dot_products_in_attention.html) (2026-09-02): From flashlight shadows to multi-dimensional Transformer attention: an intuitive, first-principles deep dive into dot products, matrix operations, and why self-attention works the way it does. - [The Great AI Compromise: How Autonomous Agents Will Reclaim Our Idling GPUs](https://karthikbalasubramanian.github.io/ml/2026/08/26/the_great_ai_compromise.html) (2026-08-26): An exploration of FLOPs, memory bandwidth, MFU across GPT-2 to Claude 3.5 Sonnet, and how long-horizon autonomous agents will unlock maximum GPU throughput. - [Facial Emotions Recognition using Heuristic Learning and Transfer Learning](https://karthikbalasubramanian.github.io/ml/2019/05/17/Karthik-_FER_Capstone_Report.html) (2019-05-17): Emotion Recognition and Pattern Recognition - [How to Train an agent to play the game of Black Jack?](https://karthikbalasubramanian.github.io/ml/2019/03/29/Monte_Carlo.html) (2019-03-29): Reinforcement learning using Monte Carlo Methods - [Dynamic Programming - An easier Reinforcement learning setup](https://karthikbalasubramanian.github.io/ml/2019/03/15/dp.html) (2019-03-15): Solve a classic MDP ( Exploring Frozen Lake) using Dynamic Programming - [Dog Breed classification using transfer learning](https://karthikbalasubramanian.github.io/ml/2019/02/14/dog_app.html) (2019-02-14): Real world deep learning application to identify a dog breed from 133 breeds in total - [Convolutional Neural Nets Introduction](https://karthikbalasubramanian.github.io/ml/2019/01/30/cifar10_cnn_adam.html) (2019-01-30): A tutorial to build a CNN to learn CIFAR 10 Images - [A Visualization tutorial](https://karthikbalasubramanian.github.io/ml/2017/01/20/AltairViz.html) (2017-01-20): A declarative statistics visualization library - [Image Classification](https://karthikbalasubramanian.github.io/ml/2017/01/15/knn.html) (2017-01-15): Road to Convolutional Neural Networks - [First Kaggle Submission - Deep learning](https://karthikbalasubramanian.github.io/ml/2017/01/07/karthik_b_redux.html) (2017-01-07): Introduction to Convolutional Neural Networks - [Challenges faced by CIOs in Big Data projects](https://karthikbalasubramanian.github.io/writing/2016/12/26/CIO.html) (2016-12-26): How tough is it for a CIO to handle multiple Big Data projects at a time in an enterprise. ## Optional: Interactive Tools - [Net Take-Home & Wealth Allocator](https://karthikbalasubramanian.github.io/paycheck-tax-investment-allocator/): Work out your real take-home pay after maxing out 401(k), HSA and ESPP. - [Rent vs. Buy & Major Purchase Planner](https://karthikbalasubramanian.github.io/rent-vs-buy-house-poor-calculator/): Test monthly rent, total homeownership costs (PITI) and other big commitments against your take-home pay. - [Child 529 & Family Wealth Growth](https://karthikbalasubramanian.github.io/child-financial-investment-planner/): Project 18 years of compounding growth in 529, UTMA and Roth rollover accounts. - [RSU Hold vs. Sell Simulator & Game](https://karthikbalasubramanian.github.io/rsu-hold-vs-sell-simulator/): Simulate vesting schedules, compare holding your RSUs with the S&P 500, and play the regret game. ## Contact - GitHub: https://github.com/karthikBalasubramanian - LinkedIn: https://www.linkedin.com/in/bks4line