VLAs - Pixels to Tokens
VLAs - Pixels to Actions On the cover: perception, solved. Grasping, still two euros a go. Moravec’s paradox in an arcade, and the reason the honest-cons section ...
VLMs - Pixels to Tokens
On the cover: Pieter Bruegel the Elder, “The Tower of Babel” (1563). Everyone on the site is competent and none of them are mutually intelligible, which is precis...
ViT - Pixels to Tokens
On the cover: “Blind monks examining an elephant” (Hanabusa Itchō, 1888). Each monk has hold of exactly one patch, and the animal only shows up once you combine t...
Cross Entropy Loss
On the cover: A weather forecasting stone meme Imagine you live, work and vacation in three different cities. In City A (home), it is sunny every single day. No c...
MoE - Is Attention All You Really Need?
On the cover: A bunch of different robots who are expert at different tasks It’s been 8 years since the landmark paper “Attention is all you need” was published. ...
RoPE - Is Attention All You Really Need?
On the cover: A RoPE, essentially It’s been 8 years since the landmark paper “Attention is all you need” was published. The paper introduced the attention mechani...
KV Caching & MLA - Is Attention All You Really Need?
On the cover: MLA Architecture. Credits: Welch Labs It’s been 8 years since the landmark paper “Attention is all you need” was published. The paper introduced the...
Markov Chain - The connection between Nuclear Bombs, Google Search, and Perplexity
On the cover: A Nuclear Bomb Explosion Recently I watched this video by Veritasium talking about Markov Chains. I really liked the examples he used to explain the...
HNSW - Finding Needles in Vector Haystacks
On the cover: HNSW graph depiction. Credits: Marqo Imagine a city with 10 million coffee shops. You (the customer) have a very specific taste vector: medium roast...
Padlocks to Prime Numbers - RSA Encryption and SSH
On the cover: RSA Encryption If you’ve ever shopped online, sent a private message, or connected to a secure website, you’ve probably used RSA encryption without ...
How can you cook your pasta fasta? :p
On the cover: A cute robot chef cooking pasta Concurrency in Python is like running a restaurant kitchen during a dinner rush - and if you’re making pasta, you’re...
LoRA - The Diet Pill for Obese Language Models
On the cover: The concept of low-rank adaptation visualized as dimension reduction If you’ve been paying any attention to the AI world lately, you might have noti...