Latest writing
All posts →Mamba - Is Attention All You Really Need? Part 1
State space models come from control theory, run like an RNN and train like a CNN. Mamba's one change lets each word decide what the model remembers.
Mamba - Is Attention All You Really Need? Part 2
Selection broke the convolution that made state space models fast to train. Here is how Mamba got the speed back, and why attention stayed in the room.
Knowledge Distillation - The Perfumer's Apprentice
The label says rose. The teacher's nose says rose, some jasmine, and definitely not vanilla, and that last part is what the small model actually learns.
Quantization - Four Bits and a Winter Coat
Cutting every weight from sixteen bits to four is nearly free. The handful of channels that shout a hundred times louder than the rest is what costs you.
Selected projects
All projects →Legal Research Assistant
On the cover: Legal Research Assistant Platform This project was delivered for MiAI.law, an Australian legal technology firm, to build a modern AI-driven system f...
AI-Driven Quantitative Risk Analysis
On the cover: Digitized Process Flow Diagram (PFD) with highlighted Elementary Process Sections (EPS) and graph This work was conducted at Saudi Aramco. The proje...