Latest writing
All posts →Mamba - Is Attention All You Really Need?
Attention keeps every word it has ever heard. Mamba keeps one small pad and decides, word by word, what is worth writing on it.
Knowledge Distillation - The Perfumer's Apprentice
The label says rose. The teacher's nose says rose, some jasmine, and definitely not vanilla, and that last part is what the small model actually learns.
Quantization - Four Bits and a Winter Coat
Cutting every weight from sixteen bits to four is nearly free. The handful of channels that shout a hundred times louder than the rest is what costs you.
GRPO and RLVR - The Answer Key and the Rank List
RLHF trained a model to mark the homework. RLVR checks the answer instead, and GRPO sacks the critic by grading every batch on a curve.
Selected projects
All projects →Legal Research Assistant
On the cover: Legal Research Assistant Platform This project was delivered for MiAI.law, an Australian legal technology firm, to build a modern AI-driven system f...
AI-Driven Quantitative Risk Analysis
On the cover: Digitized Process Flow Diagram (PFD) with highlighted Elementary Process Sections (EPS) and graph This work was conducted at Saudi Aramco. The proje...