In progress
Three things that moved this month: a training runtime under audit, a kernel library running a real model, and one you can install today.
Composed-MoE-Engine: Correctness audit. An October audit found gradients were wrong whenever any parallelism dimension was above one. Expert, tensor and sequence parallel backward are now fixed and checked against a dense reference. Pipeline parallelism is next.
simd-ml-kernels: A real model on my own kernels. Llama-3.2-1B now decodes end to end on these kernels, token-identical to Hugging Face in FP32, at or above llama.cpp's speed in all nine measured configurations on the same cores.
KANX: Shipping. Production-oriented Kolmogorov–Arnold Networks with PyTorch, TensorFlow and ONNX backends. pip install kanx.
The day-to-day is production infrastructure. What you can inspect on this page is the public work.
The systems I would defend.
An instrument is an argument about reality, and it can be wrong.
Built from scratch. Each one says what is measured and what is not.
FlashSpec
Speculative decoding · alpha
1.26–1.67× faster than compiled Hugging Face generate, across chat and code prompts, greedy and sampled.
Lossless speculative decoding for Hugging Face models: prompt lookup when the context already holds the answer, a small drafter model otherwise, on a CUDA-graph fast path.
One T4, Qwen2.5-1.5B checked by Qwen2.5-0.5B, one request at a time, ten prompts per set. Not yet faster on CPU or at larger batches.
Greedy output matched compiled Hugging Face token for token on 17 of 20 prompts; the rest differ at exact fp16 ties.
Code edits copy from the prompt, so they run faster still: 3.88× on the edit prompt replayed here.
Includes FlashSpec-ASM, a hand-written AVX2 kernel. It does not change end-to-end speed.
My bandit lost to an if-statement. The if-statement is now the default; the bandit ships labelled experimental.
GitHub
simd-ml-kernels
x86 SIMD kernels · C++17 · preprint
1.02–1.69× llama.cpp's decode speed on Llama-3.2-1B, in all nine measured configurations, on the same cores and weight files.
Hand-written AVX2, AVX-512 and VNNI kernels for the arithmetic under inference, each tested against a float64 reference and benchmarked against OpenBLAS or PyTorch in the same run. FP32 output is token-identical to Hugging Face.
It still loses to OpenBLAS and PyTorch in several measured cases, and the README lists every one. If you just want speed, use those.
GitHub
KANX
Kolmogorov–Arnold networks · PyPI
5× fewer parameters than the MLP it beats, on a smooth 2-D target, across three seeds.
Kolmogorov–Arnold Networks built for deployment: PyTorch and TensorFlow backends, real ONNX export, a FastAPI service, Docker and Kubernetes configs. pip install kanx.
It is about 8× slower at inference than that MLP, and on a real table (UCI diabetes) the spline adds nothing over the SiLU fallback beside it.
GitHub · Article: The Activation Moved to the Edge
guardrail-rs
LLM security proxy · Rust · crates.io
47 µs to screen an 8 KB request with the default regex and PII pipeline.
A zero-Python LLM security reverse proxy in Rust. Fail-open by design. Five crates on crates.io: cargo install guardrail-cli.
No formal adversarial-robustness eval yet; that is the next piece of work. The optional ONNX classifiers also miss their 5 ms target by a wide margin.
GitHub · Article: Building guardrail-rs
filing-qa
Question answering over financial filings
0 fabricated numbers across 119 graded answers.
Question answering over financial filings: chart extraction, hybrid dense + BM25 retrieval, and a guardrail that checks every generated number against the source. Multi-tenant, observable, Helm-deployable.
About 3 in 10 questions still miss at retrieval. An earlier README claimed 89% recall; I could not reproduce it, so I withdrew it and published the number I could.
GitHub
Composed-MoE-Engine
Fault-tolerant MoE runtime · preprint · under audit
10 / 10 storage-stall chaos runs pass. Node-kill recovery passes about 85% of the time.
The systems layer under a Mixture-of-Experts model: fused Triton router, 4D parallelism, elastic recovery with expert resharding, two-tier async checkpointing.
An October audit against a dense single-device reference found the backward pass wrong whenever any parallelism dimension was above one, while 409 tests passed. Expert, tensor and sequence parallel are now fixed and checked against that reference; pipeline parallelism is next. Multi-node is still open.
GitHub · The audit record
Supporting work
mcp-reliable: MCP observability · npm. Watches MCP servers while they run: health, per-tool latency and output-shape drift, with alerts. Published on npm with six @mattral packages.
PromptCanary: Silent LLM drift. Detecting providers changing behaviour under a model string that never changed.
Worldgap: Early research · pre-registered. Scoring the sim-to-real gap with a world model before touching hardware. Two pre-registered runs so far: one null, one that tracked degradation but added nothing over a simpler baseline.
Earlier work, kept on purpose: Gesture_Arm, a gesture- and voice-controlled arm measured on real hardware, and an Induction Circuit study on whether fine-tuning quietly breaks a small transformer's copy mechanism.
The work you can inspect.
Merged correctness fixes in projects other people depend on. Each excerpt is from the merged diff.
triton-lang/triton #10699 (merged)
Fixed NaN handling in argmin and argmax under tie_break_left=False in the Triton interpreter.
Merged by the maintainer, cherry-picked into Meta's internal fork, and later referenced by follow-up test-coverage PRs.
- if self.combine_fn == tl.standard._argmin_combine_tie_break_left:
+ if (self.combine_fn is tl.standard._argmin_combine_tie_break_left
+ or self.combine_fn is tl.standard._argmin_combine_tie_break_fast):
return self.min_max(input[0], val_reduce_op=np.nanmin, idx_reduce_op=np.nanargmin)
- elif self.combine_fn == tl.standard._argmax_combine_tie_break_left:
+ elif (self.combine_fn is tl.standard._argmax_combine_tie_break_left
+ or self.combine_fn is tl.standard._argmax_combine_tie_break_fast):
return self.min_max(input[0], val_reduce_op=np.nanmax, idx_reduce_op=np.nanargmax)
The pull request · Inside PR #10699: how a missing elif let NaN win in Triton's interpreter
NVIDIA/Megatron-LM #5530 (merged)
Fixed a crash in get_grad_norm_fp32 when the grads-for-norm list came back empty, the case you hit with frozen layers and tensor parallelism.
max() over an empty sequence raises. The fix gives the inf-norm path a default and takes the same care on the p-norm path.
if norm_type == inf:
- total_norm = max(grad.abs().max() for grad in grads_for_norm)
+ total_norm = max((grad.abs().max() for grad in grads_for_norm), default=torch.tensor(0.0))
The pull request · A silent crash in Megatron-LM's gradient clipping
tensorflow/tensorflow #122544 (merged)
tf.experimental.numpy.swapaxes now raises on an out-of-bounds axis instead of silently normalising it.
Silent correctness bugs are worse than loud ones. Most of my work is some version of this sentence.
- if isinstance(x, int):
+ if isinstance(x, int) and isinstance(rank, int):
+ if not (-rank <= x < rank):
+ raise ValueError(
+ f'axis {x} is out of bounds for array of dimension {rank}'
+ )
if x < 0:
x = x + rank
The pull request · Same input, two outcomes: debugging a silent eager/XLA divergence in TensorFlow
Writing
An apprenticeship in scale.
The part of the path that shaped this work began offshore. I joined PTTEP to work on instrumentation and operational data, and was promoted on what I delivered. Real industrial systems set the constraints long before I had opinions about them.
At TotalEnergies the work became production ML for offshore energy operations. A model that fails silently there has physical safety consequences, so the job was pipeline reliability and monitoring against live operational streams.
Then the scale changed. I moved to a hyperscaler, first in cloud operations, watching the infrastructure that ML workloads run on, and now as an ML systems engineer working on reliability and fault tolerance for distributed training.
I read these years as an apprenticeship in scale: what it actually costs for an intelligent system to survive messy reality — fault tolerance, observability, drift detection, behaving sensibly under uncertainty. I learned it first somewhere the stakes were physical and immediate.
AWS AI & ML Mentor Scholarship, earned along the way.
An MSc in machine learning and data science, studied online after work.
Machines first. Then the idea that intelligence had structure.
The machines came first: self-balancing robots, quadcopters, machines that moved because of what a camera saw. Each one taught the same lesson from a different angle.
Computer vision found me in the AlexNet years. I worked through MIT OpenCourseWare and Andrew Ng's course back when the assignments were still written in Matlab. What caught me was that intelligence appeared to have structure you could write down, learn, and examine.
Self-balancing robot: Falling is the default state. Staying upright is active work, every millisecond.
Quadcopter: Four motors, one intention. Distributed control that still has to agree.
Vision: The moment a machine acted on what it saw, robotics became perception.
There is a difference between implementing something and understanding it. Implementing is a weekend. Understanding is the part that takes fifteen years.
The person.
I am an introvert who makes friends easily and still often feels slightly apart from the room. I have stopped treating that as a problem to solve. It is simply how I am configured.
Apart, and fine with it
Watching the room from a slight distance turns out to be useful for engineering.
Questions I can test
The philosophy I chase is the kind that can be checked — like a study I ran on whether fine-tuning quietly breaks a transformer's copy mechanism.
The cost
Years of books and late screens. It cost something, and I do not dramatise it.
Some questions I am still following, and some decisions I have not made yet.