shreypatel

data engineer

I build data and ML systems, and then I keep them running.

Currently
Data engineer at Sepal AI, since December 2025. Observability and DevOps. Evaluation pipelines for language models, on ClickHouse, Grafana, and Prometheus.
Previously
EncryptMyWork (2025) · Kenexai (2021 to 2023) · ZF Group (2020, intern) Kafka, Flink, Spark, Iceberg, SageMaker. Streaming and ML platform work throughout.
Degree
MS, Computer Software Engineering Systems. Northeastern University, May 2025.
On the side
Coconut Labs, a two-person lab I run with a co-founder, Jay. Rented GPUs, published benchmarks, open-source releases. It is not my employer.
Open to
shreypatel@coconutlabs.org Full-time roles and contract work. Reply time is usually same day.
production data engineering

4 yearsacross three companies

Kafka, Flink, Spark, Iceberg. Billions of events a day at Kenexai, at 99.9% availability.

shared gpu, starved tenant

26xless waiting for a first token

1,585 ms under plain first-come-first-served, 61.5 ms with the scheduler I wrote. One A100, Llama-3.1-8B, vLLM 0.19.1, 311 requests. the bench →

mistakes in my own results

7found, numbered, published

A file of corrections against my own benchmarks, C1 to C7, in the public repo. read them →

running now · on a schedule · public record

Things I run

These are services, not simulations. They run on a cron when I am not looking at them, and each one publishes its own record, including the parts that failed. The counters live on the service. Nothing on this page can go stale because nothing on this page is a copy of them.

The Loop

nightly cron · github actions · public repo

A release pipeline for a model, run by a scheduled workflow with no human in it. Each night it registers a candidate version, scores it against a quality gate, sends a slice of traffic to it as a canary, then either promotes it or rolls back to the version already serving. Every night commits its own artifacts: the gate scores, the canary comparison, and a decision file naming the action and the reason for it.

So far it has rolled back every candidate it has been handed. That is the gate doing its job against a threshold set above what the serving model itself scores, not a regression caught in the wild. The decision files say so, which is the point of committing them.

the live view → · source · every run it has ever done

Pulse

continuous ingestion · public gap ledger

An ingestion pipeline that pulls live vehicle positions from the MBTA feed on a roughly 66 second cycle and lands them. When a poll is missed, it does not quietly heal. It writes the outage into a public ledger with a start time, a duration, the number of cycles lost, and a cause in plain words, such as upstream unreachable. Any pipeline can look healthy on a dashboard. This one has to name its holes.

It has had real outages, which is the whole reason the ledger exists. Freshness, volume, null-rate and disk gates run against it continuously. Their current state, the row count, the measured cadence, and every gap it has ever recorded live on the status page. None of it is copied onto this page, so none of it can be stale on this page.

the live view → · source

how i work

Four decisions, and what they cost

My rate limiter did nothing for three weeks

I had written a cap of 16 concurrent requests in front of the inference server. It was never holding anything back. The code released a request's slot the moment the response started streaming, not when it ended, so for streaming traffic the cap was effectively infinite. A poller sampled the in-flight gauge 149 times while 32 requests were running, and read zero every time. All of my benchmarks were streaming benchmarks.

I wrote it up, marked which published runs it invalidated, said which conclusions still stood and why, and fixed the release path. It is entry C5 in the corrections file, next to six others. I would rather someone find that file than find the bug.

C5 · 149 samples, cap 16, 0 in flight · results/CORRECTIONS.md

Then I ran the experiment that killed the idea anyway

Once that cap actually worked, I had to find out whether it was worth having. The bet was that holding traffic at the door would smooth the latency cliff under overload. It did not. The capped arm and the uncapped arm landed within 4 percent of each other on slowest-1-percent latency, against a threshold I had written down in advance saying the cap had to be four times better to be worth keeping. At one concurrency level the cap made things worse.

Then I checked my own bench with Little's Law and found the run was too short to have put the cap under real pressure. The result flattered my rigor and I had not earned it, so I downgraded my own verdict from FAIL to AMBIGUOUS, rented an H100, and ran it again at 16,000 requests per arm. Same answer, within 4 percent. The idea was wrong, and now it was wrong properly.

16,000 requests per arm · H100 SXM5 · about $1.30 of GPU time · PROGRESS.md · tuning_guide.md

A number does not travel without its regime

The 26x on this page is one specific thing: how much less a starved tenant waits for its first token when it shares a GPU with a noisy one. It holds on one A100 at saturation, running Llama-3.1-8B under vLLM 0.19.1, over 311 requests. On an H100 at the same offered load the starvation never happens, so the number does not transfer. An earlier run on vLLM 0.8.5 measured 523x, which looks twenty times better and was real output from a real machine. It was a weaker batcher handling cold-start backpressure badly, so 26x is what gets cited and the reason 523x does not is published next to it.

1,585 ms → 61.5 ms · solo baseline 53.9 ms · what this does not show

The name got ahead of the code

I called a project kvwarden, and then the code did not do what the name said. Version 0.1.x ships tenant-aware admission control. It does not manage the KV cache. Rather than let someone install it and find that out, I put the objection into the public FAQ as a heading, in the words a sceptical reader would use: the name says KV warden but you're not touching the KV cache, what gives. The answer underneath says exactly that, and the release that shipped it reframed the docs to match the code instead of the other way round.

v0.1.5, doc reframe to what the code does · the FAQ entry · pypi

shipped · 2026

In the order it happened

Older work sits on the GitHub profile: an RDMA NVMe offload stack, a C++20 latency workspace, a latent diffusion model trained from scratch on 10 H100 nodes, and an archived data platform kept public because the review that stopped it is worth more than pretending it still runs.

writing

Lab records, cleaned up

Tenant fairness on shared inference How one noisy tenant starves a quiet one on a shared GPU, and what fixed it.
A model in the room Three jobs a model can do inside a creative tool, and the fence that keeps it useful.
What mixing taught me about evals Studio monitors and evaluation suites fail in the same way.

Longer form: Systems That Don't Lie, five chapters on tenant fairness, single-machine inference, nanosecond signals, agents, and observability that cannot flatter itself.

contact

Full-time roles and contract work

Either address reaches me. Say what the system is and what is breaking, and I will tell you straight whether I am the right person for it.

shreypatel@coconutlabs.org · patelshrey77@gmail.com
working history · linkedin · github/ShreyPatel4 · github/coconut-labs