Bali · Asia/Makassar · Available for fractional engagements
const role = "Fractional Engineer — Data Platforms";
I’m Manoj Babu Katragadda. I design the infrastructure behind data, ML, and backend systems—turning distributed complexity into dependable paths for teams to ship.
Available: focused fractional engagements where platform complexity, delivery pressure, and the cost of a wrong decision are all high.
// operating principles
I build paved roads for engineers: declarative interfaces, safe defaults, observable execution, and explicit escape hatches. The goal is leverage without loss of control.
Over the last decade, I’ve moved from full-stack systems into experimentation, data platforms, and organization-wide infrastructure. Each layer taught me the same thing: abstractions earn trust only when they survive production.
// selected work
Products, primitives, and open-source tools built to make complex infrastructure easier to operate.
MLOps platform
A declarative ML lifecycle platform with 17 resource types, an AST-based compiler, dependency resolution, artifact versioning, and isolated workspaces across test, staging, and production.
resultOne Python SDK from data segmentation to model deployment.
Inside the Zest compilerData platform
A composable PySpark ETL framework with lifecycle hooks, plugin-based extractors and compactors, plus write-audit-publish data quality gates.
resultCut per-pipeline code by 75% and became the execution backbone for ML operations.
Read the framework designIceberg tooling
A single-binary Apache Iceberg table analyzer serving a REST API and embedded interface for file distribution, snapshot health, and orphan detection—without Spark or JVM overhead.
resultFast operational insight into lakehouse table health.
Read the lakehouse architectureOpen source
A Prefect integration for scheduling, orchestrating, and monitoring Apache Spark jobs through the Spark-on-Kubernetes operator.
resultPresented at PyCon Malaysia 2023 and released for the wider data community.
Read the Kubernetes playbook// fractional engineering
Fractional engineering works when delay is expensive. I narrow the scope, make the difficult architecture calls, ship production code, and leave the team with systems it can own.
bring_me_in_when =
A data platform is too slow, expensive, or unreliable.
A platform initiative is stuck between design and production.
The team needs hands-on senior execution, not another strategy deck.
A costly technical decision has to be made correctly—and quickly.
// career.log
From product APIs to the platforms beneath large-scale data and ML workloads.
Built the data and ML platform layer: cost-efficient Spark on Kubernetes, declarative MLOps, composable lakehouse execution, custom Airflow providers, and Rust-native Iceberg tooling.
Read the engineering notesPioneered data mesh adoption, designed event-driven backend foundations, built experimentation systems, and mentored engineers across the platform, software, and data organizations.
Designed ETL systems with Airflow, Spark, AWS Glue, and Redshift; partnered with data scientists to operationalize model workloads on Kubernetes.
Built recommendation and search systems, customer-facing product discovery, Java APIs, and the infrastructure behind caching and centralized service observability.
Developed backend APIs in Node.js at the start of a decade-long engineering career.
// education
Bachelor of Technology · Mining Engineering
2009 — 2014
// blog
Optimizing a daily Iceberg upsert from an hour to under 4 minutes with storage partition joins and shuffle hash hints.
Read article
Six ideas about feature stores, AST-generated ETLs, content-addressed artifacts, and control planes — distilled from building a semantic layer for ML.
Read article
Using S3 DAG bundles, a sidecar sync pattern, and a bundle watcher to onboard pipelines without Helm upgrades or downtime.
Read article// contact.sh
Send me the problem, the constraints, and what needs to move faster. I’ll tell you directly whether I can help and what a focused engagement should cover.
manojbabukatragadda@gmail.com