Manoj Babu Katragadda — BlogProduction notes on data platforms, Spark, Iceberg, MLOps, and Kubernetes.https://tardunge.com/en-usWhat We Learned Building Our ML Platformhttps://tardunge.com/blog/what-we-learned-building-our-ml-platform/https://tardunge.com/blog/what-we-learned-building-our-ml-platform/Six ideas about feature stores, AST-generated ETLs, content-addressed artifacts, and control planes — distilled from building a semantic layer for ML.Thu, 23 Apr 2026 00:00:00 GMTmlopssemantic-layerdata-engineeringplatform-engineeringapache-icebergManoj Babu Katragadda, Meghanath MachaAirflow DAG Bundles: Managing DAGs Across Teams Without Helm Upgradeshttps://tardunge.com/blog/airflow-dag-bundles-without-helm-upgrades/https://tardunge.com/blog/airflow-dag-bundles-without-helm-upgrades/Using S3 DAG bundles, a sidecar sync pattern, and a bundle watcher to onboard pipelines without Helm upgrades or downtime.Wed, 15 Apr 2026 00:00:00 GMTairflowkubernetesdata-engineeringdag-bundlesManoj Babu Katragadda, Meghanath MachaBuilding a Composable ETL Framework for Sparkhttps://tardunge.com/blog/building-a-composable-etl-framework/https://tardunge.com/blog/building-a-composable-etl-framework/Replacing bespoke PySpark scripts with a config-driven, hook-based framework inspired by Rust's composition model.Thu, 02 Apr 2026 00:00:00 GMTsparkdata-engineeringarchitecturepythonicebergManoj Babu Katragadda, Meghanath MachaTaming S3 Shuffle at Scalehttps://tardunge.com/blog/taming-s3-shuffle-at-scale/https://tardunge.com/blog/taming-s3-shuffle-at-scale/Fixing the GET request explosion, prefix throttling, and threading edge cases that emerge when S3 shuffle meets production scale.Mon, 30 Mar 2026 00:00:00 GMTsparks3-shuffleperformancedata-engineeringscaleManoj Babu Katragadda, Meghanath MachaSpark on Spot with S3 Shufflehttps://tardunge.com/blog/spark-on-spot-with-s3-shuffle/https://tardunge.com/blog/spark-on-spot-with-s3-shuffle/How S3 shuffle supports Spark executors on 100% spot capacity with Karpenter while cutting compute costs 70-85%.Wed, 25 Mar 2026 00:00:00 GMTsparkkubernetesspot-instanceskarpenters3-shufflecost-optimizationManoj Babu Katragadda, Meghanath MachaRunning Spark on Kubernetes: An Ops-First Approachhttps://tardunge.com/blog/running-spark-on-kubernetes/https://tardunge.com/blog/running-spark-on-kubernetes/An ops-first pattern for production Spark on Kubernetes with one SparkApplication template, pre-baked images, and sub-10-second warm starts.Thu, 19 Mar 2026 00:00:00 GMTsparkkubernetesdata-engineeringairflowdeveloper-experienceManoj Babu Katragadda, Meghanath MachaFrom 60 Minutes to 4: Optimizing Spark MERGE INTO on a 2 Billion Row Iceberg Tablehttps://tardunge.com/blog/spark-merge-optimization-from-60-to-4-minutes/https://tardunge.com/blog/spark-merge-optimization-from-60-to-4-minutes/Optimizing a daily Iceberg upsert from an hour to under 4 minutes with storage partition joins and shuffle hash hints.Thu, 19 Mar 2026 00:00:00 GMTsparkicebergperformancedata-engineeringManoj Babu Katragadda, Meghanath MachaBuilding Our Lakehouse with Apache Iceberghttps://tardunge.com/blog/building-our-lakehouse-with-iceberg/https://tardunge.com/blog/building-our-lakehouse-with-iceberg/A modern lakehouse architecture using Apache Iceberg, Lakekeeper, Spark, Trino, and AWS.Sun, 15 Mar 2026 00:00:00 GMTdata-engineeringiceberglakehousearchitectureManoj Babu Katragadda, Meghanath Macha