Author

Mohammed Yaseen

Building code-first ETL platforms at scale — PySpark · Kafka · AWS · LLMs

Bengaluru, Karnataka, India

Email Me LinkedIn GitHub

Years Experience

20+

Production Pipelines

99.9%

SLA Compliance

10M+

Events / Day

Engineers Led

60+

SolutionGigs Tools

About Me

Professional Summary

Platform Lead with 5+ years of experience architecting and owning 20+ production ETL pipelines built entirely in code — no drag-and-drop ETL tools, no black boxes. Expert in distributed systems engineering using PySpark, Scala Spark, Apache Kafka, and AWS at scale, processing billions of events with 99.9% SLA compliance.

Proven track record integrating LLM-powered workflows and RAG systems into live data platforms. AWS Certified Solutions Architect who leads cross-functional engineering teams, ships reliable financial-grade infrastructure, and drives measurable business outcomes — including $2M+ annual revenue impact for enterprise clients.

Work History

Professional Experience

Founder & Full-Stack Developer

SolutionGigs · solutiongigs.in

2024 – Present

Self-funded · Remote · India

Product & Full-Stack Engineering

Designed, built, and launched a 60+ tool free SaaS platform solo — file converters, AI-powered audio/video tools, developer utilities, and content generators.
Architected a Next.js 15 (App Router, SSG) + FastAPI (Python) full-stack system with real-time file processing, background jobs, and streaming API responses.
Integrated FFmpeg, Whisper AI, Pillow, PyMuPDF, and Claude API to power audio transcription, noise reduction, scan-to-PDF, AI captions, and image processing pipelines.
Built 30+ SEO-optimised blog posts with E-E-A-T signals, JSON-LD structured data (FAQPage, HowTo, BreadcrumbList), and OpenGraph metadata across every page.

Infrastructure & DevOps (Hostinger VPS)

Self-managed Hostinger KVM VPS (Ubuntu 22.04) — provisioned, secured, and maintained the entire server from scratch without managed hosting guardrails.
Containerised the full stack with Docker Compose (multi-stage Next.js build + FastAPI image); Nginx reverse proxy routes /api/* → FastAPI :8000 and /* → Next.js :3000.
Configured Let's Encrypt SSL, Nginx cache headers, gzip compression, and zero-downtime reload scripts for production deployments over SSH.
Set up Cloudflare CDN in front of VPS for DDoS mitigation, global caching, and HTTPS edge termination.

Platform Lead

Enterprise data platform (product engineering firm)

Jan 2024 – Present

Bengaluru, India

Custom ETL Platform Engineering

Own and maintain 20+ production ETL pipelines built entirely in PySpark and Scala Spark — zero reliance on GUI-based ETL tools; every transformation and orchestration is hand-crafted code.
Architected end-to-end data pipelines processing 10M+ events daily; reduced processing latency by 35% through custom checkpointing, partition pruning, and Spark job template libraries.
Designed data modeling frameworks achieving 99.9% SLA compliance for enterprise clients, with $2M+ annual revenue impact.
Built reusable Spark job template libraries with Maven, accelerating pipeline development by 50% and establishing architecture standards org-wide.
Implemented programmatic data quality validation frameworks (Python/Scala) that reduced data anomalies by 60%, with automated alerting and self-healing retry logic.

Real-Time Streaming & Distributed Systems

Designed real-time Kafka consumer pipelines ingesting 50,000+ messages/second into AWS Kinesis and Elasticsearch, with custom offset management and dead-letter-queue handling.
Engineered batch and micro-batch processing frameworks on AWS EMR and Glue (pure Spark jobs), processing 5TB+ daily with 99.5% accuracy.
Built AWS integrations via Boto3 (S3, Lambda, CloudWatch, Kinesis) with proactive monitoring agents preventing 20+ incidents monthly.

AI-Integrated Data Platform

Integrated LLM workflows (GPT-4 API) directly into production pipelines for automated classification and intelligent alerting, reducing manual intervention by 70%.
Built RAG-based knowledge retrieval systems using LangChain and vector databases, improving onboarding productivity by 30%.
Engineered prompt engineering patterns for automated report generation and anomaly detection, enabling scalable AI Ops for incident handling.

Engineering Leadership

Lead a team of 8 engineers delivering enterprise data solutions; reduced onboarding time by 40% via structured mentorship and code review culture.
Partnered with business, finance, and product stakeholders to ship 12+ platform initiatives on schedule.
Strengthened platform observability with Datadog, Grafana, and AWS CloudWatch — cut incident response time from 30 min to under 5 min.

🏆 Customer Appreciation Award — May 2024 & Jan 2024🏆 Technical Leadership & Innovation Award — 2024

Software Engineer

Enterprise data platform (product engineering firm)

Jan 2021 – Dec 2023

Bengaluru, India

Pipeline Development & Streaming

Built real-time streaming pipelines in PySpark and Scala Spark consuming Apache Kafka at 50,000+ messages/second — custom consumer logic and transformation layers, no ETL tooling.
Implemented data transformation code handling 5TB+ daily volume with 99.5% accuracy, powering Kibana dashboards used by 200+ users.
Developed Python-based monitoring agents using Boto3 with proactive CloudWatch alerting, preventing 20+ monthly incidents.

Backend & API Engineering

Built production CLI tools in Golang (Cobra-CLI, Viper, GorillaMux) with goroutine-based concurrency, improving throughput by 45%.
Developed Node.js backend controllers managing data ingestion from 1,000+ IoT devices with fault-tolerant message delivery.
Integrated Telegraf and InfluxDB for metrics collection; built Grafana dashboards tracking 50+ custom operational metrics.

Stack

Technical Skills

End-to-end expertise from raw data ingestion to AI-powered analytics

ETL & Pipelines

PySparkScala SparkApache KafkaAWS KinesisAWS EMRAWS GlueBatch & Streaming

Languages

PythonScalaGolangJavaNode.jsTypeScriptJavaScriptReact.js

AI & ML

LLM IntegrationGPT-4 APIClaude APIPrompt EngineeringRAGLangChainVector DatabasesAI Agent Dev

Cloud & Infra

AWS EC2S3LambdaCloudWatchKubernetesDockerTerraformCI/CDDatadogGrafana

Databases

PostgreSQLMongoDBElasticsearchInfluxDBKibana

APIs & Architecture

RESTGraphQLMicroservicesMavenBoto3FastAPI

Portfolio

Key Projects

Production-grade systems built from scratch, no drag-and-drop tools

SolutionGigs — Free Tools Platform

Live

Solo-built Next.js 15 + FastAPI SaaS platform with 60+ free tools. Self-managed on Hostinger VPS (Docker · Nginx · Let's Encrypt). Full SEO stack: JSON-LD schemas, sitemap, E-E-A-T blog, Google Analytics.

Enterprise Data Lake Architecture

AWS S3-based data lake with fully coded ingestion from 15+ heterogeneous sources processing 2TB+ daily — pure PySpark + Boto3 orchestration, zero ETL tools.

AI-Powered Log Analysis System

LLM-integrated log analysis using GPT-4 API with custom prompt engineering; reduced MTTR by 45% through intelligent pattern detection.

RAG Documentation Assistant

LangChain + vector database assistant; reduced engineer onboarding time by 60% and enabled self-serve knowledge retrieval at scale.

Custom Kafka Consumer Framework

Reusable Kafka consumer library in Python/Scala with multi-topic fan-out, schema registry integration, and dead-letter-queue routing; adopted across 10+ pipelines.

Side Project · Founder

SolutionGigs — Full-Stack Product

SolutionGigs is a free online productivity platform with 60+ tools spanning file conversion, AI-powered audio/video processing, developer utilities, and content generation. Every line of backend and frontend code, every Nginx rule, every Docker container, and every SEO structured-data block was authored by me — no team, no agency.

Hosted on a Hostinger VPS (KVM), the platform runs a FastAPI backend and a Next.js 15 frontend behind an Nginx reverse proxy, all containerised with Docker and served over HTTPS via Let's Encrypt.