Selected work, newest first. Filter by technology to see what I've built where, then expand any project for the detail.
Filter by technology
Open-Source JSON Query Library
Personal Projects2024-Present
Building a Python-native DSL query language for JSON traversal (open-source).
Python
What I did
Currently building a (soon-to-be) open-source Python-based JSON query library.
Python-native DSL query language for JSON traversal.
Stack
Python
DSL
JSON
Open Source
Library Development
Project Management & Delivery
Leadership2024-2025
Owned end-to-end delivery of data-engineering initiatives across Airflow-Snowflake and API-integration projects.
Infra & CI/CD
Leadership & Delivery
What I did
Owned end-to-end delivery of data-engineering initiatives (scope → plan → execute → release) across Airflow-Snowflake and API-integration projects.
[object Object]
Translated domain requirements into epics/user stories and design decisions; facilitated reviews to align business, data modeling, and orchestration patterns.
Established engineering standards (DoD, code review, CI/CD) to reduce deployment errors and speed up releases.
Coordinated cross-functional contributors and vendors, unblocked dependencies, and managed change requests to keep timelines intact.
Stack
Project Management
Agile/Scrum
Sprint Planning
Stakeholder Management
Requirements Analysis
CI/CD
Code Review
Cross-functional Collaboration
Vendor Management
Team Management & Leadership
Leadership2022-2024
Scaled and managed the pipeline-engineering team, driving recruitment, mentorship, and delivery excellence.
Leadership & Delivery
What I did
Managed and scaled the pipeline-engineering team from 4 to 6 engineers, leading internal recruitment, technical interviews, and onboarding.
Defined team scope and delivery workflows during project pre-engagement, ensuring smooth integration into existing client frameworks.
Advocated team capabilities during client discussions, directly contributing to contract extensions and project expansions.
Held regular 1:1s and coaching sessions to align engineers' technical growth with client and management expectations.
Provided mentorship and career guidance, supporting new joiners in both technical development and professional orientation.
Stack
Team Management
Recruitment
Technical Interviews
Onboarding
Mentorship
Career Coaching
Stakeholder Management
Project Planning
Client Relations
Leadership
AWS-Native News Ingestion & Data Scraping Pipeline
Cloud Architecture2024
Built a scalable, serverless data ingestion and scraping framework on AWS for a legal-domain AI startup.
Python
AWS
Serverless
Infra & CI/CD
What I did
Architected a scalable, serverless data ingestion and scraping framework on AWS, integrating Lambda, SQS, DynamoDB, OpenSearch, and CDK.
Designed ingestion logic for large volumes of publicly available legal documents to support LLM data pipelines.
Built an AWS-native feed → fetch → normalize system using AWS CDK (Python) with two stacks (Feed, Fetch) linked via SSM Parameter Store and orchestrated with Step Functions.
Implemented fault-tolerant processing patterns, including idempotent message handling and incremental indexing.
Extracted main text with readability-lxml + BeautifulSoup and wrote S3 normalized layer JSON only on meaningful change; maintained a latest.json pointer per document.
Redesigned social media platform marketing API integrations for a global non-profit, achieving 60% cost reduction.
Python
SQL
Azure
Serverless
Data Governance
What I did
Redesigned the legacy monolithic marketing API ingestion system, integrating metrics from LinkedIn, YouTube, Facebook, and Instagram.
Proposed an Azure serverless solution using Functions, Key Vault, Blob Storage, Azure Monitor, and Azure SQL, demonstrating improved reliability and ~60% projected cost reduction.
Successfully gained management buy-in for the Azure redesign after demonstrating its technical and cost benefits.
Identified inefficiencies and potential security issues in the legacy system.
Delivered an architecture design that improved pipeline reliability, data quality, and maintenance.
Stack
Azure
Azure Functions
Azure Key Vault
Azure Blob Storage
Azure SQL
Azure Monitor
Python
API Integration
LinkedIn API
YouTube API
Facebook API
Instagram API
Serverless Architecture
Cost Optimization
Architecture Design
Security
Snowflake Cross-Account Data Sharing Automation
Data Engineering2023-2024
Automated secure data sharing between company-managed and client-owned Snowflake accounts at scale.
Python
SQL
Snowflake
Data Governance
What I did
Designed and implemented a Python-based automation framework to streamline secure data sharing between company-managed and client-owned Snowflake accounts.
Developed automation scripts using the Snowflake Python Connector to parse and adjust DDL definitions, removing blockers such as joins in views.
Generated masking-policy scripts using Row-Level Access Control (RLAC) features.
Migrated 300+ tables and views across 60+ Snowflake instances, ensuring consistent access control, maintainability, and auditability.
Prepared and executed client-account setup commands for rapid provisioning.
Stack
Snowflake
Python
Snowflake Python Connector
Data Sharing
RBAC
Row-Level Access Control
DDL/DML
Data Security
Automation
Masking Policies
SQL
Python Auditing Framework for Pipeline Monitoring
Data Engineering2023-2024
Maintained and extended a YAML-configured Python auditing framework for business-critical pipeline monitoring.
Python
Snowflake
Data Governance
What I did
Acted as the go-to engineer for resolving business-critical issues in production data pipelines, including legacy and sunsetting systems.
Maintained and extended a YAML-configured Python auditing framework that periodically scans Unix NFS directories to detect anomalies.
Logs ingestion results to Snowflake via the Python Connector and triggers automated breach notifications via SMTP.
Supports dynamic reconfiguration through a client REST API, ensuring adaptability to evolving requirements.
Optimized a large-scale XML-to-CSV transformation (60 GB+) by implementing a parallelized processing approach in Python, cutting runtime from hours to minutes.
Stack
Python
Snowflake
Snowflake Python Connector
Unix/Linux
NFS
YAML
SMTP
REST API
XML Processing
Parallel Processing
Pipeline Monitoring
Data Quality
Performance Optimization
Problem Solving
PySpark GPS Location Data Integration
Data Engineering2024
Integrated GPS location data into Palantir Foundry using PySpark with parallelized API fetching.
Python
Spark
Palantir Foundry
Data Governance
What I did
Designed a Spark-based API fetching pipeline that parallelized per-device API calls to reduce end-to-end load time.
Implemented a controlled iterative fetch that concatenated each device's measurements into a single table, addressing unknown API concurrency limits and strict PII-removal requirements.
Flagged temporal gaps for offline devices and documented the pattern for future pipeline optimization.
Stack
PySpark
Palantir Foundry
API Integration
Python
Parallel Processing
PII Handling
Data Integration
Performance Optimization
Documentation
IoT Smart Home Camera Project
Personal Projects2024
Built an IoT smart-home project using ESP32 camera with AWS serverless backend.
Python
AWS
Serverless
Infra & CI/CD
What I did
Worked on an IoT project using an ESP32 camera using CDK as a smart-home project.
Upload photos to an AWS Lambda pipeline, process them with AWS Image Recognition Services.
Load the data into a database or S3 for further dashboard creation.
Stack
AWS
AWS Lambda
AWS CDK
S3
AWS Rekognition
IoT
ESP32
Python
Image Processing
Serverless Architecture
Snowflake Application Usage Metrics & Business Insights
Data Engineering2023
Designed and implemented aggregated usage-metrics views to enable business teams to analyze engagement patterns.
SQL
Snowflake
Data Modeling
What I did
Consulted with product stakeholders to design and implement aggregated usage-metrics views in Snowflake.
Built performant SQL logic using window functions, cross-view joins, and aggregation layers to classify users by behavioral patterns and activity levels.
Enabled business teams to analyze engagement patterns across internal web tools.
Stack
Snowflake
SQL
Window Functions
Data Modeling
Business Intelligence
Product Analytics
Stakeholder Collaboration
Performance Optimization
AWS Secrets Manager Integration
Security2023
Improved security for on-premise data-loading tool by integrating AWS Secrets Manager.
Python
AWS
Data Governance
What I did
Improved security for the company's on-premise data-loading tool by integrating AWS Secrets Manager using Python and the AWS SDK.
Ensured sensitive credentials were securely managed.