End-to-end Azure data engineering pipeline ingesting real-time earthquake data from the USGS API. Implements a Bronze–Silver–Gold lakehouse using Azure Data Factory, Databricks, ADLS Gen2, and Synapse Analytics, with both manual execution and fully automated daily-triggered workflows.

This repository is dedicated to showcasing projects built on Databricks, focusing on big data analytics, data engineering, and machine learning workflows.

Hands-on project demonstrating how to load and query semi/unstructured data in Snowflake. Covered: Loading JSON data into a VARIANT column and extracting structured values using FLATTEN, Loading Parquet files from AWS S3 into relational tables with proper column mapping.

Automated Data Loading in Snowflake using Snowpipe (End-to-End Implementation).

Batch ingestion and ELT pipeline on GCP using Airflow, BigQuery, and dbt