hotskills.tech

About Job

Skills: Azure Databricks, Apache Spark/PySpark, Azure Data Factory, Azure Functions, Delta Lake, ADLS Gen2, SQL, ETL/ELT pipelines, Data Lakehouse architecture, Data Modeling, Location: Alpharetta, Georgia Experience Required: 3 – 15 Years Compensation: USD 82000 - 123000 Visa Status: US Citizens, green card holder About The Role We are seeking a Senior/Lead Data Engineer with 8+ years of experience building scalable, high-performance data platforms on Azure & Databricks. This role involves end-to-end data engineering, architecture, and leadership across modern data lakehouse environments. Key Responsibilities • Architect and implement scalable data platforms and pipelines on Azure and Databricks. • Build and optimize batch & real-time data ingestion and transformation (Spark/PySpark). • Develop robust ETL/ELT workflows, ensuring reliability and performance. • Work with ADLS, Delta Lake, and Spark for large-scale processing. • Design data models (conceptual, logical, physical) for analytics and operational use. • Collaborate with cross-functional teams to convert requirements into technical solutions. • Troubleshoot performance issues and support production systems. • Ensure best practices for data governance, security, quality, and architecture. • Mentor junior engineers and contribute to engineering standards. Required Skills • 8+ years of enterprise data engineering experience. • Strong hands-on experience with: • Azure Databricks, Azure Functions, ADF • Apache Spark (PySpark) • SQL & performance tuning • Delta Lake (schema evolution, ACID, optimization) • Expertise in ETL/ELT, distributed systems, and cloud-native architecture. • Experience with data modelling (dimensional, normalized, lakehouse). • Exposure to streaming frameworks (Kafka, Event Hub). • Strong problem-solving and technical leadership skills. Preferred Skills • CI/CD using Azure DevOps, Git. • IaC tools (Terraform/ARM). • Azure Purview or other data governance tools. • Experience supporting ML or BI workloads on Databricks. Benefits • Medical, dental, vision insurance • Paid vacation, sick leave, holidays • Employer-matched 401(k) • Life, AD&D, short-term & long-term disability • Eligibility varies by employment type and state law job perk: Posted At: 3 days ago, job perk: Schedule Type: Full-time, job perk: Qualifications: No degree mentioned, job perk: Dental Coverage: True, job perk: Health Insurance: True, job perk: Paid Time Off: True