RJersey City, NJSenior Azure Databricks Engineer
Context
A large enterprise is running a Databricks implementation and cloud transformation initiative. The role is onsite in Jersey City, New Jersey, on a contract basis with a possible future conversion to full-time.
What you'll do
- Build and maintain ETL and ELT pipelines on Azure Databricks
- Tune Apache Spark and PySpark workloads for performance
- Implement data governance and organization with Unity Catalog and Delta Lake
- Support data ingestion and storage on ADLS Gen2 and Azure Data Factory
- Contribute to CI/CD and infrastructure as code with Git and Terraform
Profile
- Senior experience with Azure Databricks, Unity Catalog, Delta Lake and MLflow
- Strong skills in Python, PySpark, SQL and Apache Spark
- Experience with ADLS Gen2, Azure Data Factory and ETL/ELT pipelines
- Familiarity with Git, CI/CD, Terraform, Kubernetes and OpenShift
- Local candidates only, eligible to work on W2 or 1099 contract terms