Data Engineer
Context
A client in Chicago needs a Data Engineer on a contract of 6 months or more, with 3 days per week onsite in Chicago and an in-person interview required. The work centers on production data pipelines and reliable, scalable data processing.
What you'll do
- Build and maintain ETL pipelines in production environments
- Transform and script data using Python and SQL
- Develop distributed data processing with Apache Spark
- Build and operate data pipelines on Azure Data Factory and Databricks
- Support reporting in Power BI
Profile
- 9+ years of proven experience building and maintaining ETL pipelines in production
- Proficiency in Python and SQL for data transformation and scripting
- Hands-on experience with Apache Spark
- Experience with Azure cloud services, especially Azure Data Factory and Databricks
- Strong communication skills and ability to work in agile, collaborative teams