Data Engineer PySpark
Context
Long term contract for a corporate client in Charlotte, NC, with a hybrid setup of 3 days onsite per week. The team works on large data volumes and reliable data solutions across functions.
What you'll do
- Design, develop and maintain scalable data pipelines using PySpark
- Process and transform large volumes of data
- Build and optimize ETL and ELT workflows
- Ensure data quality, accuracy and pipeline performance
- Collaborate with cross functional teams to deliver reliable data solutions
Profile
- 9+ years of experience in data engineering or related roles
- Strong hands on experience with PySpark and Apache Spark
- Strong Python programming skills
- Experience with SQL, ETL/ELT pipelines and Spark job optimization and tuning
- Cloud experience with AWS, Azure or GCP is a plus, Databricks and Delta Lake experience is a plus