SPlano, TXData Engineer (Python, PySpark, AWS)
Context
A US employer is hiring a contract data engineer for a 12+ month onsite assignment in Texas, working on Big Data pipelines on AWS. Candidates must be US citizens or green card holders and will go through a video interview.
What you'll do
- Build and maintain data engineering pipelines with Python and PySpark on AWS
- Work with S3, EMR and Lambda to process and move large data volumes
- Write infrastructure as code with Terraform
- Schedule and orchestrate data jobs with Airflow, Control-M or native schedulers
Profile
- Strong hands-on experience with Python, PySpark and Big Data technologies
- Practical experience with AWS S3, EMR and Lambda
- Working knowledge of Terraform, Java, REST APIs, MQ messaging, Kafka and Cassandra
- Experience with Airflow, Control-M or native schedulers; Databricks and machine learning model workflows are a plus