NAustin, TXSenior Data Engineer, Databricks and GenAI
Context
A client needs a senior data engineer to build and maintain data pipelines and organize large volumes of data for AI and generative AI applications. The role is onsite four days a week, with a possible conversion to a full-time position with the client.
What you'll do
- Build and maintain data pipelines on Databricks
- Organize large volumes of data and prepare it for AI and generative AI applications
- Build and manage production data pipelines
- Apply data quality checks and code testing practices
Profile
- 5+ years of experience in data engineering
- Strong hands-on experience with Databricks, PySpark, Python and SQL
- Experience with Databricks on Azure or GCP, including Delta Lake and Unity Catalog
- Knowledge of Git and good communication and problem-solving skills