Data Engineer

Data Engineer

$100,000- $140,000

This position is a hybrid opportunity for a Data Engineer and contract-to-hire in nature.

The hiring team is not open to providing sponsorship now, or in the future and is not open to a corp-to-corp arrangement

Responsibilities:

• Develop and maintain scalable data pipelines and workflows using Databricks.
• Process and integrate structured and unstructured data with Delta Live Tables and Streaming Data Processing.
• Prepare and manage datasets for machine learning workflows, optimizing for training and inference.
• Collaborate with data scientists and AI teams to support AI/BI Genie integrations.
• Implement Snowflake and Star Schema architectures to enhance data storage and query performance.
• Ensure BI reporting pipelines are compatible with advanced analytics tools.
• Document data processes and architectures thoroughly.
• Optimize Spark jobs and workflows for performance and cost-efficiency.
• Troubleshoot and resolve issues related to data quality and pipelines.
• Stay current with advancements in data engineering, machine learning, and AI technologies.

Requirements:

• Databricks Certification (e.g., Databricks Certified Professional Data Engineer) preferred.
• 3–7 years of experience in data engineering roles.
• Proficiency in Python, SQL, and Apache Spark.
• Experience with Delta Live Tables and big data processing frameworks.
• Understanding of machine learning models, pipelines, and AI tools.
• Familiarity with AI/BI Genie or similar business intelligence tools.
• Experience with Data Warehouse Architecture (e.g., Snowflake, Star Schema, Kimball methodology).
• Knowledge of cloud platforms like AWS, Azure, or GCP.
• Strong problem-solving skills and ability to work collaboratively.