Lead Data Engineer – Databricks & PySpark
Location: Boynton BeachSalary Range: CompetitiveIntroduction Join a dynamic team where you will lead the charge in designing and implementing data solutions using cutting-edge technologies. We are seeking a highly skilled professional with a passion for data engineering and a commitment to excellence.Required Skills & Qualifications
- 10 years of experience with Databricks and PySpark
- Proficient in designing and implementing data ingestion pipelines using Apache Spark and Databricks
- Strong experience in data architecture principles and practices
- Hands-on experience in building ETL data warehouse transformation processes
- Proficient in Big Data components such as Spark SQL, Data frames, HIVE, implemented using Databricks
- Experience in cloud-based environments, with AWS preferred
- Strong SQL and data analysis skills
- Understanding of Databricks-AWS integrations and Delta Lake on AWS S3
- Excellent technical, diagnostic, and troubleshooting skills
- Strong leadership and organizational abilities
- Effective communication and interpersonal skills
Preferred Skills & Qualifications
- Exposure to Data Vault Modelling
- Experience in building relational and dimensional data models
- Highly motivated and willing to learn new technologies
Day-to-Day Responsibilities
- Oversee the technical team and all projects they undertake
- Lead role in requirements analysis and solution design of varied design initiatives
- Responsible for design artifacts, data flow diagrams, and data models
- Contribute to strategic product roadmap definition and delivery
- Determine project requirements and develop work schedules for the team
- Engage with key stakeholders to ensure solutions meet current and future business demand
- Assist Product Owner in defining use cases, requirements, and project planning
- Provide technical direction, mentoring, and best practices to teams
- Collaborate with the team to identify and fix technical problems
- Communicate with clients to ascertain their needs
Company Benefits & Culture
- Inclusive and diverse work environment
- Opportunities for professional growth and development
- Supportive team culture with a focus on collaboration and innovation
For immediate consideration please click APPLY to begin the screening process with Alex.
Artech Information Systems