Principal Data Platform Engineer (Databricks)

We are currently looking for a Principal Data Platform Engineer (Databricks) to join a modern data platform initiative for healthcare and life sciences clients. This role will focus on owning end-to-end architecture and delivery of scalable data solutions, working closely with data architects, analysts, and client stakeholders.

Location: Remote, United States

Key Responsibilities:

• Own end-to-end architecture and delivery of scalable data solutions, with strong emphasis on Databricks-based platforms and modern cloud ecosystems
• Lead the design and implementation of data pipelines, data models, and transformation frameworks supporting analytics, reporting, and advanced use cases
• Serve as the primary client-facing technical lead, building trusted relationships and guiding stakeholders through complex data decisions
• Translate ambiguous business requirements into clear technical architectures and delivery plans
• Establish and enforce best practices across data engineering — ingestion, pipeline orchestration, testing, optimization — and DevOps
• Drive platform strategy and architecture decisions, including lakehouse design, medallion architecture, and governance frameworks
• Lead and mentor delivery teams, providing technical guidance, code reviews, and hands-on support
• Collaborate with cross-functional teams — data architects, analysts, client stakeholders — to ensure alignment and value delivery
• Identify risks and proactively address challenges to ensure high-quality, on-time delivery
• Contribute to internal capability building: reusable frameworks, accelerators, and thought leadership
• Support business development by shaping technical solutions and contributing to proposals and client discussions

Requirements:

• 7+ years of data engineering experience, with clear progression into technical leadership and architecture ownership
• Deep expertise in Databricks and modern lakehouse architectures, including Delta Lake and Spark-based processing
• Advanced SQL and Python skills, with strong experience building and optimizing large-scale data pipelines
• Hands-on experience with cloud platforms (AWS, Azure, or GCP), including data services and infrastructure design
• Solid understanding of data modeling concepts, ETL/ELT patterns, and distributed data processing
• Experience with orchestration tools (e.g., Airflow) and transformation frameworks (e.g., dbt)
• Proven ability to lead technical delivery while staying hands-on
• Strong client-facing experience: requirements gathering, solution design, executive communication
• Ability to navigate ambiguity, prioritize effectively, and drive clarity in complex environments

 

Nice to Have:

• Healthcare data experience (e.g., Epic, HL7, FHIR, claims data)
• Experience with CI/CD, DevOps practices, and infrastructure-as-code tools