Specialist Databricks Engineer w/ Genie Experience

We are seeking an experienced Senior Databricks Engineer to lead the design, implementation and modernisation of enterprise data platforms using Databricks. This is a hands-on technical role suited to someone with deep Databricks and data engineering expertise, particularly across complex migration programs from legacy data platforms and data warehouses into modern Lakehouse environments.

The successful candidate will take technical ownership of key components of the Databricks implementation, working across architecture, engineering and business stakeholders to deliver scalable, production-ready solutions. Exposure to emerging AI capabilities is also highly regarded, particularly Databricks Genie, Databricks AI/ML capabilities, Azure AI Foundry, Retrieval-Augmented Generation (RAG) and Agentic AI.

Key Responsibilities

• Design, build and implement scalable enterprise data solutions using Databricks.
• Lead the technical delivery of migrations from legacy data platforms and data warehouses into Databricks.
• Assess existing architectures, pipelines and workloads and define appropriate migration and modernisation strategies.
• Design and develop robust data ingestion, transformation and processing pipelines.
• Develop and optimise solutions using Apache Spark, PySpark and SQL.
• Apply Databricks engineering best practices across architecture, development, testing, deployment, performance and maintainability.
• Troubleshoot complex data, integration and performance issues.
• Work closely with engineering, architecture and business stakeholders to translate requirements into production-ready technical solutions.
• Contribute to engineering standards, code reviews and technical governance.
• Provide technical leadership and guidance to other engineers where required.
• Support the adoption of AI-enabled capabilities across the data platform, including GenAI and conversational data experiences.



Highly Desirable Experience

Practical experience or exposure to one or more of the following would be highly regarded:

• Databricks Genie
• Databricks AI/ML capabilities
• Azure AI Foundry or comparable enterprise AI platforms
• Generative AI implementation
• Retrieval-Augmented Generation (RAG)
• Vector search and embedding-based solutions
• Agentic AI / AI agents
• Integration of AI capabilities with enterprise data platforms