GIS Jobs Ltd logo
Games

Databricks Solution Architect

Job Details

Work Mode

On-site

Experience

Mid

Employment

Full-time

Salary

£90,000 - £120,000 p/y

Posted Today

Tech Stack

Required technologies & tools

Azure Data Factory Key Vault Databricks Python Unity Catalog Azure Managed Identities PySpark Spark Sql Medallion Architecture Azure Data Lake Storage Gen2 Delta Lake Private Endpoints Lakehouse Architecture Azure Databricks Apache Spark

Sign in to see how your skills match.

About the Role

Consultancy client are looking to recruit a Databricks Solution Architect to define and deliver enterprise-scale data platform architectures using Azure Databricks.

The candidate must possess deep hands-on Databricks engineering knowledge alongside experience designing secure, scalable and highly available data platforms for UK Public Sector organisations. This is a technical architecture role requiring the ability to move between architecture design, technical implementation and engineering leadership.

Essential Technical Skills

  • 5+ years of hands-on Databricks platform experience.
  • Extensive Azure Databricks architecture experience.
  • Expert-level PySpark, Spark SQL and Python knowledge.
  • Deep understanding of Apache Spark processing architecture.
  • Extensive Delta Lake and Lakehouse architecture experience.
  • Strong experience designing Medallion Architecture.
  • Expert knowledge of Unity Catalog, governance, permissions and data lineage.
  • Experience designing Databricks workspaces, clusters, compute policies and networking.
  • Knowledge of private endpoints, managed identities, Key Vault and secure connectivity.
  • Experience with Azure Data Lake Storage Gen2, Azure Data Factory and Azure services.

Public Sector Experience

Extensive experience delivering enterprise data architectures

Understanding of UK Government security principles and data governance requirements.

Experience architecting platforms handling sensitive and restricted government data.

Knowledge of UK GDPR, Data Protection Act 2018 and relevant government security standards.

Experience producing architecture documentation, HLDs, LLDs and technical design authorities.

Experience working with public sector enterprise architecture and governance boards.

Security Clearance Requirements

Be eligible and willing to undergo SC clearance

Previous experience working on sensitive government projects is advantageous.

Responsibilities

  • Own the end-to-end Databricks solution architecture.
  • Define scalable and secure Lakehouse architecture.
  • Establish engineering and architecture standards.
  • Design data ingestion, transformation and consumption patterns.
  • Define Unity Catalog and enterprise data governance models.
  • Provide technical direction to data engineering teams.

Similar jobs

Connells Group HQ logo
Report Engineer
Connells Group HQ
Milton Keynes
£40,000 - £45,000 p/y
Power BI
Python
PySpark
Warehouse
Rodney Clarke Associates logo
Data Engineer
Rodney Clarke Associates
Coventry
£40,000 - £40,000 p/y
Azure Data Factory
Databricks
Python
Azure
VIA MATCH LIMITED logo
Data Scientist
VIA MATCH LIMITED
Milton Keynes
£45,000 - £53,000 p/y
Power BI
Databricks
Python
Azure
Amtis professional Ltd logo
Cloud Platform Manager
Amtis professional Ltd
Burton-on-Trent
£75,000 - £85,000 p/y
Python
Bicep
Paas
Infrastructure As Code
Michael Page Business Support logo
Azure Data Engineer
Michael Page Business Support
Manchester
£60,000 - £65,000 p/y
Databricks
Python
Data Factory
Azure