Purpose of the role:
This role will centre on the design, implementation and monitoring of data solutions for a Data Platform. Providing technical expertise in evaluating requirements, developing efficient artefacts and maintaining technical design documents.
Responsibilities:
- Provide expertise in evaluating requirements, creating technical design documents and applying an Agile development methodology
- Develop high-performing and secure data pipelines and data stores that support the cloud data strategy including ingestion, validation, data quality, transformation, integration and modelling activities
- Monitor and optimise the data platform to support data pipeline performance and contribute to continuing process improvement
- Deploy new or implement changes to existing data workflows in line with the Change Management Framework
Skills required:
- Proven experience working with Azure data platform services including Data Factory, Data Lake Storage Gen 2, Databricks, SQL Database, Synapse and Active Directory
- Experience using Azure DevOps to manage tasks and CI/CD development within an Agile framework
- Good working knowledge of parallel processing technology and data mart architectures
- Experience working with non-relational databases and unstructured data is beneficial