My client a clinical data company is seeking a Senior Data Engineer to join their team. The Role:
The Senior Data Engineer will provide hands-on data engineering and best practice support across all departments and customers.
Lead the way in defining architecture for all data engineering projects Be hands-on, always developing, running and enhancing data pipelines Define and maintain data models Review and analyse data for data quality Implement and maintain best practices for all software engineering Liaise with other departments to meet their data engineering requirements Provide technical leadership and mentor other data engineers Requirements Essential: Two or more years' experience designing, building and running data engineering projects in Azure (using, for example, Data Factory, Event Grid, Azure Storage) Experience of automating pipelines and container orchestration with AKS, App Service, service fabric, etc. Experience of building infrastructure with ARM templates or equivalent Experience of full life-cycle software development, to include AGILE, git, CI/CD Experience operating and supporting complex software products Two or more years' experience developing software, ideally in Python Data analytics skills and experience with SQL, R, and Python Knowledge, exposure and experience with Numpy, Pandas, Dask, Modin, Ray, Tidyverse, d(b)plyr Experience building self-service tooling and workflows for Machine Learning and Analytics users Knowledge, experience and strong leadership opinions of specific tooling (Hadoop, Kafka, Spark) to support technical architecture choices Desirable: Familiarity and experience with Medical data Experience running machine learning algorithms in the cloud (e.g. Azure AutoML)