DAY TO DAY:

We are looking for an exceptional Data Engineer to build the robust data pipelines that will power our client’s internal business analytics. Working under the guidance of the Principal Data Engineer, you will ensure that the raw data from multiple systems is consistently ingested, cleaned, and made ready for analysis. By building stable and efficient pipelines, you will directly support the timely generation of visualizations that leadership relies on to make informed decisions. This is a demanding role in a results-oriented environment with high expectations for agency, speed, and ownership. This team is a newly created team and operates as a stand alone function not part of any other function/team (not a part of a traditional IT team). This group will report in through the CEO of our client, and is a highly impactful team focused on the delivery of end to end consumables. 

• Pipeline Execution: Build and maintain Azure Data Factory pipelines to ingest data from multiple sources. 
• Silver Layer Transformation: Write the Python code in Databricks to clean raw data and move it into the silver layer, handling deduplication, type casting, and validation. 
• Reliability: Monitor daily jobs and troubleshoot failures. You are the first line of defense in ensuring that pipelines are stable and do not break. 
• Data Quality: Implement automated checks to verify that data arriving in the lake matches the source systems