Data Engineer

As a Data Engineer within the Production team, you will play a critical role in the swift, accurate, and secure progression of client jobs. You will be responsible for ensuring the reliability of data outputs, standardising external data, and driving process improvements that enhance our overall efficiency.

This role bridges technical execution with operational excellence, requiring a proactive individual who is detail-driven, process-orientated, and eager to grow their skills within AWS and Databricks. You will work closely with Account Managers, Sales, and cross-functional teams to deliver high-quality data solutions that meet our clients' needs.

Key Responsibilities

Data Processing & Accuracy

• Ensure the swift, accurate, and secure progression of client jobs, performing bespoke file matches, standardisation, enhancement, and deduplication of external data.
• Ensure the highest standards of data accuracy and reliability in all outputs, actively monitoring for discrepancies to reduce errors and rework over time.
• Maintain strict adherence to security, confidentiality, and data compliance protocols in all data handling.

Technical Execution & Process Improvement

• Perform ad-hoc queries, counts, and data manipulation using Databricks, SQL, and FastStats.
• Identify, propose, and implement process improvements to enhance productivity, accuracy, and the overall efficiency of the Production team.
• Develop and maintain robust ETL (Extract, Transform, Load) logic tailored to production requirements.

Documentation & Operations

• Create, maintain, and update all in-scope documentation for the Production Team.
• Map and document comprehensive process flows for each job type within Databricks to ensure operational resilience and knowledge sharing.

Cross-Functional Collaboration & Customer Focus

• Collaborate effectively with Account Managers, Sales, Development, and Product teams to align data outputs with business and client expectations.
• Actively gather and respond to feedback from stakeholders to measure customer satisfaction and continuously improve the usefulness and quality of data outputs.

Training & Development

• Take ownership of your own learning path, setting self-objectives for skill growth, particularly in AWS and Databricks ecosystems.
• Promote and implement knowledge transfer amongst team members to elevate the collective technical capability of the Production team.

Skills and Experience

Essential

• Data Processing: 1-3 years of "hands-on" data processing experience, preferably working with name and address data used for marketing.
• Technical Proficiency: Strong practical experience with Databricks and SQL.
• Data Manipulation: Deep understanding of logical data manipulation processes, including data reformats, hygiene, enhancement, and deduplication.
• Quality Assurance: Proven ability to analyze datasets, spot anomalies, and implement rigorous testing/validation to ensure data integrity.
• Tools: Good working knowledge of the Microsoft Office suite (Word, Excel, Outlook).

Desirable

• Familiarity or experience with FastStats.
• Programming experience in Python or similar languages used for data engineering.
• Basic understanding or exposure to cloud platforms, specifically AWS.
• Knowledge of various industry suppression files.
• Experience with project management or ticketing tools (e.g., ClickUp).

Personal Attributes & Behaviours

To succeed in this role and align with Sagacity's core values, we are looking for someone who embodies the following traits:

• Self-Starter & Autonomous: Highly organised, efficient, and deadline-focused. You manage your own time effectively and take the initiative to solve problems.
• Detail-Driven & Process-Orientated: You pride yourself on quality delivery, paying meticulous attention to detail, and ensuring completeness in all the work you do.
• Agile & Curious: You have an inquisitive mind, embrace change, and are never afraid to ask questions to deepen your understanding or challenge the status quo.
• Trusted & Customer-Focused: You build strong relationships with clients and internal stakeholders by demonstrating uncompromised integrity, openness, and accountability.
• Clear Communicator: You practice open, honest, and simple communication, translating complex data concepts into understandable insights for non-technical stakeholders.
• One Team Player: You work in unity and collaboration with colleagues and clients, treating everyone as one big team working toward a shared purpose