Senior Data Engineer
Senior Data Engineer
Join a rapidly growing scale-up where adaptability and problem-solving are key. As a Senior Data Engineer, you'll play a crucial role in designing, building, and optimizing data architecture to support evolving business needs. With expertise in data pipeline stacks (open-source & AWS) and ClickHouse, you'll develop scalable, high-performance solutions. Collaborate closely with engineering teams to implement robust data pipelines that ensure efficiency, reliability, and alignment with business objectives. If you're looking for a role where you can make a real impact, we'd love to hear from you.
Key Responsibilities:
Data Pipeline Design:
- Design and implement scalable, high-performance data architectures using AWS services and ClickHouse (or equivalent OLAP databases).
Data Pipeline Development:
- Design, build, and maintain robust ETL/ELT pipelines that efficiently handle large-scale data ingestion, transformation, and storage.
Cloud Infrastructure:
- Combine open-source data stack and AWS technologies to build and optimize data workflows.
Data Governance & Quality:
- Ensure data accuracy and consistency through best practices in data governance, lineage, and monitoring.
Performance Optimization:
- Optimize data storage, retrieval, and processing to support high-performance analytical workloads using partitioning, indexing, and query optimization techniques.
Collaboration & Leadership:
- Work closely with data scientists, analysts, and software engineers to understand requirements and deliver data-driven solutions, mentoring junior engineers.
Automation & CI/CD:
- Implement automated data pipeline deployment and monitoring strategies.
Requirements:
- 5+ years of experience in data engineering with solid experience on open-source data stack, and cloud native experiences.
- Expertise in ClickHouse, or similar OLAP databases is a plus.
- Strong experience in designing and architecting large-scale data systems.
- Proficiency in programming languages like Python, or productivity libraries scripting languages for data processing and pipeline development.
- Deep understanding of ETL/ELT processes, data modeling (Star Schema, Snowflake Schema), and data warehousing.
- Experience with orchestration tools such as Apache Airflow, Step Functions, or Dragster.
- Hands-on experience with infrastructure-as-code (Terraform, CloudFormation, CDK) is a plus.
- Strong problem-solving skills and ability to work in a fast-paced environment.
- Knowledge of SQL query performance tuning, materialized views, and sharding strategies for large datasets.
- Familiarity with containerization and serverless computing (Docker, Kubernetes).
- Experience with monitoring and observability tools such as Prometheus, Grafana, AWS CloudWatch.
FAQs
Congratulations, we understand that taking the time to apply is a big step. When you apply, your details go directly to the consultant who is sourcing talent. Due to demand, we may not get back to all applicants that have applied. However, we always keep your CV and details on file so when we see similar roles or see skillsets that drive growth in organisations, we will always reach out to discuss opportunities.
Yes. Even if this role isn’t a perfect match, applying allows us to understand your expertise and ambitions, ensuring you're on our radar for the right opportunity when it arises.
We also work in several ways, firstly we advertise our roles available on our site, however, often due to confidentiality we may not post all. We also work with clients who are more focused on skills and understanding what is required to future-proof their business.
That's why we recommend registering your CV so you can be considered for roles that have yet to be created.
Yes, we help with CV and interview preparation. From customised support on how to optimise your CV to interview preparation and compensation negotiations, we advocate for you throughout your next career move.