Senior Data Engineer
Technical Requirements:
- Skilled in designing and building complex distributed systems.
- Proficient in modern batch and streaming data ecosystems.
- Expertise in Java, Scala, or Python, along with SQL and Bash.
- Capable of leading, organizing, and managing team projects.
- Experienced in project management and reporting.
- Mentorship abilities for junior team members on engineering best practices and code quality.
Responsibilities:
- Develop and oversee data platform and assist its users.
- Collaborate with Traders and Researchers to identify and integrate new data sources for analysis to enhance trading opportunities.
- Design and create data solutions for discovering, purchasing, organizing, tracking, and managing data rights and quality to meet the strategic needs of trading teams.
- Continuously monitor data ingestion pipelines and data quality to ensure system stability, reliability, and data integrity. Also, contribute to the development and enhancement of monitoring and quality control systems and processes.
Qualifications:
- Over 7 years of experience in modern data technologies and building data-heavy distributed systems.
- Advanced proficiency in Java, Scala, or Python with a track record of producing high-quality, maintainable code.
- In-depth knowledge of SQL and Bash.
- Experience in utilizing and developing cloud-native technologies for scalable data processing.
- Previous work with both batch and streaming data systems, with a clear understanding of their limitations.
- Familiarity with various data processing technologies like Flink, Spark, Polars, Dask, etc.
- Knowledge of diverse data storage technologies such as S3, RDBMS, NoSQL, Delta/Iceberg, Cassandra, Clickhouse, Kafka, etc.
- Proficiency with various data formats and serialization systems including Arrow, Parquet, Protobuf/gRPC, Avro, Thrift, JSON, etc.
- Experienced in managing intricate data ETL pipelines using tools like Kubernetes, Argo Workflows, Airflow, Prefect, Dagster, etc.
- Prior experience in schema governance and managing schema evolution.
- Experience in developing quality control processes to identify and rectify data discrepancies.
- Motivation to mentor junior team members and uphold high standards of engineering practices and code quality.
- Strong problem-solving skills in technical contexts.
- Demonstrated ability to thrive in an agile, fast-paced environment, managing multiple tasks and projects efficiently in a trading context.
FAQs
Congratulations, we understand that taking the time to apply is a big step. When you apply, your details go directly to the consultant who is sourcing talent. Due to demand, we may not get back to all applicants that have applied. However, we always keep your resume and details on file so when we see similar roles or see skillsets that drive growth in organizations, we will always reach out to discuss opportunities.
Yes. Even if this role isn’t a perfect match, applying allows us to understand your expertise and ambitions, ensuring you're on our radar for the right opportunity when it arises.
We also work in several ways, firstly we advertise our roles available on our site, however, often due to confidentiality we may not post all. We also work with clients who are more focused on skills and understanding what is required to future-proof their business.
That's why we recommend registering your resume so you can be considered for roles that have yet to be created.
Yes, we help with resume and interview preparation. From customized support on how to optimize your resume to interview preparation and compensation negotiations, we advocate for you throughout your next career move.