Engineering & Technology
Middle
Ukraine

Senior Big Data Engineer (Python+Apache Iceberg+AWS experience))

About the role

In this role, you will contribute to large-scale data pipeline migrations, moving trading venue data from existing HDF5-based pipelines to modern Parquet and Apache Iceberg-based storage, while ensuring uninterrupted delivery to customers and downstream systems.

Our Client is a leading provider of harmonised historical market data and analytics, supporting trading and research decisions.

You will work within a focused team of Python and C++ engineers, modernising data parsing pipelines and infrastructure to improve performance, reliability, and scalability.

Responsibilities

  • Develop robust Python code for Iceberg-based data storage, retrieval, migration, and validation, working closely with C++ and Python engineers.
  • Migrate existing parsing processes to the new Parquet and Iceberg-based pipeline, focusing on simplicity, performance, and reliability.
  • Collaborate with business stakeholders and engineering teams to gather requirements, understand constraints, design technical solutions, and validate outcomes.
  • Build, test, and deliver production-ready functionality to support business requirements and ongoing pipeline improvements.
  • Contribute to the development and optimisation of existing parsing pipelines and C++ parsers (feed handlers).
  • Leverage AI coding tools to accelerate development while maintaining responsibility for code quality, consistency, and maintainability.
  • Understand end-to-end data flows, including data ingestion, processing, storage, and delivery to customers.
  • Communicate with internal technical teams and external data providers, including financial exchanges, to support data processing and operational requirements.
  • Provide second-line support for daily parsing processes and data-related issues, working alongside the wider development team.

Requirements

Essential

  • Professional experience in Python development, including writing reliable and maintainable production code.
  • Hands-on experience with Apache Iceberg and pyiceberg libraries.
  • Experience with AWS services, particularly S3 storage, and ideally AWS Glue Data Catalog.
  • Experience with system migrations, ensuring existing end-to-end functionality is maintained.
  • Experience working with large datasets and data-intensive applications.
  • Professional experience developing software in Linux environments.
  • Familiarity with industry-standard development practices, including version control, unit testing, and continuous integration.
  • Strong foundational software engineering skills, with the ability to develop correct and maintainable code independently and with AI assistance.
  • Ability to self-organise and deliver development tasks over projects lasting several months.
  • Strong problem-solving and communication skills.
  • Computer Science or another STEM-related degree.

Nice to Have

  • Experience with Apache Spark.
  • Experience working with large-scale distributed systems.
  • Experience delivering well-tested, production-level code as part of a larger codebase.
  • Exposure to equities or derivatives market data from financial exchanges, including protocols such as ITCH, GTP, EOBI, iMpact, Pillar, or Optiq.
  • Knowledge of containerisation technologies such as Docker.

SoftServe is an equal opportunity employer. Qualified applicants will receive consideration regardless of race, color, ancestry, ethnicity, national origin, religion, sex, sexual orientation, gender identity or expression, age, citizenship, disability, health condition, marital or family status, veteran status, or any other characteristic protected by applicable law.

#LI-Remote

Role Summary

Location

Ukraine

Work type

Remote/Office

Direction

Engineering & Technology

Subdirection

Data & Analytics

Tech level

Middle

Personal recruiter:

Anastasiia Spodynets

Personal recruiter

Apply Now

Fill out the form, and we'll be in touch shortly.

CV/Resume in English will speed up its processing time
Upload file

About us

We are a digital engineering and technology consulting company where expertise grows alongside people. For more than 30 years, we have been elevating technology: helping organizations navigate complex business challenges by combining deep engineering knowledge with thoughtful, research-backed innovation. Our teams work across key areas: digital engineering, data and analytics, Сloud, and AI/ML. In each, we deliver practical, scalable solutions rooted in real business needs and measurable human impact.

You bring your perspective and ambition. We create an environment where your work meets clarity, confidence, and purpose.

About us

We offer

Flexible work model

Work from home, from the office, or in a hybrid format that supports focus and collaboration.

Compensation & Benefits

Competitive, market-based pay, benchmarked by role and location — plus health coverage, paid time off, wellness support, and learning opportunities.

People-first Leadership

Approachable leaders who communicate openly, keep teams close to the strategy, and support long-term planning.

Advanced tech communities

Stay close to AI/ML, Cloud, Quantum Computing, IoT, and Robotics communities, with projects built on modern frameworks.

More opportunities available

Browse all open positions to find the best fit for your experience.

Browse all positions
103538