Flexible work model
Work from home, from the office, or in a hybrid format that supports focus and collaboration.
ABOUT THE ROLE
In this role, you will drive a critical migration of AWS data pipelines from HDF5 files to Iceberg tables, working with C++, Python, and large datasets to modernize a high-performance data architecture. Our Client is a London-based leader in harmonized historical order book and market data, powering advanced trading analytics and AI-driven insights. You’ll collaborate in a focused team to deliver scalable, efficient data foundations that directly impact global capital markets.
RESPONSIBILITIES
- Design and implement new AWS data pipeline components in C++ and Python to switch output from HDF5 files to Iceberg tables, improving efficiency and modernizing the data architecture
- Own end-to-end migration tasks within the AWS-based data pipelines, ensuring scalable handling of large datasets and minimizing risk to existing data flows and performance
- Collaborate closely with a small, focused team to coordinate changes across C++ services, Python tooling, and AWS data workflows during the migration from HDF5 to Iceberg
- Analyze existing C++ logic that generates HDF5 files and refactor or extend it to produce Iceberg tables, maintaining data integrity and schema consistency across large datasets
- Integrate Python-based utilities and scripts into the AWS data pipelines to support data transformation, validation, and loading of large datasets into Iceberg tables
- Optimize data processing paths in C++ and Python for large-scale datasets within AWS, aiming to enhance throughput, reliability, and scalability of the new Iceberg-based pipelines
- Troubleshoot, debug, and resolve issues arising during the transition from HDF5 file generation to Iceberg table creation in AWS, ensuring stable and predictable pipeline behavior
REQUIREMENTS
- Strong proficiency with modern C++
- Practical experience with C++ development for data processing pipelines, including performance optimization and debugging
- Hands-on Python skills and experience with building and maintaining data processing scripts and utilities
- Hands-on experience with designing, implementing, and maintaining data pipelines in AWS environments
- Upper-Intermediate English level or higher
- Analytical and solution-oriented mindset with strong problem-solving and troubleshooting skills in complex data environments
- Practical experience with data formats and serialization, including HDF5/Iceberg, and migrating or transforming data between formats (nice-to-have)
- Experience with big data frameworks or ecosystems (as an advantage)
SoftServe is an equal opportunity employer. Qualified applicants will receive consideration regardless of race, color, ancestry, ethnicity, national origin, religion, sex, sexual orientation, gender identity or expression, age, citizenship, disability, health condition, marital or family status, veteran status, or any other characteristic protected by applicable law.
Role Summary
Location
Poland, Romania, Ukraine
Work type
Remote/Office
Direction
Engineering & Technology
Subdirection
Software Development
Tech level
Senior
Apply Now
Fill out the form, and we'll be in touch shortly.
About us
We are a digital engineering and technology consulting company where expertise grows alongside people. For more than 30 years, we have been elevating technology: helping organizations navigate complex business challenges by combining deep engineering knowledge with thoughtful, research-backed innovation. Our teams work across key areas: digital engineering, data and analytics, Сloud, and AI/ML. In each, we deliver practical, scalable solutions rooted in real business needs and measurable human impact.
You bring your perspective and ambition. We create an environment where your work meets clarity, confidence, and purpose.
We offer
Compensation & Benefits
Competitive, market-based pay, benchmarked by role and location — plus health coverage, paid time off, wellness support, and learning opportunities.
People-first Leadership
Approachable leaders who communicate openly, keep teams close to the strategy, and support long-term planning.
Advanced tech communities
Stay close to AI/ML, Cloud, Quantum Computing, IoT, and Robotics communities, with projects built on modern frameworks.