Engineering & Technology
Lead
Chile

Lead Big Data Engineer (Azure)

ABOUT THE ROLE

In this role, you will support and enhance a modern cloud-based data platform built on AWS, focusing on data ingestion, processing, and platform reliability. You will work with Python, PySpark, and Apache Spark to maintain and optimize existing data pipelines while collaborating with a distributed engineering team. The role involves working with healthcare data in a regulated environment and includes a three-month business trip to the United States in Q1 2027.

RESPONSIBILITIES

  • Support, maintain, and enhance an existing AWS-based data platform and data ingestion pipelines
  • Develop and optimize data processing workflows using Python, PySpark, Apache Spark, and AWS Glue
  • Monitor and troubleshoot data pipelines, ensuring reliability, performance, and data quality
  • Work with AWS services such as S3, Aurora PostgreSQL, Athena, Lambda, SQS, SNS, and EventBridge
  • Maintain and improve workflow orchestration using Apache Airflow
  • Support infrastructure and deployment processes using Docker, Terraform, and Amazon EKS
  • Apply software engineering best practices, including automated testing, code quality checks, and CI/CD processes
  • Utilize AI-assisted development tools such as Claude Code to support implementation, testing, and code reviews
  • Collaborate with remote engineering teams to resolve technical issues, implement improvements, and ensure platform stability
  • Support data security, governance, and compliance practices within a regulated healthcare environment

REQUIREMENTS

  • 5+ years of experience in Big Data or Data Engineering, including supporting production data platforms
  • Strong proficiency in Python, PySpark, and Apache Spark for large-scale data processing
  • Hands-on experience with AWS data services such as Glue, S3, Athena, Aurora PostgreSQL, and Lambda
  • Experience with Apache Airflow for data pipeline orchestration and scheduling
  • Strong knowledge of SQL, data ingestion, ETL processes, and data quality practices
  • Experience with Docker, Terraform, and CI/CD pipelines using GitHub Actions or similar tools
  • Familiarity with Python development and testing tools such as pytest, Pydantic, ruff, or mypy
  • Understanding of healthcare data regulations such as HIPAA/HITRUST, multi-tenant architectures, or data retention requirements is a plus
  • Readiness to travel to the United States for a three-month business trip in Q1 2027
  • Upper-intermediate or higher level of English, with strong communication and problem-solving skills


SoftServe is an equal opportunity employer. Qualified applicants will receive consideration regardless of race, color, ancestry, ethnicity, national origin, religion, sex, sexual orientation, gender identity or expression, age, citizenship, disability, health condition, marital or family status, veteran status, or any other characteristic protected by applicable law.

#LI-Remote

Role Summary

Location

Chile

Work type

Remote/Office

Direction

Engineering & Technology

Subdirection

Data & Analytics

Tech level

Lead

Personal recruiter:

Paulina Cortes

Personal recruiter

Apply Now

Fill out the form, and we'll be in touch shortly.

CV/Resume in English will speed up its processing time
Upload file

About us

We are a digital engineering and technology consulting company where expertise grows alongside people. For more than 30 years, we have been elevating technology: helping organizations navigate complex business challenges by combining deep engineering knowledge with thoughtful, research-backed innovation. Our teams work across key areas: digital engineering, data and analytics, Сloud, and AI/ML. In each, we deliver practical, scalable solutions rooted in real business needs and measurable human impact.

You bring your perspective and ambition. We create an environment where your work meets clarity, confidence, and purpose.

About us

We offer

Flexible Work Model

Work from home, from the office, or in a hybrid format that supports focus and collaboration.

Compensation & Benefits

Competitive, market-based pay, benchmarked by role and location — plus health coverage, paid time off, wellness support, and learning opportunities.

People-first Leadership

Approachable leaders who communicate openly, keep teams close to the strategy, and support long-term planning.

Advanced tech communities

Stay close to AI/ML, Cloud, Quantum Computing, IoT, and Robotics communities, with projects built on modern frameworks.

More opportunities available

Browse all open positions to find the best fit for your experience.

Browse all positions
103534