Technology stack
Azure Databricks, Microsoft Azure, Apache Spark, PySpark, Spark SQL, Python, SQL, Azure Data Factory, Azure Data Lake Storage Gen2, Delta Lake, Lakehouse architecture, Git
Client
•••••••'s client is a London-based international investor that operates in the private equity market.
Project overview
The Data Platform project focuses on delivering cloud based data engineering solutions in the Microsoft Azure ecosystem. You will work with Databricks, Spark, and Azure services to build and optimize reliable data pipelines and support modern Lakehouse architecture.
Team
Small team (1-10 people)
•••••••'s team in this project consists of 8 people who constantly interact with the technical team on the client side which consists of 20+ technical specialists. The work is SCRUM-based with weekly sprints.
Position overview
We are looking for a Strong Middle / Senior Databricks Developer with Azure to build scalable data platforms, develop complex data pipelines, and support cloud based data engineering solutions.
Responsibilities
Design data solutions using Databricks
Develop data pipelines for structured and unstructured data
Build ETL and ELT workflows in Azure
Develop high performance solutions using PySpark and Spark SQL
Design Lakehouse architectures using Delta Lake
Integrate data from databases APIs cloud services and streaming platforms
Optimize Spark workloads for performance scalability and cost efficiency
Ensure data quality governance security and monitoring standards
Collaborate with Data Engineers Data Architects Analytics teams and business stakeholders
Participate in architectural discussions and technical decision making
Support CI and CD processes and infrastructure automation practices
Requirements
4+ years of experience in Data Engineering
3+ years of hands on experience with Azure Databricks
Experience with Apache Spark and PySpark
Knowledge of Python and SQL
Experience with Azure Data Factory
Experience with Azure Data Lake Storage Gen2
Experience with Delta Lake and Lakehouse architecture
Experience implementing end to end data pipelines in Azure
Understanding of data modeling principles and best practices
Experience working with large scale datasets in cloud environments
Experience with Git based version control systems
Upper Intermediate or higher English level
Nice to have
Experience optimizing Spark workloads
Experience collaborating with cross functional technical and business teams
Experience supporting CI and CD processes
Experience with infrastructure automation practices
What We Offer:
Vacation days: Up to 26 business days per year.
10 illness/special days off per year (fully paid, no medical papers needed) for all contract types
Health and life insurance
MyBenefit platform with Multisport option
Internal psychological support service
English language classes from the first working day
Access to external learning platforms: O’Reilly, LinkedIn Learning, Udemy, and a wide catalog of diverse internal training
Flexible workplace: work from the office, from home, or choose a hybrid option
Tech Skills Mentoring Program
Opportunities to develop as a public speaker, mentor, or technical interviewer
Certification reimbursement (AWS, GCP, Microsoft, etc.)