Overview

Join our team as a Lead Data DevOps Specialist (Azure) where you will take charge of designing and refining data infrastructures and workflows. You will oversee scalable data implementations on Azure and collaborate with data engineers and other departments to enhance pipeline efficiency, system dependability, and performance. Apply now to contribute to cutting-edge data operations and cloud solutions.

Responsibilities

  • Create and manage data infrastructure utilizing Azure services such as Data Lake (ADLSv2), Databricks, Synapse, and Data Factory
  • Work in partnership with data engineering teams to design and optimize pipelines and workflows
  • Develop automation scripts in Python to increase operational reliability and efficiency
  • Implement and maintain CI/CD pipelines with Jenkins, GitHub Actions, or other technologies
  • Coordinate with cross-functional teams to boost the scalability, reliability, and overall effectiveness of data systems
  • Install and manage data tools including Apache Spark and Apache Kafka in cloud and on-premises settings
  • Supervise data system performance and troubleshoot issues related to scalability
  • Resolve complex technical problems involving data pipelines and platforms

Requirements

  • At least 5 years in Data Engineering or related disciplines
  • Minimum 1 year experience leading development teams
  • Strong Python skills, especially in batch processing workflows
  • Expertise in SQL for large-scale data management and queries
  • Comprehensive experience with Azure cloud services supporting data infrastructure
  • Hands-on experience with Infrastructure as Code tools such as Ansible, Terraform, or CloudFormation
  • Proficiency in setting up CI/CD pipelines using Jenkins, GitHub Actions, or similar tools
  • Practical knowledge of data tools like Spark, Airflow, or R
  • Advanced Linux OS skills including scripting and system admin tasks
  • In-depth understanding of network protocols including TCP, UDP, ICMP, DHCP, DNS, and NAT
  • B2+ level or higher English communication skills

Nice to have

  • Knowledge of additional cloud platforms such as AWS or GCP
  • Experience with Kubernetes for workflow orchestration
  • Familiarity with monitoring tools like Prometheus, Grafana, or Azure Monitor
  • Understanding of Big Data technologies and analytics
  • Experience with cloud-based data governance and security practices

[GTS] Benefits (generic, except India)

  • International projects with top brands
  • Work with global teams of highly skilled, diverse peers
  • Healthcare benefits
  • Employee financial programs
  • Paid time off and sick leave
  • Upskilling, reskilling and certification courses
  • Unlimited access to the LinkedIn Learning library and 22,000+ courses
  • Global career opportunities
  • Volunteer and community involvement opportunities
  • EPAM Employee Groups
  • Award-winning culture recognized by Glassdoor, Newsweek and LinkedIn