Bluelight is a leading software consultancy dedicated to designing and developing innovative technology that enhances users' lives. We are looking for a skilled Data Engineer to join our rapidly growing team and contribute to challenging and meaningful projects, developing high-quality applications that stand out in the market.

Requirements

  • ETL Data Engineering: Develop and maintain ETL data engineering processes using Python (PySpark) within Azure Synapse Analytics Notebooks, and/or Azure Synapse Analytics Pipelines
  • Data Warehousing: Apply your expertise in data warehousing, understanding star schemas, facts, and dimensions, to design and build effective data storage structures in a Massively Parallel Processing (MPP) SWL Pool
  • Data Source Expertise: Extract data from various sources, including REST APIs, SWL database tables, and CSV files
  • Azure Synapse Analytics Expertise: Utilize your deep knowledge of Azure Synapse Analytics to design and optimize data notebooks/pipelines for scalability and performance
  • Data Fabric Concepts: Contribute to the implementation and understanding of other Data Fabric concepts, such as data lakes, lakehouses, delta lakes, and data cataloging, to enhance data management capabilities
  • Data Modeling: Collaborate with data architects to create data models and schemas that align with business requirements
  • Data Quality: Implement data quality checks and validation processes to maintain data accuracy and consistency
  • Performance Tuning: Identify and resolve performance bottlenecks and optimize ETL data notebooks/pipelines to meet SLAs
  • Monitoring and Troubleshooting: Monitoring ETL jobs, diagnose issues, and implement solutions to ensure data pipeline reliability
  • Documentation: Maintain comprehensive documentation of ETL data engineering processes, data flows, and data transformations
  • Collaboration: Work closely with cross-functional teams to understand data requirements and provide support for data-related initiatives
  • Security and Compliance: Ensure data security and compliance with data governance and privacy standards

Benefits

  • Competitive salary and bonuses
  • Generous paid-time-off policy
  • Flexible working hours
  • Work remotely
  • Continuing education, training, conferences
  • Company-sponsored coursework, exams, and certifications