Assignment Description

We are seeking a skilled Data Architect / Team Leader to design and implement scalable data pipelines using Databricks (Spark + Delta Lake) across both batch and streaming workloads. This role includes integrations with Kafka and medallion architecture patterns (Bronze/Silver/Gold).

The successful candidate will:

  • Develop and orchestrate metadata-driven ingestion frameworks.

  • Manage custom business logic transformations through Databricks notebooks and workflow orchestration.

  • Build and maintain semantic data models to support Power BI analytics.

  • Ensure reliable, governed datasets for business reporting.

  • Work with event-driven data architectures.

  • Utilize cloud orchestration tools such as Azure Data Factory (ADF) for automated data ingestion and processing.

Key Requirements:

  • Strong experience with Databricks, Spark, and Delta Lake.

  • Familiarity with Kafka and medallion architecture patterns.

  • Proficiency in developing metadata-driven ingestion frameworks.

  • Experience in building semantic data models for analytics.

  • Knowledge of event-driven data architectures.

  • Working knowledge of Azure Data Factory (ADF).

  • Experience in migration to Fabric.

  • Experience in AI is a strong plus.

Other Details:

This project is a remote opportunity with very occasional trips to the offices in Lisbon, allowing for flexible work arrangements. Ideal for professionals aiming to work on cutting-edge data architecture initiatives.