Data Engineer

Participate in the design, development, maintenance, and evolution of data models on the corporate data platform, guaranteeing the quality, stability, reuse, and maintainability of the implemented solutions. Develop data integration processes, apply engineering best practices, and collaborate in the adoption of new technologies and methodologies within the Microsoft ecosystem. Capable of both understanding and improving legacy code, as well as designing models from scratch. You will act as a technical data engineering reference within the Area, contributing to the use and definition of patterns, standards, and development criteria for the new architecture.

• Data engineering: design, development, and maintenance of ETL/ELT processes; integration of multiple heterogeneous data sources; development of loading and transformation processes; optimization of processes and performance; advanced use of SQL, Python, and Microsoft data integration and analytics technologies, especially Microsoft Fabric.

• Development and deployment: use of Git and version control; application of deployment processes between environments (CI/CD); following development and deployment methodologies and best practices; production and supervision of technical documentation.

• Modeling and architecture: data modeling with a focus on performance, reuse, and maintainability; application of Lakehouse architectures, modern analytical models, and Medallion architecture. Ability to evaluate and justify decisions between storage and processing patterns based on the needs of each case. Analysis, stabilization, and evolution of legacy processes, as well as design and execution of their progressive migration toward the target architecture, ensuring service continuity during the transition.

• Operation and reliability: design of resilient, idempotent, and recoverable processes; error management, retries, and reprocessing; execution monitoring; root cause analysis and resolution of production incidents.

• Data quality and testing: definition and implementation of automated data quality controls, reconciliation with source systems, unit and integration testing, and validation of business rules across the different layers of the model.

• Security and technical governance: application of security criteria, segregation of access, protection of sensitive information, traceability, lineage, and compliance with data governance standards.

• Supervision and continuous improvement: understanding and supervision of third-party developments, including providers and junior profiles; identification of technical debt; proactive communication of risks, blockers, and deviations. Optimization of performance and capacity consumption, analyzing volumes, partitioning, storage formats, concurrency, and execution cost.

• Documentation: ability to document data models and ETL/ELT processes in an objective and clear manner.