The English-language job board for Spain

Data Engineer – Spark / Scala / AWS

Remote from Spain·Added today (last checked 11 Nov)

AD4 Octógono

95 open roles

Apply

Our summary

Salary: €25,000 - €32,000 a year
About €1,650 - €2,040 a month after tax.
Work mode: Remote from Spain
Spanish: Expected (ad in Spanish)
English: Required
Experience: 2+ years

Visa options

The ad doesn’t mention sponsorship
Not a refusal, but not a yes. Worth asking in your first message.

Find jobs with visa support

Skills

This job advert was originally written in Spanish.

Requirements

What we're looking for

At least 2 years of professional experience in Data Engineering, with hands-on experience in development with Spark and Scala.
Solid knowledge of Apache Spark, Scala, and Hadoop.
Experience in the development and maintenance of large-scale data processing pipelines.
Experience in transformations, migrations, and table replacement, guaranteeing data integrity and consistency.
Knowledge of AWS services:
AWS EMR.
AWS Glue.
AWS Lambda.
AWS Step Functions.
Experience in orchestration, scheduling, and monitoring of batch processes.
Knowledge of Terraform for infrastructure configuration.
Experience in monitoring and resolving production incidents, including Hadoop/YARN log analysis.
Proficiency in SQL and experience with Amazon Athena or equivalent query engines.
Experience in data quality controls, reconciliation, duplicate detection, and information validation.
Scripting knowledge with Bash and Python.
Experience working in Unix/Linux environments.
Version control with Git and build tools such as Maven.
Knowledge of CI/CD processes and tools such as Jenkins.
Experience with Jira.
English C1, essential for communication in an international professional environment.

Nice to have

Experience in banking or financial services.
Knowledge of regulatory reporting processes.
Experience in retail, business, or corporate banking.
Ability to interpret business requirements and convert them into technical specifications.
Experience in defining test scenarios and independent validation of developments.
Knowledge of Control-M for scheduling and managing batch processes.
Familiarity with Business Intelligence platforms, especially Power BI.
Experience preparing technical and operational documentation.

Pay & benefits

What you'll get

Salary: €25,000 - €32,000
Remote Modality
Participation in a specialized project in data engineering and processing.
Technological environment based on Apache Spark, Scala, and AWS cloud services.
Work on critical data processes and regulatory reporting.
Possibility of joining at one of our offices, with other locations also being considered.

If you have experience in Data Engineering, you master Spark and Scala, and you want to participate in data processing projects in cloud environments, we want to meet you!

The role

We are looking for a Data Engineer specialized in Spark and Scala, with experience in the development and maintenance of large-scale data processing processes and knowledge of AWS cloud services.

The selected person will participate in a data engineering project within a financial and regulatory reporting environment, collaborating with business analysts and data modeling specialists to transform functional and regulatory requirements into technical solutions.

We are looking for a profile with the ability to develop ETL processes, manage data migrations, automate processing workflows, and guarantee the quality, integrity, and availability of information in production environments.

What you'll do

Design, develop, maintain, and optimize data processing pipelines using Spark and Scala.
Implement ETL processes and complex information transformations.
Develop migration and table replacement strategies, guaranteeing data integrity and consistency.
Collaborate with business analysts and data modeling specialists to translate functional and regulatory requirements into technical specifications.
Orchestrate and monitor batch processes using AWS services, especially Step Functions.
Work with AWS EMR, Glue, and Lambda.
Manage parameterized executions and processing workflows.
Participate in the configuration and automation of infrastructure using Terraform.
Develop and execute data validation and reconciliation processes.
Perform quality controls using SQL and Amazon Athena.
Detect null values, duplicates, and inconsistencies.
Implement automated tests and controls that guarantee the reliability of information.
Monitor and resolve incidents in production processes.
Analyze Hadoop/YARN logs, including stdout and stderr, to identify errors and determine their causes.
Perform root cause analysis and propose corrective and preventive measures.
Document developments, technical changes, tests, and operating procedures.
Guarantee compliance with development standards and regulatory requirements.

Personal competencies

Analytical capacity and structured problem-solving.
Autonomy and responsibility for deliverables.
Results-oriented with attention to detail.
Ability to investigate incidents and find their root cause.
Organization and priority management.
Adaptability and ease in understanding existing architectures and systems.
Ability to work autonomously and collaborate with multidisciplinary teams.

About AD4 Octógono

AD4 Octógono is a Spanish IT services company offering specialized consulting in IT and utilities. Its services include IT consulting, IT outsourcing, IT headhunting, cloud services, managed services, business intelligence, and outsourcing and headhunting for the utilities sector.

AD4 Octógono is the company within Grupo Audinfor specialized in outsourcing and IT solutions, aiming to support organizations in their digital transformation by providing talent, technology and methodology. The company describes itself as dynamic and growing, with continuous development programs for its employees and collaborative project work.

Good to know if you are moving

Many roles are offered as fully remote within Spain (e.g. 'Desarrollador/a Python Junior - REMOTO', 'Full Stack Developer – Java / Angular - REMOTO (ESPAÑA)').
Other roles are hybrid, based in cities such as Madrid, Sevilla, Bilbao, Málaga and Barcelona.

All open roles at AD4 Octógono