Solution Architect (Databricks)

• Support pre-sales phases by providing the necessary information to define an architectural solution and its associated cost estimation.

• Definition of the solution architecture—from conceptual design to implementation decisions—documenting the fundamentals and the discarded alternatives.

• Management of conflicting constraints—such as performance, operating costs, security, maintenance, and delivery timelines—and clearly communicate the resulting trade-offs to the client.

• Guarantee architectural consistency throughout the project, verifying that implementations align with the design and intervening if the solution begins to deviate.

• Coordinate technical team operations: task assignment, verification of results, conducting technical reviews, and managing escalations.

• Management of architectural communication with the client's technical and business stakeholders: gathering requirements, explaining design decisions, resolving objections, and agreeing on implementation priorities.

• Design of agent-based architectures: defining the division between agents and tools, access to knowledge bases, model gateways, human-in-the-loop control points, and governance frameworks.

• Foster the technical growth of the team by evaluating new technologies and integrating findings validated in practice into the organization's capabilities.

We are looking for a Senior Data Architect (Databricks) profile with the following responsibilities:

• Data architecture: the ability to design a comprehensive data platform encompassing ingestion from heterogeneous sources, organization into refinement layers, batch and real-time processing, and data exposure to various consumers.

• Modeling: mastery of analytical models and data history strategies, with the ability to select the appropriate approach based on the specific use case rather than relying on custom.

• Large-scale processing: deep knowledge of distributed computing mechanisms, their limitations, and the factors that determine cost and performance.

• Big Data and Apache Spark in Databricks: mastery of the Spark ecosystem and the ability to size and optimize distributed workloads involving large volumes of data; this includes partitioning strategies, minimizing data shuffling between nodes, selection of storage formats and transactional tables, and management of computing costs. Hands-on experience in executing and optimizing these workloads in the Databricks runtime environment is required.

• Databricks Platform (preferred and differentiating skill): deep knowledge of the lakehouse architecture and its components, including the Delta Lake transactional table format and optimization best practices, the organization of refinement layers, declarative pipelines and workflow orchestration, the SQL engine for analysis, and the management of computing environments and access policies. Candidates must be able to justify configuration decisions and anticipate their impact on performance and cost.

• Databricks integration into the cloud ecosystem: ability to position the platform within the client's architecture in Azure or AWS, encompassing workspace design, networking and isolation, identity and access profiles, underlying storage, and integration with existing ingestion, orchestration, and data exposure services.

• Generative systems architecture: ability to design the framework for a model-based solution—including agent orchestration, vector and graph knowledge bases, gateways to model providers, observability, and access governance—leveraging Databricks native services for vector search, model deployment, and generative application evaluation.

• Data governance: cataloging, data lineage, quality, access management, and compliance with data protection regulations. Competence in using the Databricks unified catalog as a governance tool is required, covering object hierarchy, granular permission definition, lineage, and external data sharing.

• Integration: knowledge of application integration patterns—synchronous and asynchronous interfaces, messaging, events—sufficient to design the comprehensive solution and collaborate effectively with application architects.

• Industrialization: management of infrastructure as code, deployment automation, environment segregation, and platform observability; specifically in Databricks, this implies code version control, packaging and promoting assets between environments, and the automation of deployment pipelines.

• Economic aspects of the solution: ability to estimate and control the operating costs of a data platform and, in the case of generative systems, the cost per execution; at this level, it constitutes a fundamental architectural responsibility.

• Technological breadth: although Databricks is the primary platform, experience with at least one other cloud ecosystem or third-party data platform is required, as well as the ability to identify functional equivalents, evaluate vendor lock-in risks, and articulate Databricks' positioning against alternatives.

• Security: information classification, encryption, access segregation, and auditing, including knowledge of the specific requirements of regulated sectors.

• Madrid - hybrid modality.
• English, level B2-C1 (conversational level).