Observability Engineer – Behavioral & Auditability Signals

We are looking for an Observability Engineer to build behavioral monitoring and auditability capabilities for an enterprise AI Agent Platform being developed from the ground up. The role will focus on transforming agent telemetry, behavioral anomalies, safety signals, and governance events into actionable dashboards and alerts.

You will work at the intersection of observability, AI agent behavior, and governance, helping ensure that production agent activity can be effectively monitored, validated, and surfaced for both operational and audit purposes.

The project is focused on building an enterprise-grade AI Agent Platform from the ground up. The platform will provide a standardized foundation for developing, deploying, orchestrating, securing, and observing AI agents across multiple teams and use cases.

The initiative covers the full agent lifecycle and brings together agent orchestration, observability, security, governance, integrations, evaluation, and platform engineering. Engineers joining the project will have an opportunity to influence key architectural and technical decisions and contribute to a new platform rather than maintaining an existing solution.

• Design and implement behavioral monitoring dashboards and alerts using New Relic and NRQL.
• Define and monitor behavioral signals including anomalies, drift, safety events, and policy denials.
• Build auditability views that surface relevant governance and compliance evidence.
• Maintain clear segregation between technical observability and audit/security evidence across monitoring systems.
• Configure, tune, and validate alerts based on production agent telemetry.
• Verify dashboard accuracy and telemetry consistency to ensure monitoring data is reliable.
• Develop Python-based utilities for metrics emission, validation, and observability testing.
• Implement monitoring patterns for AI agent anomaly and behavioral drift detection.
• Validate that sensitive information, including PII, is appropriately handled in telemetry and traces.
• Incorporate security and governance signals, such as policy decisions and access denials, into behavioral monitoring.
• Collaborate with security and governance stakeholders to support enterprise compliance and audit requirements.

• New Relic and NRQL for behavioral dashboards and alert conditions

• Behavioral monitoring signals (anomaly detection, drift, safety signals, policy denials)

• Auditability views surfacing governance evidence for consumption

• Observability domain segregation (Technical in CloudWatch/New Relic vs Audit in dedicated evidence/SIEM stores)

• Alert validation and dashboard data-accuracy verification

• Python (metrics emission and validation)

• 4+ years of observability engineering

• Behavioral / anomaly monitoring and alert tuning on production telemetry

• Working in environments with formal audit/compliance evidence requirements

• Anomaly/drift detection techniques for agent telemetry

• PII redaction validation in traces surfaced through dashboards

• Cedar policy decisions / Contract-Policy Engine denials as monitoring inputs

• Enterprise security/compliance standards mapping