R&D Engineer 3, Software

Overview

Keysight is at the forefront of technology innovation, delivering breakthroughs and trusted insights in electronic design, simulation, prototyping, test, manufacturing, and optimization. Our ~16,800 employees create world-class solutions in communications, 5G, automotive, energy, quantum, aerospace, defense, and semiconductor markets for customers in over 100 countries. Learn more about what we do.

Our award-winningculture embraces a bold vision of where technology can take us and a passion for tackling challenging problems with industry-first solutions. We believe that when people feel a sense of belonging, they can be more creative, innovative, and thrive at all points in their careers.

We are looking for a Senior Observability Engineer to design, build, and continuously improve monitoring and observability for our enterprise environment. Most of the work focuses on on-premises systems: physical servers, virtualized infrastructure, and on-prem Kubernetes clusters, along with the infrastructure layers underneath them. Some cloud use cases are also in scope. Your goal is to give system owners and business stakeholders an accurate, actionable view of the health and uptime of the systems they depend on.

Responsibilities

Architecture and Design

Implementation

• Build and maintain monitoring with Zabbix, covering templates, discovery, triggers, proxies, and integrations.
• Design and maintain log collection, parsing, enrichment, and routing pipelines into Splunk, including forwarders, sourcetypes, field extractions, indexes, and searches.
• Contribute to metrics and visualization with Prometheus and Grafana, particularly for Kubernetes workloads and unified dashboards.
• Automate the configuration and onboarding of monitoring using infrastructure-as-code and APIs.

Agentic AI for Observability

• Design and implement AI agents that support observability workflows. Examples include alert triage and deduplication, cross-source event correlation, root-cause hypotheses, incident summarization, and runbook-driven remediation with human-in-the-loop approval.
• Integrate LLM-based agents with monitoring and logging platforms through APIs and tool protocols such as MCP.
• Define guardrails, permission boundaries, and auditability for AI-driven actions on production systems.
• Measure the real impact of these agents on operations, for example on MTTD, MTTR, alert noise, and on-call load.

Collaboration

• Work closely with system owners to understand architecture, failure modes, and dependencies, and to translate them into meaningful monitoring.
• Partner with consumers and stakeholders of those systems to define what “healthy” and “available” mean to them, and to make sure observability gives them the visibility they need.
• Provide technical leadership and mentoring, and run knowledge sharing across operations, infrastructure, and development teams.

Continuous Improvement

• Regularly review alert quality, coverage gaps, and signal-to-noise ratio, and tune or retire monitoring accordingly.
• Use incidents and post-mortems as input to improve detection and response.
• Track the evolution of the observability toolset and the wider technology landscape.

Required

Nice to Have

• Experience with Prometheus (exporters, alerting rules, Alert manager, long-term storage).
• Experience with Grafana (dashboard design, multi-source visualization, alerting).
• Experience monitoring cloud workloads (AWS, Azure, or GCP) and hybrid environments.
• Software development experience, such as building integrations, tools, exporters, or services.

Careers Privacy Statement***Keysight is an Equal Opportunity Employer.***