Harness Brings CI/CD to AI Agents: The End of Unpredictable Deployments

AI agents are notoriously unpredictable, making traditional software testing nearly impossible. To bridge this gap, Harness is extending its continuous integration and continuous delivery (CI/CD) platform to help DevOps teams build, test, and deploy artificial intelligence agents with the same rigor as traditional applications.

This update is critical for platform engineers and DevOps teams struggling to govern generative AI tools. By integrating agent management into familiar workflows, organizations can finally track how probabilistic models make decisions and prevent rogue agents from causing production failures.

The core challenge with AI agents is that they are probabilistic rather than deterministic. Given the exact same input, an agent might choose a different tool or take a different action from one run to the next, meaning a test that passes once offers no guarantee it will pass again.

AI agents are essentially a new type of software artifact that DevOps teams need to be able to build and deploy much like any other application.

- Trevor Stuart, Senior Vice President and General Manager, Harness

To address this unpredictability, Harness has introduced a suite of deployment and tracing tools designed specifically for agentic workflows:

• AI Asset Catalog: Automatically discovers agents, skills, and plugins across repositories while identifying their internal owners.
• Harness AI Evals: Measures agent quality and establishes deployment gates to automatically catch regressions when underlying models or prompts change.
• Agent Deployments: Extends canary releases, approvals, and Open Policy Agent (OPA) guardrails to managed runtimes, including Amazon Bedrock AgentCore and Google's Agent Runtime.
• AI Configs: Provides feature management capabilities to adjust prompts and models dynamically at runtime.
• Harness AgentTrace: Records the exact decision path an agent takes during multi-step sessions. Harness is also open-sourcing the foundational components, harness-sdk and harness-evals, allowing developers to embed tracing primitives directly into their own applications.

Beyond deployment, securing autonomous agents requires mapping their dependencies and restricting their access to sensitive data. Harness has rolled out several security capabilities to constrain what agents can do before they ship.

• AI Bill of Materials (AIBOM): Captures every model, tool, and dependency associated with a specific agent.
• AI Testing: Subjects agents to adversarial inputs based on the OWASP Top 10 large language model (LLM) and Agentic AI risks framework.
• Agent Discovery and Posture Management: Continuously maps agents as they spin up and connect across the enterprise infrastructure.
• AI Firewall: Enforces real-time policies to block prompt injection, tool misuse, and data exfiltration.

The fundamental conflict here is that CI/CD was built on determinism - if a test passes, it should always pass. AI agents break this rule entirely. Harness's approach acknowledges that we can no longer test for exact outputs; instead, we must test the boundaries of an agent's behavior.

As enterprises scale from dozens to thousands of autonomous agents, the bottleneck will not be building them, but rather proving they will not hallucinate or leak data in production. This suite positions Harness not just as a deployment pipeline, but as a mandatory governance layer for the agentic web.