Agentic AI for ITOps: Is your organization ready for the security challenges?
AI is set to provide increasing value in IT by enabling more ways to simplify complex IT operations, offering actionable insights, automating routine tasks with context-aware automation, and detecting abnormal events with advanced machine learning algorithms.
With agentic AI, systems can understand what is happening in an IT environment, reason about the most probable root causes, determine the right course of action, and implement these changes without requiring additional human intervention.
This capability creates an ideal shift in the way we think about AI systems. The question then is no longer about what an agentic AI system can do, but rather what degree of control the organization has over the agent.
This is where the challenges of AI governance come in.
As organizations provide AI agents with more autonomy, governance needs to evolve with it. An agent that can only analyze information is a much smaller risk than one that can access systems, use tools, and make independent changes. Applying the same controls to both will either place too much control on low risk use cases or leave a more risky agent without safeguards.
The focus, therefore, should be on creating the appropriate safeguarding around the level of autonomy an agent has. That means controlling access, limiting its impact, monitoring its progress, and making sure that human intervention is possible if the risk becomes unacceptable.
There are however several important security implications for IT leaders to consider before empowering AI agents with greater autonomy.
1. Knowing what the AI agents can access
An AI agent has to see in order to act. As such, it should be considered a privileged user within your organization.
AIOps agents could potentially interact with monitoring and ticketing systems, endpoints, clouds, databases, APIs, and security tools. What these agents can access and what they can do with the accessed data needs to be carefully controlled and audited at all times. This is why identity and access management (IAM) is the foundation of agent governance.
Organizations should define the following policies and controls:
What systems, actions, and data can an agent access?
Who owns the agent?
What IAM reviews are required?
An agent must have only the access required to perform its tasks. The agent access should be limited to what is necessary, even if this entails reduced automation.
2. Aligning autonomy to risk
Not every action performed by an agent carries the same risk.
An agent that performs an analysis of the root cause of an incident poses significantly less risk than one that takes direct action to resolve the incident. For example, an agent that recommends changes to the firewall rule set is considerably less risky than one that updates the rule set. The level of autonomy granted to an agent must therefore be aligned to the risk it poses.
One example of a risk-based approach is provided by ManageEngine’s Agentic AIOps framework, which defines four levels of operational autonomy.
Observe: The agent monitors the environment, analyzes it, and identifies incidents.
Recommend: The agent provides human-approved input for incident resolution.
Act: The agent performs predefined, low-risk operational tasks within the clearly defined guardrails.
Escalate: The agent stops operational tasks and escalates to human operators for higher-risk tasks.
The agent is authorized to perform predefined, low-risk operational tasks within clearly defined guardrails. The main point to consider is that as the autonomy of an agent increases, so should the level of controls and safeguards.
3. Defining data security requirements
An agent requires information in order to operate. The more data an agent can access, the greater the impact of a potential security incident.
IT environments contain a wealth of sensitive data, including configuration management databases (CMDBs), incident records, personnel details, network infrastructure information, internal documentation, and credentials.
The same principles apply to data security as they do to IAM. ManageEngine's approach to security places data governance alongside IAM and observability as a foundation for securing AI agents.
Prior to connecting an agent to a data source, IT leaders should ask:
Does the agent really need this data?
Is this data sensitive?
Is it subject to any regulatory requirements?
Is the integrity of this data verifiable and who can update it?
How long does the agent need access to this data?
What would happen if the information contained in this data was incorrect?
When designing the data access policy for an agent, it is important to remember that the wider the scope of visibility is, the more likely the policy is to be subject to regulatory requirements. This is especially true if an agent is capable of taking direct action based on the information it processes.
4. Considering prompt injection and other manipulation scenarios
An agent is only as secure as the information it consumes. It researches the potential solutions to an incident by scanning the internet, documents, emails, support tickets, or any other untrusted source of information that can be manipulated via prompt injection.
ManageEngine's research on agentic AI security highlights threats including prompt injection, memory poisoning and insecure tool integrations. AI prompt injection attacks can be used to influence the output of an agent. This is not necessarily a vulnerability in the design of an agent but a potential attack that should be understood and mitigated.
The risk is especially significant for agents that utilize tools to perform their tasks.
A prompt injection attack could be used to influence which tools an AI agent selects or their parameters. Similarly, an attacker could attempt to bypass any existing security controls and inject additional prompts that cause the agent to violate security policies.
It is therefore crucial for organizations to conduct comprehensive testing of agents under attack scenarios. Some of these scenarios should include:
Testing the behavior of an agent when instructed to perform actions not permitted by the agent governance policy.
Testing the behavior of an agent when:
Exposed to untrusted external content.
Presented with falsified operational data.
Instructed to utilize unapproved tools.
Trying to circumvent authorization, approval, or any other safeguards.
Presented with unconventional or unanticipated request chains.
By testing the behavior of an agent under such scenarios, IT leaders can better understand the ways in which an agent can be exploited. This, in turn, will allow the development of appropriate safeguards.
5. Establishing observability
With conventional automation tools, it is relatively straightforward for an administrator to determine the root cause of an observed behavior by tracing back to the rule script that triggered it. This is not necessarily the case with an agent that consumes multiple sources of information, interacts with various tools, and performs a number of operations in an attempt to resolve an incident.
IT administrators should be able to define what caused an agent to perform specific actions and what tools it used to enact these actions. In other words, they should be able to answer the following questions:
What did the agent do?
What triggered the agent to do it?
What information did the agent access?
What actions have been made?
What tools were used?
What authorization did the agent receive?
Did a human approve of the action?
What happened next?
Such visibility into the operations of an agent is vital not only for auditing purposes but also for operational security. Observability enables IT administrators to detect incidents, investigate the actions of an agent, and determine whether its autonomy should be restricted and how.
6. Designing agent brake policies
An autonomous agent needs an emergency brake. IT teams must always have the option to stop an agent, limit its permissions, or revoke its ability to enact changes to the IT environment.
ManageEngine's white paper on ITOps guidance includes reversibility and circuit-breaker mechanisms among the controls needed for safe autonomy.
Depending on the use case, organizations may want to consider implementing the following safeguards:
✓ Approval gates for high-impact operations
✓ Operational boundaries
✓ Escalation procedures
✓ Emergency brakes
✓ Reversibility mechanisms
✓ Operational autonomy thresholds
These policies and controls should be tested before an agent is deployed in a production environment. Ideally, such policies should always be in place, regardless of whether an agent is operating in a test or production environment.
The ability to stop an agent should not depend on the agent deciding to stop itself.
7. Developing life cycle governance policies for AI agents
An agent’s ability to act on its own carries risk, but an agent that continues to evolve poses an even greater threat. Designing governance strategies for AI agents that operate according to their risk profiles and degrees of autonomy is highly essential.
This means that IT leaders should consider their governance approaches for agents at five distinct points during an agent’s life cycle:
Before launch: Determine what the agent can do and who is responsible.
At the time of deployment: Establish monitoring and approval policies and procedures.
During operations: Review activity. Set incident management and approval policies.
During modifications: Take appropriate action whenever changes to an agent’s model, tools, data, or operations are made.
At retirement: Revoke access and remove any unauthorized tools and data access.
This way, organizations can ensure that an AI agent does not acquire unwarranted privileges by remaining in operation long after its initial purpose has ceased to exist.
Securing your future with agentic AI
Agentic AI could give IT more autonomy, but that autonomy needs to be tempered with the right amount of control. Organizations that are adopting AI agents must put their focus on what the agents can access and change, and what tasks require humans to step in. ManageEngine approaches agentic ITOps with an emphasis on controlled autonomy, observability, and safeguards.
To learn how agentic AI can be incorporated into your IT operations safely, download our white paper, From alerts to action: How agentic AI will change your ITOps.