Security Measures for the Agentic and Physical AI Era

A Practical Guide for IT and Information-System Officials in Korean Public Institutions

At a Glance

Artificial intelligence is moving beyond systems that simply answer questions or generate content.

The next generation of AI includes Agentic AI, which can plan and execute tasks by using external tools, and Physical AI, which can interact with the physical world through robots, drones, vehicles, sensors, and actuators.

For public-sector organizations, the most important change is this:

AI is evolving from a system that provides answers into a system that can take actions.

This changes the security model.

A public institution must consider not only the AI model and data, but also the AI agent's memory, tools, permissions, communications with other agents, external services, sensors, actuators, physical interfaces, safety mechanisms, and emergency controls.

The AI Security Guidebook identifies 11 security threats and 17 security measures for Agentic AI, and 5 security threats and 10 security measures for Physical AI.

1. Why Agentic and Physical AI Security Matters

AI has evolved from predictive AI to generative AI and is now moving toward systems that can autonomously plan, coordinate, and act.

The AI Security Guidebook reflects this development by providing a dedicated chapter on Agentic AI and Physical AI. It also notes that these technologies are evolving rapidly and that security threats and required controls may change as new systems are deployed.

NIST similarly notes that AI agents can perceive environments and use software tools to perform actions beyond simply producing text, creating new security and reliability considerations.

OWASP has also identified reasoning, memory, tools, identity, human oversight, and multi-agent interaction as important attack surfaces for agentic systems.

2. What Is Agentic AI?

A conventional generative AI interaction can be summarized as:

User → Question → AI → Answer

An agentic system can instead operate like this:

User Request → Goal Analysis → Planning → Information Retrieval → Tool Invocation → Result Evaluation → Additional Actions → Final Result

The AI therefore becomes an execution component, not merely a conversational interface.

The AI Security Guidebook describes Agentic AI as a system in which multiple AI agents can interact and autonomously perform planning, information retrieval, and tool invocation to achieve a specific goal.

Typical Components

An agentic architecture may include:

• AI models

• Planning components

• Short-term and long-term memory

• External tools

• APIs

• Databases

• MCP servers

• Agent-to-agent communication

• Multiple AI agents

• Human approval and control mechanisms

The guidebook describes MCP as a protocol that standardizes interactions between AI agents and external tools, data, and APIs.

3. A Public-Sector Example

The AI Security Guidebook presents an example of an agentic AI-based one-stop administrative service.

Suppose an employee asks:

"Please prepare my business trip to Busan from October 27 to October 31."

An agentic system could:

1. Analyze the request

2. Review previous information

3. Create a travel plan

4. Check the calendar

5. Search transportation

6. Access an administrative portal

7. Retrieve relevant documents

8. Combine the results

9. Present the final response

The important point is that the AI is not necessarily performing every function itself. Instead, multiple agents and external tools may work together to achieve one goal.

The guidebook recommends controlling the tools that an agent can call through a whitelist, applying least privilege, and monitoring abnormal behavior.

4. Major Agentic AI Security Threats

The AI Security Guidebook identifies 11 major threats.

The central difference from conventional generative AI is that an agent can act.

If a chatbot produces an incorrect answer, the immediate impact may be misinformation.

If an agent can modify databases, send emails, delete files, make transactions, or change administrative records, the same type of incorrect reasoning can result in an actual operational incident.

5. The Core Principle: Least Privilege + Approval + Control

The AI Security Guidebook emphasizes that Agentic AI should be given only the permissions necessary to accomplish its objectives.

Control the Available Tools

An agent should not automatically have access to every available API or external service.

For example:

Allowed
├─ Administrative portal
├─ Internal document search
├─ Calendar service
└─ Approved public APIs

Blocked
├─ Unregistered external APIs
├─ Personal email
├─ Arbitrary file uploads
└─ Unverified MCP servers

This corresponds to A-M04, whitelist-based tool use, in the AI Security Guidebook.

OWASP also identifies tool misuse and excessive privileges as significant agent security concerns.

6. Why AI Should Not Have Administrator-Level Access

Consider an AI agent with the following permissions:

AI Agent
 ├─ Read database
 ├─ Modify database
 ├─ Delete files
 ├─ Send email
 ├─ Change system settings
 └─ Call external APIs

A compromised or incorrectly behaving agent could cause a much broader incident than an ordinary chatbot.

Instead, permissions should be separated by business function.

Travel Agent
 ├─ Read travel information       YES
 ├─ Register calendar event       YES
 ├─ Search transportation        YES
 ├─ Modify HR information        NO
 ├─ Change system settings       NO
 └─ Use administrator account    NO

The AI Security Guidebook identifies this as A-M11, restriction of excessive privileges, and recommends A-M12, approval procedures for sensitive commands.

7. Should Every AI Action Require Human Approval?

Not necessarily.

If an employee has to approve every routine operation, the system may create excessive approval requests.

For example:

AI → Approval request
Human → Approve

AI → Approval request
Human → Approve

AI → Approval request
Human → Approve
...

This can lead to approval fatigue.

The AI Security Guidebook identifies excessive human intervention as a separate threat and recommends thresholds for sensitive-command approval through A-M13.

A practical policy might therefore look like this:

8. AI Memory Is Also a Security Asset

One of the important characteristics of agentic AI is memory.

The AI Security Guidebook distinguishes between short-term and long-term memory.

Short-Term Memory

This can include current-session conversations and temporary working information.

Long-Term Memory

This can contain accumulated user interactions and external knowledge, including information stored in RAG vector databases.

If an attacker inserts malicious information into memory, the AI may use that information during future operations.

A simplified attack path is:

Legitimate information
        ↓
AI Memory
        ↓
Malicious information inserted
        ↓
AI treats it as trusted context
        ↓
Incorrect future action

The guidebook therefore recommends A-M02, memory inspection, to periodically examine short- and long-term memory for abnormal or malicious information.

OWASP also treats memory poisoning as a distinct risk for AI agents.

9. Protect Communication Between AI Agents

A multi-agent system may look like this:

User
  ↓
Coordinator Agent
  ├── Document Agent
  ├── Scheduling Agent
  ├── Budget Agent
  └── Reporting Agent

If one agent is compromised, malicious information may propagate to other agents.

The AI Security Guidebook identifies this as A10, attacks on communication between AI agents, and recommends:

• A-M08 blocking malicious propagation

• A-M15 agent identity verification

• A-M16 protecting agent communication channels

In other words, AI agents should be treated as identifiable and controllable participants in a system rather than as anonymous software components.

10. MCP Servers and External Tools Must Be Secured

Agentic AI becomes powerful partly because it can use external tools.

But that also expands the attack surface:

AI
 ↓
MCP Server
 ↓
External Tool
 ↓
Email / Database / API / File / Business System

If an MCP server or external tool contains a vulnerability or malicious modification, an otherwise legitimate AI request may become an attack path.

The AI Security Guidebook categorizes this as A11, malicious behavior caused by vulnerable components, and recommends periodic vulnerability assessment and security updates.

Public institutions should therefore treat MCP servers as security-sensitive AI infrastructure, not merely as integration software.

11. What Is Physical AI?

Physical AI refers to AI systems that interact with the physical world through devices such as:

• Robots

• Drones

• Autonomous vehicles

• Robotic arms

• Smart manufacturing equipment

• Facility-inspection robots

• Medical devices

• Logistics robots

The AI Security Guidebook describes the basic process as:

Sensors → Preprocessing → Perception → Decision → Actuator → Physical Action

For example:

Camera / LiDAR / Microphone
             ↓
       Sensor Data
             ↓
        Preprocessing
             ↓
        AI Perception
             ↓
        Decision Making
             ↓
       Action Planning
             ↓
     Motors / Actuators
             ↓
       Physical Action

The key difference is that a security failure can potentially affect people, equipment, and facilities.

NIST's robotics-related work likewise emphasizes the need to evaluate AI-enabled robotic systems in realistic environments, considering both machine-learning and robotics characteristics.

12. Five Major Physical AI Security Threats

The AI Security Guidebook identifies five major Physical AI threats.

P01. AI Backdoors, Malfunction, and Incorrect Actions

A hidden condition could cause the AI to behave incorrectly.

For example, a patrol robot could be induced to move into an unauthorized area.

P02. Excessive Computation and Resource Exhaustion

An attacker could induce computationally expensive tasks, increasing battery consumption, heat, or control latency.

P03. Evasion Attacks

A malicious physical pattern could cause cameras or other sensors to misclassify an object or environment.

P04. Attacks on Sensor Preprocessing

Manipulated data could be inserted before sensor information reaches the AI model.

P05. Physical Safety Threats

An external attack could cause unpredictable movement or loss of control, potentially affecting people or infrastructure.

13. Safety Mode Is Critical for Physical AI

For Physical AI, cybersecurity and safety are closely connected.

Even if an AI makes an incorrect decision, the system should prevent that decision from becoming a dangerous physical action.

The guidebook's P-M05 Safety Mode Operation recommends automatically stopping or transitioning the system to a safe mode when predefined behavioral limits or safety thresholds such as heat or power consumption are exceeded.

A possible design is:

Normal Operation
      ↓
Abnormal Behavior Detected
      ↓
Speed / Power Limitation
      ↓
Stop
      ↓
Safety Mode
      ↓
Human Verification
      ↓
Recovery or Manual Control

This becomes especially important when Physical AI is deployed around critical facilities.

14. Sensors Must Also Be Protected

Protecting the AI model alone is not enough.

If a camera, LiDAR sensor, or other sensor provides incorrect information, a perfectly functioning AI model can still make the wrong decision.

The guidebook recommends P-M07, setting sensor input ranges, to prevent abnormal sensor values from causing malfunction.

It also recommends P-M08, protecting Physical AI communication channels, including communication between sensors and AI and between AI and actuators.

15. The Physical Device Itself Can Be Attacked

A Physical AI system deployed outdoors or in facilities may be exposed to direct physical attacks.

Examples include:

• USB-port attacks

• Debug-port access

• Storage-device theft

• Sensor cable manipulation

• Device tampering

• Communication equipment manipulation

The guidebook recommends P-M06, strengthening hardware security, including physically protecting exposed communication and USB ports.

16. Practical Questions for Public-Sector IT Officials

When planning an Agentic or Physical AI project, the following questions can be used during the initial security review.

1. Does the AI only provide answers?

Yes → Conventional AI security review
No  → Agentic AI security review

2. Can the AI call external tools?

Ask:

• What tools?

• Who provides them?

• What permissions do they have?

• Are they whitelisted?

• Have they been vulnerability-tested?

3. Can the AI modify data?

Consider:

• Database changes

• File deletion

• Document modification

• System configuration changes

If yes, apply least privilege and approval controls.

4. Can the AI access sensitive information?

Review:

• Data classification

• Access control

• Storage location

• Transmission paths

• Logging

• External transmission

5. Does the AI communicate with other agents?

If yes, consider:

• Agent identity

• Authentication

• Authorization

• Communication protection

• Propagation controls

6. Does the AI control physical devices?

If yes, additionally review:

• Allowed actions

• Least privilege

• Sensor security

• Communication security

• Safety mode

• Emergency stop

• Physical security

• Operational monitoring

17. A Conceptual Agentic AI Security Architecture

A public-sector deployment could conceptually use the following structure:

                 [User]
                   │
                   ▼
          [AI Agent Gateway]
                   │
        ┌──────────┴──────────┐
        │                     │
 [Policy / Authorization] [Monitoring]
        │                     │
        └──────────┬──────────┘
                   ▼
              [AI Agent]
                   │
        ┌──────────┼──────────┐
        ▼          ▼          ▼
     [Memory]    [MCP]   [Other Agent]
                   │
          ┌────────┼────────┐
          ▼        ▼        ▼
       Business  Calendar  Approval
          DB       API       API

The key idea is to protect not only the AI model but also:

memory + tools + APIs + agents + permissions + communications

18. A Conceptual Physical AI Security Architecture

A Physical AI system can use multiple defensive layers:

              [Physical Environment]
                       ↓
              Camera / LiDAR / Sensors
                       ↓
                [Sensor Validation]
                       ↓
                    [AI Model]
                       ↓
                [Action Validation]
                       ↓
             [Policy / Authorization]
                       ↓
               Is Action Allowed?
                    /       \
                  Yes        No
                   ↓          ↓
             [Actuator]   [Safety Mode]
                              ↓
                         [Emergency Stop]

The objective is to avoid sending an AI decision directly to an actuator without policy and safety validation.

19. Turn Security Controls into Project Requirements

Security controls should not remain only in an explanatory document.

They can be translated into actual requirements for an RFP, security specification, or system design.

Agentic AI Requirements

• Only approved tools shall be callable by AI agents.

• Least privilege shall be applied to AI agents.

• Human approval shall be required for designated sensitive commands.

• Agent behavior and tool invocation shall be logged.

• AI memory shall be subject to integrity and anomaly checks.

• Agent-to-agent communications shall use appropriate authentication and protection.

• MCP servers and external components shall undergo periodic vulnerability assessments.

• AI agents shall support automatic shutdown for predefined prohibited or anomalous behavior.

Physical AI Requirements

• Permitted physical actions shall be defined in advance.

• Only minimum required control privileges shall be granted.

• Safety thresholds shall be defined.

• The system shall transition to a safety mode when abnormal conditions occur.

• Authorized personnel shall have an emergency-stop capability.

• Sensor input ranges shall be defined.

• Sensor-AI and AI-actuator communications shall be protected.

• Exposed physical interfaces shall be secured.

• AI behavior, power consumption, temperature, and other operational indicators shall be monitored.

These requirements are practical interpretations of the A-M01~A-M17 and P-M01~P-M10 measures in the AI Security Guidebook.

20. Agentic AI vs. Physical AI

21. Security Must Expand Beyond the AI Model

The progression can be summarized as follows:

Conventional AI
      ↓
Data Protection
Model Protection
Input / Output Protection
Logging
Authorization
      ↓
Agentic AI
      +
Memory Protection
Tool Control
Agent Identity
Agent Communication
Sensitive-Command Approval
Automatic Shutdown
      ↓
Physical AI
      +
Sensor Protection
Actuator Protection
Safety Mode
Emergency Stop
Physical Security

As AI becomes more autonomous, the security boundary expands beyond the model itself.

22. International Guidance and the Public-Sector Approach

The approach in the AI Security Guidebook is consistent with broader international AI-security discussions.

NIST's AI Risk Management Framework provides a framework for managing AI risks across the AI lifecycle, while the Generative AI Profile provides additional guidance for risks specific to generative AI.

OWASP has developed threat models and mitigation guidance specifically for Agentic AI, including practical guidance for designing, developing, and deploying secure agentic applications.

A public institution can therefore consider a layered approach:

Korean Public-Sector Requirements
             +
AI Security Guidebook
             +
NIST AI RMF
             +
OWASP Agentic AI Guidance
             +
Institutional Security Policies
             ↓
AI Project Security Requirements

However, actual Korean public-sector projects should also follow applicable national information-security requirements and security-review procedures. The AI Security Guidebook itself explains that it focuses on AI models, data, and infrastructure, while other cybersecurity requirements should follow relevant national security guidelines.

23. Ten Things Public-Sector IT Officials Should Remember

1. Do not give AI unnecessary privileges.

Apply least privilege.

2. Limit the tools AI can use.

Use verified and approved tools.

3. Require human approval for sensitive actions.

Especially for sensitive data transfers, modifications, payments, and control operations.

4. Protect AI memory.

Consider both short-term and long-term memory.

5. Log AI behavior.

Monitoring should cover tool calls and privilege use, not only prompts and responses.

6. Continuously assess AI components.

Include models, libraries, MCP servers, APIs, and other supply-chain components.

7. Protect agent-to-agent communications.

Use identity verification, authorization, and communication protection.

8. Build safety modes into Physical AI.

Prevent AI errors from becoming physical incidents.

9. Provide emergency-stop capabilities.

Human control remains important even in highly automated systems.

10. Build security into the design.

Security should be addressed during planning and architecture, not added only after deployment.

24. Conclusion

Agentic AI and Physical AI can significantly expand the automation capabilities of public services.

However, unlike conventional generative AI, these systems can:

remember → plan → communicate → call tools → make decisions → take actions

and Physical AI can additionally:

sense → decide → move → interact with the physical world.

Therefore, public-sector IT officials should ask not only:

"What can this AI do?"

but also:

"What should this AI be allowed to do?"

"What happens if the AI makes a wrong decision?"

"When can a human intervene?"

"How can the system be stopped immediately?"

The central message of Chapter 3 of the AI Security Guidebook can be summarized as follows:

For Agentic AI, allow autonomy while enforcing least privilege, tool control, human approval, and continuous monitoring. For Physical AI, combine these principles with safety modes, emergency response, sensor protection, actuator controls, and physical security.

Because these technologies are evolving rapidly, public institutions should continuously reassess emerging threats and update security requirements as systems, tools, protocols, and deployment environments change.

Keywords:
Agentic AI, Physical AI, AI Security, Public Sector AI Security, AI Agent Security