# AI Gateway and AI Security Architecture: Prompt Firewall, DLP and Agent Control

**URL:** https://securesys.com.tr/en/learning/software-and-ai-security/what-is-an-ai-gateway-enterprise-ai-security-architecture

![AI Gateway and AI Security Architecture: Prompt Firewall, DLP and Agent Control](/images/bilgi-merkezi/covers/cover-yzguv-11.webp)

### Introduction: How Can Enterprise AI Use Be Brought Under Control?

AI technologies are no longer tools used only by software development teams.

Sales teams use AI to prepare customer correspondence.

Human resources departments use it to analyse CVs.

Finance teams have it interpret their reports.

Legal departments use it to review contracts.

Software developers send their source code to AI assistants.

Cyber security teams draw on large language models in incident analysis.

These use cases can deliver significant productivity gains.

However, they also raise new security questions for organisations.

Which AI services are employees using?

Which data is being transferred to external systems?

Are personal data and trade secrets being protected?

Which corporate systems can AI applications access?

Which actions can AI Agents perform on behalf of users?

Are there approval mechanisms for critical actions?

Can suspicious AI usage be monitored by security operations?

Answering these questions is not possible simply by sending employees an AI usage policy.

Policies need to be backed by technical controls.

This is where the **AI Gateway** stands out as one of the key components of an enterprise AI security architecture.

An AI Gateway is a control layer that helps manage the communication between AI applications, users, model providers and, in some cases, tool integrations through centralised policies.

However, there is an important distinction:

**Using an AI Gateway does not mean that all AI security risks disappear automatically.**

For an effective security architecture, the AI Gateway must work together with identity management, data protection, application authorisation, monitoring and incident response processes.

In this article, we examine the enterprise AI Gateway architecture, its security components and implementation methods in detail.

### \1. What Is an AI Gateway?

An AI Gateway is a central software or security layer that manages the communication between AI applications and model services and enforces defined policies.

Depending on its intended use, it can offer different functions.

For example:

- Model access management.
- User- and application-based policy enforcement.
- Prompt and response inspection.
- Sensitive data detection.
- Data masking.
- Model routing.
- Usage quotas.
- Transaction logs.
- Security monitoring.
- Tool call control.

Not every AI Gateway product offers all of these capabilities.

Some only route model API traffic.

Some focus on cost and performance management.

Others offer security policies and data protection features.

For this reason, when selecting an enterprise AI Gateway, the product's real capabilities should be verified technically.

### \2. Why Is an AI Gateway Necessary?

A traditional enterprise security architecture contains a number of different control points.

A firewall controls network traffic.

A WAF helps protect web applications.

DLP aims to reduce the unauthorised transfer of sensitive data.

IAM manages user and service identities.

A SIEM supports the monitoring of security events.

AI systems, in turn, create new data and transaction flows that touch all of these control points.

For example, an employee may send a customer contract to an external LLM service.

A software developer may pass proprietary source code to an AI coding assistant.

An AI Agent may query customer records through the corporate CRM.

Each of these actions requires different security controls.

With the right integrations, an AI Gateway can help manage certain parts of AI communication centrally.

However, visibility depends on the traffic actually passing through the control layer.

The traffic of applications that connect directly to the internet or do not use the Gateway cannot be inspected automatically.

### \3. The Difference Between a Traditional API Gateway and an AI Gateway

An API Gateway is used to manage API communication between applications.

An AI Gateway, on the other hand, can provide additional controls specific to AI usage.

| Feature | API Gateway | AI Gateway |
| --- | --- | --- |
| API routing | Core function | Usually available |
| Authentication | Can be supported | Can be supported |
| Rate limiting | Can be supported | Can be supported |
| Model selection and routing | May require custom development | Common use case |
| Prompt analysis | Usually not built in | May be found in security-focused products |
| Sensitive data masking | May require additional integration | Available in some products |
| LLM response analysis | Usually not built in | Available in some products |
| Token and model cost monitoring | May require custom integration | Common use case |
| Agent tool policies | May require custom integration | May be found in advanced solutions |

This comparison describes general product categories.

Actual capabilities may vary depending on the solution used.

### \4. Enterprise AI Gateway Architecture

In a typical enterprise AI Gateway architecture, users and applications can communicate through a central control layer instead of accessing the model provider directly.

#### Example architecture

ENTERPRISE USERS

|

v

Applications / AI Agents

|

v

AI GATEWAY

|

+---------------+---------------+

| | |

v v v

Identity and Prompt / Data Policy and

Access Control Security Audit

| | |

+---------------+---------------+

|

v

Model Routing

|

+------------+------------+

| | |

v v v

Cloud LLM Private Model On-Prem LLM

| | |

+------------+------------+

|

v

Response Security Check

|

v

Application

This architecture is a reference example.

Additional control points may be required for Agent tool calls, MCP integrations and corporate data access.

### \5. Trust Boundaries in an AI Gateway Architecture

One of the fundamental topics in AI security architecture is defining trust boundaries correctly.

A request coming from a user and the answer produced by a model do not have the same level of trust.

Similarly, text inside a PDF should not carry the same authority as a policy defined by a system administrator.

#### Core trust boundaries

User and application.

Application and AI Gateway.

AI Gateway and model provider.

Model and RAG data source.

Agent and tools.

Tools and corporate systems.

Different security controls should be applied at each of these boundaries.

For example, an AI Gateway can detect content that is suspected of containing Prompt Injection.

However, whether an Agent is authorised to modify a CRM record must be checked separately in the trusted application or tool layer.

### \6. What Is a Prompt Firewall?

A Prompt Firewall is a control mechanism used to detect, classify or block specific security risks in the content sent to AI systems or processed by them.

These controls can be implemented using different methods.

Rule-based analysis.

Content classification.

Use of a security model.

Contextual policy evaluation.

Anomaly analysis.

#### Risks that may be targeted for detection

Prompt Injection.

Jailbreak attempts.

Inputs containing sensitive data.

Unauthorised action requests.

Out-of-policy use.

However, a Prompt Firewall should not be expected to block every manipulation attempt with certainty.

Because of the flexible nature of natural language and the variety of attack variations, false positives and false negatives can occur.

### \7. How Does a Prompt Firewall Work?

A Prompt Firewall can apply controls at different stages.

#### Input control

The content submitted by the user is evaluated.

#### Context control

Content coming from RAG or external tools can be inspected.

#### Policy evaluation

The suitability of the request is assessed in terms of the user, the application and the data classification.

#### Decision

The operation can be allowed.

The content can be masked.

The request can be blocked.

Additional approval can be requested.

The event can be recorded for incident investigation.

#### Output control

The model's answer can be evaluated for sensitive data or policy violations.

The stages at which these controls can be applied depend on the product's architecture.

### \8. Layered Security in Prompt Injection Protection

Using a Prompt Firewall alone may not be sufficient against Prompt Injection attacks.

This is because a model may interpret untrusted content in unexpected ways.

For this reason, controls should be applied at different layers.

#### Application layer

User identity must be verified.

Transaction permissions must be checked.

#### Data layer

Access to RAG documents must be restricted.

Untrusted content must be labelled appropriately.

#### Model communication layer

Prompt security controls can be applied.

#### Tool layer

Tool Calling permissions must be verified independently.

#### Operations layer

Suspicious events must be monitored.

This approach helps prevent the defence from depending on a single model or a single filter.

### \9. What Is AI DLP?

AI DLP refers to data loss prevention controls aimed at reducing the unauthorised or inappropriate sharing of sensitive data in AI usage.

Traditional DLP systems can inspect file, email, network or endpoint traffic.

AI DLP, on the other hand, can focus on the content sent to AI applications and, in some cases, on model responses.

#### Data types that need protection

Personal data.

Customer records.

Financial information.

Trade secrets.

Source code.

Contracts.

Credentials.

API keys.

Corporate strategy documents.

#### Example

An employee may want to send a document containing a customer list to an external AI service.

An AI DLP control can evaluate this transfer against the defined data classification policy.

### \10. The Difference Between AI DLP and Traditional DLP

AI DLP does not have to replace traditional data protection controls.

For most organisations, it should be regarded as a complementary layer.

For example, traditional DLP can control the uploading of a file to an external web service.

AI DLP, on the other hand, can evaluate the sensitive content within a specific LLM request.

However, for AI DLP to be effective, visibility must be ensured over the AI channels in use.

Uncontrolled applications and direct API connections can create visibility gaps.

For this reason, in an enterprise data security architecture, endpoint, network, application and AI Gateway controls should be considered together.

### \11. Data Masking in AI

Data masking aims to hide or transform sensitive information in specific usage scenarios.

For example, in a customer support application, a user's name or ID number can be masked before being sent to the model service.

#### Example

**Original content:**

“Summarise the request relating to customer Ayşe Yılmaz's identity record number 12345678901.”

**Masked content:**

“Summarise the request relating to customer [PERSON_1]'s record [ID_NO_1].”

This approach can help the model perform some tasks without seeing real personal data.

However, the suitability of masking depends on the task.

For example, some analyses may need real data.

For this reason, data minimisation and masking policies should be designed according to the business process.

### \12. Tokenization and Pseudonymization

In an AI Gateway architecture, transformation methods other than data masking can be used.

#### Tokenization

A controlled token can be used in place of sensitive data.

The mapping between the real data and the token can be held in a secure system.

#### Pseudonymization

Information that directly identifies a person can be converted into pseudonymous identifiers using appropriate methods.

However, pseudonymised data can retain the character of personal data if the person concerned can be re-identified.

#### Security requirements

Mapping records must be protected.

Key management must be secure.

Unauthorised re-identification must be prevented.

Sensitive data must not be kept in logs.

These controls support the data protection dimension of the AI DLP architecture.

### \13. What Is Shadow AI?

Shadow AI refers to AI tools and services used outside the organisation's knowledge, approval or management.

For example, employees may use free AI services to speed up their work.

This usage is not always malicious.

However, from a corporate security perspective, it can create a visibility and control problem.

#### Shadow AI risks

Transfer of sensitive data.

Uncontrolled SaaS usage.

Access outside corporate accounts.

Absence of audit logs.

Unknown data retention terms.

Non-compliance with corporate policies.

#### An important distinction

Detecting Shadow AI usage and controlling that usage are different capabilities.

An AI Gateway can directly inspect only the traffic that passes through it.

Different visibility mechanisms may be needed for usage outside the Gateway.

### \14. How Is Shadow AI Managed?

Shadow AI management should not rely solely on a prohibition approach.

It is necessary to understand why employees use particular tools.

Secure alternatives suited to corporate needs should be offered.

#### Management approach

Approved AI services must be identified.

Usage policies must be established.

Data classification must be applied.

Corporate identity management must be used.

Endpoint and network visibility must be assessed.

Appropriate AI Gateway integrations must be put in place.

Awareness training must be given to employees.

This approach can reduce uncontrolled usage while helping preserve the productivity gains obtained from AI.

### \15. What Is AI Access Control?

AI Access Control is an authorisation approach that determines which AI services, models and operations users and applications can access.

For example, not all employees may need to access the same model services.

Some departments may use only approved models.

Some applications may process certain data classes.

Some Agent systems may have read-only permissions.

#### Control areas

User identity.

Department.

Application.

Model provider.

Data classification.

Operation type.

Tool permission.

More granular access policies can be built using this information.

### \16. AI Authorisation with RBAC and ABAC

RBAC stands for the Role-Based Access Control approach.

Permissions are defined according to user roles.

ABAC, on the other hand, is the Attribute-Based Access Control approach.

Attributes of the user, resource, operation and environment can be used in authorisation decisions.

#### Example RBAC policy

Human resources employees can access only the personnel data they are authorised for.

The sales team can view only the relevant customer records.

The finance team can query specific reports.

#### Example ABAC policy

The user's department, the data classification and the operation type are evaluated together.

A request containing sensitive data can be processed only under certain conditions.

#### Critical principle

The authorisation decision must not be left solely to the LLM's interpretation.

It must be enforced in trusted application and data access layers.

### \17. The Role of the Gateway in AI Agent Security

AI Agent systems can interact with a variety of tools.

These tools can read corporate data or make changes to systems.

In a suitable architecture, an AI Gateway can inspect certain parts of Agent communication.

#### Control areas

Model access.

Prompt security.

Sensitive data transfer.

Tool call policies.

Transaction logs.

Approval flows.

However, Agent security should not be ensured by the Gateway alone.

The tools themselves must also apply secure authentication and authorisation.

### \18. Tool Calling Security

Tool Calling allows an AI model to carry out operations by using defined tools.

For example:

Querying the CRM.

Sending email.

Creating calendar entries.

Reading files.

Preparing reports.

Opening support tickets.

#### The security problem

The fact that a model can generate a tool call does not mean that it is authorised to perform that operation.

#### Secure approach

Tools must be managed with an allow list.

Parameters must be validated.

User permissions must be checked.

Critical operations must require approval.

Transaction logs must be kept.

These controls must work in coordination between the AI Gateway and the application layer.

### \19. MCP Security and the AI Gateway

MCP is the abbreviation of Model Context Protocol.

It offers a standardised communication approach for connecting AI applications to tools and data sources.

MCP servers can connect to different corporate systems.

For this reason, a security assessment is important.

#### Risks

Untrusted MCP servers.

Excessive tool permissions.

Access to sensitive data.

Manipulated tool descriptions.

Untrusted tool outputs.

Third-party integration risks.

#### Security approach

MCP servers must be inventoried.

Trusted sources must be used.

Permissions must be restricted.

Tool calls must be controlled.

Security logs must be kept.

If its architecture supports it, an AI Gateway can be used as an additional policy point for MCP or tool traffic.

However, using MCP does not by itself guarantee secure authorisation.

### \20. What Is Human-in-the-Loop?

Human-in-the-Loop refers to supporting certain AI decisions or operations with human review.

It is particularly important for high-impact operations.

#### Example operations

Payment approval.

Deleting a customer record.

Changing a policy on a critical security device.

Deploying software to the production environment.

Updating a personnel record.

#### Security approach

The Agent proposes the operation.

The trusted system displays the details of the operation.

An authorised user reviews the operation.

The operation is carried out after approval.

For this approach to be effective, the approval must be enforced through an independent and trustworthy control mechanism.

An “approved” statement generated by the model itself must not be considered sufficient.

### \21. Approval Mechanism for Critical Operations

Let us consider an AI Agent connected to a finance system.

The Agent can analyse payment records.

However, the authority to create or approve payments is high-risk.

#### Secure transaction flow

- The Agent creates an operation proposal.
- The application validates the operation parameters.
- An authorisation check is performed.
- If required, independent approval is requested.
- If the approval is valid, the operation is executed.
- The operation is recorded.

#### Key point

The approval mechanism must clearly show which operation the user is approving.

For example, if the payment amount or recipient changes, the previous approval must not automatically be considered valid.

This principle helps preserve transaction integrity.

### \22. AI Gateway and Audit Logging

In AI systems, audit records are important for security and governance.

#### Information that can be recorded

User or application identity.

Model provider.

Model name and version.

Time of the operation.

Policy decision.

Tool call type.

Approval status.

Security event.

Resource consumption.

#### Data minimisation

Storing all prompt and response content indefinitely may not be the right approach.

This is because such content may contain sensitive data.

Logging policies should be designed taking into account data classification, retention period and access permissions.

### \23. AI Gateway and SIEM Integration

Security events generated by the AI Gateway can be correlated with SIEM systems.

This approach supports the monitoring of AI usage by corporate security operations.

#### Example events

Repeated suspected Prompt Injection.

Attempted transfer of sensitive data.

Unauthorised model access.

Abnormal API usage.

Critical tool call.

Policy violation.

#### Benefits for the SOC

Event correlation.

User behaviour analysis.

Security alert generation.

Incident investigation.

Audit traceability.

However, for security events to be meaningful, false positive rates and alert quality must be evaluated regularly.

### \24. AI Gateway and SOAR Integration

SOAR systems can support the automation of certain response processes for security events.

AI Gateway events can be passed to SOAR processes under suitable conditions.

#### Example

Consider a case in which a large number of sensitive data transfer attempts are detected from a single user account within a short period.

The event can be correlated in the SIEM.

An investigation process can be initiated through SOAR.

If necessary, the user's access can be reviewed by the security team.

However, automated response policies should be designed taking business continuity impacts into account.

It may not be appropriate for every security alert to trigger high-impact actions such as automatically disabling an account.

### \25. What Is an On-Premises AI Gateway?

An on-premises AI Gateway means running the security control layer on the organisation's own infrastructure.

This approach can be considered particularly by organisations with data control and infrastructure management requirements.

#### Potential advantages

Operating the control layer inside the organisation.

Ability to apply private network policies.

Integration with corporate log management.

Connection to private model services.

Ability to design the data processing architecture to the organisation's needs.

#### An important limitation

Running an AI Gateway on-premises does not mean that data sent to the model provider never leaves the organisation.

If an external LLM service is used, data transfer may continue.

Data residency and transfer requirements must be assessed end to end.

### \26. Comparison of Cloud AI Gateway and On-Premises AI Gateway

| Criterion | Cloud AI Gateway | On-Premises AI Gateway |
| --- | --- | --- |
| Operating model | Can be managed by the provider | Managed by the organisation |
| Initial setup | Can generally be faster | Infrastructure preparation may be required |
| Control over data processing | Depends on the provider's architecture | Can be designed according to the organisation's architecture |
| Update management | May be the provider's responsibility | May be the organisation's responsibility |
| Enterprise integration | Varies by product | Custom integrations can be built |
| Operational responsibility | May be shared | Higher organisational responsibility |

Which approach is appropriate depends on the organisation's security, cost, operational and regulatory requirements.

### \27. Hybrid AI Gateway Architecture

Some organisations may use both cloud-based and in-house AI models.

In this case, a hybrid architecture may be preferred.

#### Example

An external model can be used for general text generation.

An in-house model can be preferred for sensitive document analysis.

The AI Gateway can route requests to the appropriate model according to defined policies.

#### Policy example

General data: Approved cloud model.

Internal data: Authorised private model.

Highly sensitive data: In-house processing or blocking.

For these policies to be enforceable, data classification must be performed reliably.

### \28. Model Routing and Security

Model Routing is the directing of AI requests to different models.

This routing can be done according to performance, cost or security policies.

#### Security considerations

The model provider's data processing terms.

The use cases supported by the model.

Data classification.

User permissions.

Geographic data processing requirements.

#### Example

A request containing sensitive customer data can be routed only to an approved processing environment.

However, model routing alone does not ensure data security.

The access and data protection controls of the target environment must also be verified.

### \29. AI Gateway and Cost Control

In AI systems, security and cost management are interrelated.

Especially with API-based models, heavy usage can generate costs.

#### Risks

Uncontrolled API usage.

Unauthorised account use.

Excessive token consumption.

Agent transaction loops.

Abuse.

#### Controls

User-based quotas.

Application-based budgets.

Rate limiting.

Token usage monitoring.

Anomaly detection.

Operation timeouts.

These controls support both operational sustainability and resilience against resource abuse.

### \30. AI Gateway High Availability

When an AI Gateway is used as a central control point, availability can become critical.

If the Gateway goes out of service, it can affect the operation of AI applications.

#### Areas to assess

High Availability.

Load Balancing.

Redundant infrastructure.

Disaster Recovery.

Capacity Planning.

Monitoring.

#### Fail-Open and Fail-Closed

How the system will behave when security controls are unavailable must be determined in advance.

In a Fail-Open approach, certain operations can continue without controls.

In a Fail-Closed approach, the operation can be blocked.

This decision should be made according to data sensitivity and business continuity requirements.

For example, for a high-sensitivity data transfer, the controls being disabled may be unacceptable.

### \31. AI Gateway Security Testing

Having an AI Gateway installed is not enough.

The effectiveness of the controls must be verified.

#### Test areas

Prompt Injection detection.

Jailbreak control.

Sensitive data masking.

Unauthorised model access.

Tool Calling policies.

Approval mechanisms.

Audit logging.

Rate limiting.

#### Testing approach

Controlled test inputs are prepared.

Expected policy decisions are defined.

The Gateway's behaviour is examined.

False positive and false negative results are evaluated.

The real security impact on the application is measured.

This work can be carried out together with AI Red Teaming and LLM Pentest processes.

### \32. Example Scenario: Software Development Team

Consider an organisation whose software developers use AI coding assistants.

Developers may from time to time send pieces of source code to external models.

#### Risks

Transfer of proprietary source code.

API key leakage.

Confidential configuration information.

Uncontrolled model usage.

#### Security approach

Approved model services must be identified.

Source code must be classified.

The transfer of sensitive information must be controlled.

Secret detection must be applied.

Appropriate Gateway integrations must be assessed.

#### Key point

If not all code assistant traffic passes through the Gateway, additional endpoint or application controls may be needed.

### \33. Example Scenario: Finance Department

An organisation's finance team uses an AI-assisted report analysis system.

The system interprets financial statements and produces summaries.

#### Risks

Transfer of sensitive financial data to an external model.

Incorrect data classification.

Unauthorised user access.

Erroneous evaluation of model outputs.

#### Security approach

Financial data must be classified.

Model access must be restricted.

Sensitive data policies must be applied.

Critical financial decisions must be verified through independent controls.

In a suitable architecture, an AI Gateway can support data transfer controls.

### \34. Example Scenario: Autonomous AI Agent

An organisation's AI agent is connected to the CRM, email and document management systems.

The Agent can analyse customer requests and carry out certain operations.

#### Risks

Excessive authorisation.

Manipulated email content.

Unauthorised changes to customer records.

Transfer of sensitive data.

Critical operations performed without approval.

#### Security approach

Agent tools must be inventoried.

Permissions must be restricted.

External content must be treated as untrusted data.

Critical operations must require approval.

Transaction logs must be kept.

With appropriate integrations, an AI Gateway can provide an additional layer of control and visibility.

### \35. Bastiyon AI Gateway: The SECURESYS Approach

In SECURESYS's approach to AI security, **Bastiyon AI Gateway** is positioned as a solution architecture for managing enterprise AI usage through centralised policies.

The main security areas that can be assessed according to enterprise needs are as follows:

#### Prompt Security

Control mechanisms for Prompt Injection and Jailbreak risks.

#### AI DLP

Sensitive data detection, data classification and appropriate masking policies.

#### Shadow AI Governance

Assessing uncontrolled AI usage in terms of visibility and policy needs.

#### Agent and Tool Calling Governance

Controlling AI Agent tool calls and operation permissions.

#### Approval Workflow

Applying approval mechanisms for critical operations.

#### Audit and Monitoring

Monitoring security events and transaction logs relating to AI usage.

#### On-Premises Deployment

Evaluating architecture options that can be run on corporate infrastructure.

The actual implementation scope of these areas must be verified technically according to the product version, integrations and project needs.

In the Bastiyon AI Gateway approach, the primary goal is not to restrict AI usage altogether.

It is to strengthen data security, operation authorisation and auditability while organisations retain the benefits they gain from AI.

**AI security is not only about controlling what the model says; it is also about managing which data it accesses, which system it uses and which operation it performs.**

### \36. Enterprise AI Gateway Selection Criteria

When selecting an AI Gateway solution, the product's feature list alone should not be assessed.

Compatibility with the actual enterprise architecture should also be examined.

#### Technical criteria

Supported model providers.

On-premises model support.

API compatibility.

RAG integrations.

Agent and Tool Calling control.

MCP integrations.

Identity management.

SIEM integration.

#### Security criteria

Prompt Injection controls.

Sensitive data detection.

Masking.

Access policies.

Audit logs.

Data retention policies.

#### Operational criteria

High availability.

Performance.

Scalability.

Ease of management.

Update processes.

Support model.

#### Commercial criteria

Licensing model.

Usage-based costs.

Integration costs.

Operating costs.

These criteria should be weighted according to the organisation's needs.

### \37. AI Gateway Implementation Roadmap

The move to an enterprise AI Gateway can be carried out in stages.

#### Phase 1: Inventory

The AI services in use are identified.

Model providers and applications are inventoried.

#### Phase 2: Risk assessment

Sensitive data flows are examined.

Agent and tool permissions are assessed.

#### Phase 3: Policy design

A list of approved models is prepared.

Data classification policies are created.

Authorisation boundaries are defined.

#### Phase 4: Pilot implementation

Controls are applied to a specific user group or application.

False positives and performance impacts are measured.

#### Phase 5: Rollout

Suitable applications are brought under the central control layer.

#### Phase 6: Monitoring and improvement

Security events are evaluated.

Policies are updated.

Security tests are repeated.

This approach supports the controlled development of enterprise AI security.

### \38. AI Gateway Checklist

#### Visibility

- Have enterprise AI services been inventoried?
- Have Shadow AI risks been assessed?
- Have model providers been identified?
- Have usage channels outside the Gateway been examined?

#### Data security

- Has sensitive data been classified?
- Have AI DLP policies been created?
- Have data masking requirements been identified?
- Has data transfer to external models been assessed?

#### Prompt security

- Are Prompt Injection controls in place?
- Have Jailbreak controls been assessed?
- Is untrusted RAG content being checked?
- Are model outputs being evaluated from a security perspective?

#### Agent security

- Have Agent tools been inventoried?
- Have Tool Calling permissions been restricted?
- Have MCP integrations been assessed?
- Do critical operations require approval?

#### Operational security

- Is audit logging in place?
- Has SIEM integration been assessed?
- Have usage quotas been defined?
- Has high availability been planned?
- Have security tests been performed?

### Frequently Asked Questions

#### What is an AI Gateway?

An AI Gateway is a control layer that helps manage the communication between AI applications and model services centrally and enforce specific security policies.

#### What does an AI Gateway do?

It can support model access management, data protection, usage monitoring, policy enforcement and, in some architectures, Agent tool controls.

#### What is a Prompt Firewall?

A Prompt Firewall is a security mechanism aimed at detecting specific manipulation and policy violation risks in AI inputs or related content.

#### Does a Prompt Firewall block all Prompt Injection attacks?

No. Prompt Firewall controls can reduce risk, but they do not guarantee that all attack variations will be blocked.

#### What is AI DLP?

AI DLP refers to data protection controls aimed at reducing the unauthorised or inappropriate transfer of sensitive data in AI usage.

#### What is Shadow AI?

Shadow AI refers to AI tools and services used without the organisation's knowledge or approval.

#### Can an AI Gateway completely detect Shadow AI usage?

Not always. For usage outside the Gateway, endpoint, network or SaaS visibility controls may be needed.

#### Is an AI Gateway the same thing as an API Gateway?

No. In addition to API management, an AI Gateway can provide AI-specific model, prompt and data controls.

#### Can an AI Gateway run on-premises?

Some solutions can be run on corporate infrastructure. This capability depends on the product's architecture.

#### When an on-premises AI Gateway is used, does data never leave the organisation?

Not necessarily. Content sent to an external model provider can be transferred outside the organisation. The data flow must be assessed end to end.

#### Does an AI Gateway ensure AI Agent security?

An AI Gateway can provide certain control and monitoring capabilities. However, the Agent tools' own authorisation controls are also required.

#### Can an AI Gateway control MCP integrations?

Depending on the product's architecture and integration support, certain MCP or tool traffic controls can be applied.

#### Why is Human-in-the-Loop important?

It helps ensure that high-impact AI operations are controlled through independent human review.

#### Can an AI Gateway integrate with a SIEM?

Solutions with suitable integration support can forward security events to SIEM systems.

#### Does using an AI Gateway eliminate the need for an LLM Pentest?

No. The effectiveness of AI Gateway controls and the other security layers of the application must be tested separately.

#### Which organisations is an AI Gateway suitable for?

It can be considered by organisations that use multiple AI applications, process sensitive data or want to manage AI Agent permissions centrally.

### Conclusion: Enterprise AI Security Must Be Centralised and Auditable

AI systems are playing an ever-greater role in organisations' business processes.

Employees use external models.

Applications connect to LLM APIs.

RAG systems process corporate documents.

AI Agents communicate with different tools.

And some systems can carry out operations on behalf of users.

For this reason, AI security should not be left solely to the model provider's security features.

Organisations must be able to enforce their own data and operation policies.

Which models will be used must be determined.

Sensitive data transfers must be controlled.

Prompt Injection risks must be assessed.

Agent tool permissions must be restricted.

Critical operations must require approval.

Security events must be monitored.

An AI Gateway can be an important architectural component for centrally managing certain parts of these needs.

However, for a successful security approach, the Gateway must be considered together with IAM, DLP, SIEM, application authorisation and security testing.

This is because AI security is not just about inspecting communication traffic.

Data access, operation permissions and operational controls are also integral parts of security.

**In enterprise AI security, the goal is not to stop AI usage but to make it sustainable in a secure, controlled and auditable way.**