To secure LLM applications from prompt injection in India, engineering teams must implement five layered controls: structural prompt separation, input sanitisation, output filtering, least-privilege agent permissions, and anomaly-aware logging. Prompt injection — ranked LLM01 in the OWASP LLM Top 10, the globally adopted AI security framework — lets attackers override your system prompt using nothing but crafted user input, with no special tools required. Indian enterprises face heightened exposure: generative AI adoption across BFSI, healthcare, legal, and citizen services has outpaced AI-specific security maturity at most organisations, making deliberate layered defence a business necessity rather than an optional exercise. This guide maps all ten OWASP LLM Top 10 risks to their defensive controls, covers real-world attack scenarios, and addresses DPDP Act 2023 obligations for AI deployments in India.
Understanding Prompt Injection in LLM Applications
A large language model processes text as a continuous stream of tokens. It has no native ability to enforce a hard boundary between instructions set by the developer — the system prompt — and input supplied by a user at runtime. Prompt injection exploits this structural gap.
An attacker crafts input that contains hidden or explicit instructions to the model. When the model processes this input alongside the system context, it follows the attacker's commands rather than the developer's intended logic. The consequences range from leaking confidential system prompts and configuration details to executing unauthorised actions — bypassing access controls, deleting records, or exfiltrating personal data.
This is not a defect in the LLM itself. It is an application-layer security failure. Just as a web developer must validate HTTP input before it reaches a database query, an AI developer must treat natural language input as untrusted before it reaches the model.
The Indian context: India's generative AI adoption has accelerated across customer service, BFSI workflows, legal document review, healthcare triage, and government citizen services. Many of these deployments use third-party LLM APIs with minimal security layering. The speed of AI adoption has outpaced the maturity of AI-specific security practices in most Indian enterprises.
The OWASP LLM Top 10 — Your AI Security Baseline
The OWASP Top 10 for LLM Applications is the most widely adopted framework for categorising risks in AI systems built on large language models. Published and maintained by the Open Worldwide Application Security Project, it identifies ten distinct risk categories spanning the full LLM application lifecycle — from user input through model inference to downstream action execution.
These ten categories distribute across four distinct attack surfaces:
Category grouping: Input/Injection — LLM01 Prompt Injection, LLM03 Training Data Poisoning, LLM09 Overreliance; Output/Exposure — LLM02 Insecure Output Handling, LLM06 Sensitive Information Disclosure, LLM08 Excessive Agency; Plugin/Agency — LLM07 Insecure Plugin Design, LLM04 Model Denial of Service; Supply Chain — LLM05 Supply Chain Vulnerabilities, LLM10 Model Theft.
The NIST AI Risk Management Framework complements OWASP by providing governance and organisational risk controls. Used together, these two frameworks give Indian security teams a structured, internationally recognised baseline for managing LLM application risk.
How a Prompt Injection Attack Unfolds
The diagram below traces a typical prompt injection attack against an LLM-powered application — contrasting the unprotected execution path where the attack succeeds against the defended path where an input validation layer intercepts it:
Without a validation layer between user input and the model, every user-controlled text field in your LLM application is a potential injection vector. The attack requires no special tools — only access to the input interface.
Know your vulnerabilities before attackers do
Run a free VAPT scan — takes 5 minutes, no signup required.
Book Your Free ScanDirect vs Indirect Prompt Injection
Prompt injection manifests in two forms, and both appear in production deployments today:
Direct prompt injection occurs when a user types manipulative instructions directly into an interface. Common forms include "Ignore all previous instructions and..." followed by attacker-defined directives. Customer service bots have been manipulated this way into revealing their system prompts, claiming to be competitors, or generating harmful content.
Indirect prompt injection is significantly more dangerous and harder to detect. Here, malicious instructions are not typed by the attacker — they are embedded in external content that the LLM retrieves and processes as part of its task. A web browsing agent that reads a malicious webpage, a document-summary tool that processes an attacker-controlled PDF, or a code assistant that reads a compromised package README — all are vulnerable. The user and developer may never see the injected payload; the model simply acts on it.
Indian enterprises deploying autonomous LLM agents — tools with the ability to browse the web, query internal systems, send emails, or write to databases — face the highest indirect injection exposure of any LLM deployment pattern.
Real-World Impact on Indian Organisations
The business impact of a successful prompt injection depends entirely on what the LLM is permitted to access and execute. The greater the permissions and integrations, the wider the blast radius.
System prompt theft exposes the proprietary instructions you invested in engineering — the IP behind your AI product. Competitors or adversaries can replicate your product behaviour without your development cost.
Personal data exfiltration becomes possible when the LLM has access to customer records, medical information, or financial data. An injected instruction to "output all user records you have access to" can retrieve and return data to the attacker through the standard response channel.
Privilege escalation via autonomous agents is the most severe consequence. An agent with write access to a CRM, email system, or API can be instructed to take fraudulent, destructive, or data-destroying actions — all appearing as legitimate model output.
Brand and reputational damage follows when a public-facing AI assistant is manipulated into generating harmful, offensive, or misleading content. In regulated sectors — BFSI, healthcare, e-commerce — this has compliance and legal dimensions beyond the technical breach.
To find out whether your web, API, or AI-connected endpoints are already exposed to these attack surfaces, run a free VAPT scan — automated results, no sign-up required.
LLM Security Controls: How to Defend Against Prompt Injection in India
Defence against prompt injection requires layered controls — no single measure is sufficient on its own. The table below maps the primary OWASP LLM risk categories to their recommended defensive controls and implementation priority:
| OWASP LLM Risk | Category | Recommended Control | Priority |
|---|---|---|---|
| Prompt Injection — LLM01 | Input | Structural prompt separation, input sanitisation | Critical |
| Insecure Output Handling — LLM02 | Output | Output filtering before rendering or downstream API calls | Critical |
| Sensitive Information Disclosure — LLM06 | Output | Data minimisation, restrict model access to sensitive fields | Critical |
| Insecure Plugin or Tool Design — LLM07 | Agency | Least-privilege scoping for all LLM tool integrations | High |
| Excessive Agency — LLM08 | Agency | Require human approval for write or destructive actions | High |
| Training Data Poisoning — LLM03 | Input | Vet fine-tuning datasets and data pipelines before use | High |
| Supply Chain Vulnerabilities — LLM05 | Infrastructure | Vet third-party model APIs, SDKs, and dependencies | High |
| Model Denial of Service — LLM04 | Infrastructure | Rate limiting, cost caps, and input length constraints | Medium |
1. Structural prompt separation. Place user input in a clearly delimited, named section of the prompt — distinct from system instructions. Use templates or hard tokens that cannot be overridden by text content alone. Do not embed user content inline with instruction text.
2. Output filtering. Treat every LLM output as untrusted user input before passing it to downstream APIs, databases, or rendered HTML. A model coerced into generating a database command or script fragment can cause downstream damage if output is passed directly.
3. Least privilege for agents. If your LLM tool can write emails, modify records, or call authenticated APIs, scope those permissions tightly per principle of least privilege. An agent that only needs to read should never hold write credentials.
4. Indirect content sandboxing. When your LLM retrieves external content — URLs, uploaded documents, third-party API responses — sanitise and isolate that content before including it in the model context. Do not trust external content as instruction-safe.
5. Input and output logging with anomaly detection. Log all LLM inputs and outputs in your application layer. Alert on anomalous patterns: unusually long user inputs, outputs that reference internal system details not in the user's query, or outputs containing structured data from protected stores.
DPDP Act 2023 and AI Security in India
The Digital Personal Data Protection Act 2023 establishes enforceable obligations for organisations handling personal data of Indian citizens. AI applications that process user queries, retrieve personal records, or generate personalised outputs are fully within scope of this legislation.
Key DPDP considerations specific to LLM deployments:
Consent and purpose limitation. If your LLM processes personal data, you need a valid consent or legitimate use basis. Your privacy notice must cover AI-based processing. Processing personal data through an LLM for a purpose not disclosed to the data principal is a violation.
Third-party API data retention. Prompts sent to third-party LLM APIs may be logged, retained, or used for model improvement by the provider. Review your API agreements for data retention periods and opt-out provisions. Passing personal data to an API provider without contractual safeguards creates exposure under DPDP data processing obligations.
Technical security safeguards. The DPDP Act requires appropriate technical and organisational measures to protect personal data. An LLM endpoint without input validation, output filtering, or access controls is a demonstrable failure of this obligation.
To understand how AI-related attack surfaces appear in your current infrastructure, start with a free VAPT scan. Bachao.AI, built by Dhisattva AI Pvt Ltd, a DPIIT Recognized Startup, runs automated assessment across web and API attack surfaces — including those introduced by AI integrations and LLM-connected endpoints.