Prompt Firewall: A Server-Side Enforcement Layer for Securing LLM-Based Chat Applications Against Prompt Injection

Author

Sikkandhar A, Saran S, Sujith Kumar G, Sanjay Prasath M,Mr. Nishmitha KR

Abstract

 Large language model (LLM) applications are exposed to prompt injection, in which crafted input tries to override the instructions or safeguards of the application. Checks that run only in the browser cannot stop such input, and many prototype tools can label text as risky without controlling what is actually sent to the AI provider. This paper presents Prompt Firewall, a security-enforcing chatbot application in which the server, not the client, decides whether a prompt may reach an AI provider. The system combines a React chat and administration interface with an Express backend that applies request validation, rate limiting, layered detection, and an ALLOW, REVIEW, or BLOCK decision. Blocked prompts are never forwarded, prompts under review wait for an authorised reviewer, and detector failures are designed to fail safely instead of defaulting to ALLOW. SQLite stores a minimal audit trail, a review queue, and operational statistics, with message-content retention off by default. The work is organised as a phased implementation plan covering repository hygiene, detector contracts, enforcement, authentication, privacy, frontend behaviour, and deployment readiness, together with a test and verification matrix. At the starting point, both React frontends built successfully, the standalone Prompt Wall Detector passed its four API test cases, and all 45 Prompt Firewall backend source and test files passed a Node.js syntax check; the backend test suite could not yet be run because a native SQLite dependency could not be built in the available Windows environment. Detection accuracy and enforcement tests are therefore reported as pending, and the system should be regarded as one layer of defence in depth rather than a guarantee against prompt injection.

Keywords

Prompt Injection, LLM Security, Prompt Firewall, Server-Side Enforcement, Rule-Based Detection, Human Review, Audit Logging, OWASP LLM Top 10.

Full Text:

Download Paper PDF

References

[1] OWASP Foundation. OWASP Top 10 for Large Language Model Applications. Available at: https://owasp.org/www-project-top-10-for-large-language-model-applications/

[2] F. Perez and I. Ribeiro, “Ignore Previous Prompt: Attack Techniques for Language Models,” in NeurIPS ML Safety Workshop, 2022.

[3] K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, and M. Fritz, “Not What You’ve Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection,” in Proc. 16th ACM Workshop on Artificial Intelligence and Security (AISec), 2023.

[4] Y. Liu, Y. Jia, R. Geng, J. Jia, and N. Z. Gong, “Formalizing and Benchmarking Prompt Injection Attacks and Defenses,” in Proc. 33rd USENIX Security Symposium, 2024.

[5] A. Wei, N. Haghtalab, and J. Steinhardt, “Jailbroken: How Does LLM Safety Training Fail?” in Advances in Neural Information Processing Systems (NeurIPS), 2023.

[6] J. H. Saltzer and M. D. Schroeder, “The Protection of Information in Computer Systems,” Proceedings of the IEEE, vol. 63, no. 9, pp. 1278–1308, 1975.

[7] National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1, 2023.

[8] React. React Documentation. Available at: https://react.dev/

[9] Express. Express Documentation. Available at: https://expressjs.com/

[10] SQLite. SQLite Documentation. Available at: https://www.sqlite.org/docs.html

[11] better-sqlite3. better-sqlite3 Repository. Available at: https://github.com/WiseLibs/better-sqlite3

[12] OpenJS Foundation. Node.js Documentation. Available at: https://nodejs.org/docs/latest/api/

[13] Helmet. Helmet Documentation. Available at: https://helmetjs.github.io/

[14] OWASP Foundation. OWASP Cheat Sheet Series. Available at: https://cheatsheetseries.owasp.org/

Share your valuable work from Social Media Buttons