Adaptive API Abuse Detection with Hybrid Anomaly–Classifier Risk Scoring and Graduated Mitigation

Author

Deepak Marlecha, Zarana J Shah, Aditi Singh, Dr. G. Paavai Anand

Abstract

 Most public APIs are still protected by fixed rules: a request-per-minute ceiling, a cap on failed logins, a blocklist. These rules are easy to reason about and easy to defeat. I observed that a scraper moving below the ceiling usually goes unnoticed but a normal power user who momentarily goes above that ceiling is stopped. This paper explains a gateway‑side detector. The gateway‑side detector swaps the threshold, for a risk score. Every client session is described by eleven features that are calculated over a sliding sixty-second window. These behavioural features, which include request cadence, endpoint diversity, error rate, authentication-failure rate, payload size, burstiness and the fraction of requests that target paths are used by the gateway-side detector to compute the risk score. An Isolation Forest trained only on benign sessions produces an anomaly score; an XGBoost classifier trained on labelled abuse profiles produces an attack probability and a category; a per-client exponential-moving-average profile measures how far the current window departs from that client's own history. We combine these signals with a term that measures how sensitive the endpoint is. The result is a score that ranges from zero to one hundred. That score determines four actions: allow, monitor, token‑bucket throttle or a five‑minute block. On a held-out set of 2,000 sessions the hybrid system reaches 99.15% accuracy with no false positives on 1,300 benign sessions, compared with 180 false positives for the Isolation Forest alone. We report the comparison with rule-based and single-model baselines, show that the risk distribution separates the two classes cleanly, and discuss at some length why the headline numbers should be read cautiously: the holdout set comes from the same synthetic generator as the training data, and the rule-based baseline scores far higher on it than it would on real traffic.

Keywords

API security; abuse detection; anomaly detection; Isolation Forest; XGBoost; behavioural profiling; risk scoring; rate limiting; API gateway; adaptive mitigation.

Full Text:

Download Paper PDF

References

[1] OWASP Foundation, “OWASP API Security Top 10,” 2023. https://owasp.org/www-project-api-security/

[2] Tangodelta, “API access behaviour anomaly dataset,” Kaggle, 2021. https://www.kaggle.com/datasets/tangodelta/api-access-behaviour-anomaly-dataset

[3] Ariel University Cyber Innovation Lab and Cisco, “API security challenge dataset (ATRDF),” GitHub, 2022. https://github.com/ArielCyber/Cisco_Ariel_Uni_API_security_challenge

[4] F. T. Liu, K. M. Ting, and Z.-H. Zhou, “Isolation forest,” in Proc. 8th IEEE International Conference on Data Mining (ICDM), pp. 413–422, 2008.

[5] C. Kruegel and G. Vigna, “Anomaly detection of web-based attacks,” in Proc. 10th ACM Conference on Computer and Communications Security (CCS), pp. 251–261, 2003.

[6] T. Chen and C. Guestrin, “XGBoost: A scalable tree boosting system,” in Proc. 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 785–794, 2016.

[7] F. Pedregosa et al., “Scikit-learn: Machine learning in Python,” Journal of Machine Learning Research, vol. 12, pp. 2825–2830, 2011.

[8] S. Ramírez, “FastAPI,” 2018–. https://fastapi.tiangolo.com/

[9] V. Chandola, A. Banerjee, and V. Kumar, “Anomaly detection: A survey,” ACM Computing Surveys, vol. 41, no. 3, pp. 1–58, 2009.

Share your valuable work from Social Media Buttons