Real-time NLP moderation on XMPP with GPU-accelerated inference and adaptive batching

Real-time communication platforms generate continuous streams of short, latency-sensitive messages, creating a demanding environment for automated toxicity detection. Transformer-based language models offer strong contextual accuracy, but their inference cost makes them difficult to deploy efficiently in decentralized messaging systems such as the Extensible Messaging and Presence Protocol, where requests arrive asynchronously from multiple servers. Traditional batching strategies struggle in this setting because traffic patterns fluctuate, message lengths vary, and strict latency budgets prevent the accumulation of large batches. This work introduces a full-stack moderation system that combines an OpenFire server plugin, a Graphics Processing Unit (GPU) accelerated microservice for toxicity classification, and an adaptive batch processing method suitable for use in multi-server systems. The plugin intercepts messages at the packet-processing layer and forwards them to a lightweight external inference service that can run on either a Central Processing Unit or a GPU. We introduce Adaptive Cross-Domain Batching (ACDB) as a method to dynamically adjust batch sizes based on the state of the queues, characteristics of the messages being processed, and real-time feedback from GPU usage. Experiments demonstrate that GPU inference provides significant improvements in both throughput and latency, with compute accelerations of up to 28× and total end-to-end accelerations of up to 9×. Comparative evaluation against static batching baselines across multiple traffic scenarios shows that ACDB dynamically adapts batch sizes from 3 to 25 depending on load conditions, reducing latency by 52–59 % compared to large-batch static configurations while maintaining 85–94 % of their throughput. Overall, this system enables scalable, real-time toxic content moderation capabilities for federated chat systems using adaptive batching for Transformer-based inference on dynamic workloads, while preserving strict message delivery constraints under realistic operating conditions.

1. Park Y., Budhathoki K., Chen L., Kübler J.M., Huang J., Kleindessner M., et al. Inference optimization of foundation models on AI accelerators. Proc. of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD ’24), 2024, pp. 6605-6615. doi: 10.1145/3637528.3671465

2. Aminabadi R.Y., Rajbhandari S., Awan A.A., Li C., Li D., Zheng E., et al. DeepSpeed-inference: Enabling efficient inference of transformer models at unprecedented scale. Proc. of the SC22: International Conference for High Performance Computing, Networking, Storage and Analysis, 2022, pp. 1–15. doi: 10.1109/SC41404.2022.00051

3. Khan Z. Natural language processing techniques for automated content moderation. International Journal of Web of Multidisciplinary Studies, 2025, vol. 2, no. 2, pp. 21–27.

4. Liaskovska S., Bacarra R., Martyn Y., Baidych V., Alsayaydeh J. Artificial intelligence for automatic moderation of textual content in online chats and social networks. International Journal of Electrical and Computer Engineering (IJECE), 2025, vol. 15, no. 3, pp. 3396-3409. doi: 10.11591/ijece.v15i3.pp3396-3409

5. Fillies J., Mitsikas T., Schäfermeier R., Paschke A. A hate speech moderated chat application: Use case for GDPR and DSA compliance. arXiv, 2024. arXiv:2410.07713. doi: 10.48550/arXiv.2410.07713

6. Mengade S., Chopade P., Tate P., Patil S. Facilitating anonymous communication on social networks via AI-driven content moderation. Journal of the ACS Advances in Computer Science, 2025, vol. 16, no. 1, pp. 19–32. doi: 10.21608/asc.2025.351475.1033

7. Wagner N., Kraus M., Tonn T., Minker W. Comparing moderation strategies in group chats with multi-user chatbots. Proc. of the 4th Conference on Conversational User Interfaces, 2022, pp. 1–4. doi: 10.1145/3543829.3544527

8. Zhang D., Luo Y., Wang Y., Kui X., Ren J. BatOpt: Optimizing GPUbased deep learning inference using dynamic batch processing. IEEE Transactions on Cloud Computing, 2024, vol. 12, no. 1, pp. 174–185. doi: 10.1109/TCC.2024.3350561

9. Cang Y., Chen M., Huang K. Joint batching and scheduling for highthroughput multiuser edge AI with asynchronous task arrivals. IEEE Transactions on Wireless Communications, 2024, vol. 23, no. 10, pp. 13782–13795. doi: 10.1109/TWC.2024.3404811

10. Fang J., Yu Y., Zhao C., Zhou J. TurboTransformers: An efficient GPU serving system for transformer models. Proc. of the 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP ’21), 2021, pp. 389–402. doi: 10.1145/3437801.3441578

11. Kasoju A., Vishwakarma T.V.C. Optimizing transformer models for low-latency inference: Techniques, architectures, and code implementations. International Journal of Science and Research (IJSR), 2025, vol. 14, no. 4, pp. 857–866. doi: 10.21275/sr25409073105

12. Cong P., Chen Q., Zhao H., Yang T. Baton: Enhancing batch-wise inference efficiency for large language models via dynamic rebatching. Proc. of the ACM on Web Conference, 2025, pp. 2309–2318. doi: 10.1145/3696410.3714950

13. Lam M., Yedidia Z., Banbury C.R., Reddi V.J. Precision batching: Bitserial decomposition for efficient neural network inference on GPUs. Proc. of the 30th International Conference on Parallel Architectures and Compilation Techniques (PACT), 2021, pp. 129-141. doi: 10.1109/PACT52795.2021.00017

14. Speckhard D.T., Bechtel T., Kehl S., Godwin J., Draxl C. Training speedups via batching for geometric learning: An analysis of static and dynamic algorithms. arXiv, 2025. arXiv:2502.00944. doi: 10.48550/arXiv.2502.00944

15. Aldarf A., Shaker A., Bessmertny I., Sobeih M.Y. Accelerating hate speech classification with GPUs: A multi-layer approach with string matching and transformers. Lecture Notes in Networks and Systems, 2025, vol. 5588, pp. 535–548. doi: 10.1007/978-981-96-1918-4_38

16. Yang Z., Grenon-Godbout N., Rabbany R. Towards detecting contextual real-time toxicity for in-game chat. Findings of the Association for Computational Linguistics: EMNLP, 2023, pp. 9894-9906. doi: 10.18653/v1/2023.findings-emnlp.663