On the properties of compromise M-estimators optimizing weighted L2-norm of the influence function
https://doi.org/10.17586/2226-1494-2025-25-3-527-535
Abstract
The paper develops a theory of M-estimators optimizing the weighted L2-norm of the influence function. The specified criterion of the estimation quality is quite general and, in addition, allows obtaining solutions related to the class of redescending estimators, i.e., possessing the property of stability to asymmetric contamination. Such estimators, in particular, were studied within the framework of the locally stable approach of A.M. Shurygin, based on the analysis of the estimator instability functional (L2-norm of the influence function), or his approach based on the model of a series of samples with random point contamination (point Bayesian contamination model). In this paper, a compromise family of estimators is studied for which the optimized functional is a convex linear combination of two basic criteria. The compromise family is similar to the conditionally optimal family of estimators proposed by A.M. Shurygin, but the criteria used can be squares of the weighted L2-norms of the influence function with arbitrary pre-known weight functions. The considered subject area has remained little-studied to date. In the course of the research, we used a theory we had developed earlier, which describes the properties of estimators that optimize the weighted L2-norm of the influence function. As a result of the study, a number of properties of compromise estimators were obtained, and the uniqueness of the family elements was shown. A family member that delivers equal values to the criteria was considered separately: it was shown that this estimator corresponds to the saddle point of the optimized functional, and is also a minimax solution with respect to the basic criteria on the set of all regular score functions. The constructed theory is illustrated using the example of the problem of mathematical expectation estimating of a normal distribution under conditions of targeted malicious influence on a data set (similar to a data poisoning attack in malicious machine learning).
About the Authors
D. V. Lisitsin
Novosibirsk State Technical University
Russian Federation
Russian Federation
Daniil V. Lisitsin — D.Sc., Full Professor
Novosibirsk, 630073
sc 57199594779
K. V. Gavrilov
Novosibirsk State Technical University
Russian Federation
Russian Federation
Konstantin V. Gavrilov — PhD, Associate Professor
Novosibirsk, 630073
sc 58966384700
References
1. Borovkov A.A. Mathematical Statistics. Amsterdam: Gordon and Breach, 1998, 570 p.
2. Shurygin A. M. Applied Stochastics: Robustness, Estimation, Prediction. Moscow, Finansy i statistika Publ., 2000, 223 p. (in Russian)
3. Huber P., Ronchetti E. Robust Statistics. John Wiley & Sons, 2009, 354 p. https://doi.org/10.1002/9780470434697
4. Hampel F., Ronchetti E., Rousseeuw P., Stahel W. Robust Statistics: The Approach Based on Influence Functions. John Wiley & Sons, 2005, 536 p. https://doi.org/10.1002/9781118186435
5. Lisitsin D.V., Gavrilov K.V. The use of maximum entropy principle to construct robust estimators under point Bayesian contamination. Part I. Applied Mathematics and Control Sciences, 2024, no. 1, pp. 55–72. (in Russian). https://doi.org/10.15593/2499-9873/2024.1.04
6. Lisitsin D.V. Robust Methods for Parameters Estimating of Statistical Models. Novosibirsk, NSTU Publ., 2013, 76 p. (in Russian)
7. Lisitsin D.V., Gavrilov K.V. The use of maximum entropy principle to construct robust estimators under point Bayesian contamination. Part II. Applied Mathematics and Control Sciences, 2024, no. 2, pp. 18–33. (in Russian). https://doi.org/10.15593/2499-9873/2024.2.02
8. Lisitsin D.V., Gavrilov K.V. On the properties of M-estimators optimizing weighted L2-norm of the influence function. Scientific and Technical Journal of Information Technologies, Mechanics and Optics, 2024, vol. 24, no. 2, pp. 267–275. (in Russian). https://doi.org/10.17586/2226-1494-2024-24-2-267-275
9. Lisitsin D.V. Robust estimation of model parameters in presence of multivariate nonhomogeneous incomplete data. Science Bulletin of the Novosibirsk State Technical University, 2013, vol. 50, no. 1, pp. 17–30. (in Russian)
10. Lisitsin D.V., Gavrilov K.V. On stable estimation of models parameters in presence of asymmetric data contamination. Science Bulletin of the Novosibirsk State Technical University, 2008, vol. 30, no. 1, pp. 33–41. (in Russian)
11. Shevlyakov G.L., Oja H. Robust Correlation: Theory and Applications. John Wiley & Sons, 2016, 352 p. https://doi.org/10.1002/9781119264507
12. Gavrilov K.V., Veretel’nikova E.L. On one way to choose a compromise in a family of conditionally optimal estimators. Tomsk State University Journal of Control and Computer Science, 2024, no. 67, pp. 60–68. (in Russian). https://doi.org/10.17223/19988605/67/7
13. Coretto P., Hennig C. Robust improper maximum likelihood: tuning, computation, and a comparison with other methods for robust Gaussian clustering. Journal of the American Statistical Association, 2016, vol. 111, no. 516, pp. 1648–1659. https://doi.org/10.1080/01621459.2015.1100996
14. Rieder H., Kohl M., Ruckdeschel P. The cost of not knowing the radius. Statistical Methods and Applications, 2008, vol. 17, no. 1, pp. 13–40. https://doi.org/10.1007/s10260-007-0047-7
15. Smolyak S.A., Titarenko B.P. Stable Estimation Methods: Statistical Processing of Heterogeneous Aggregates. Moscow, Statistika Publ., 1980, 210 p. (in Russian)
16. Lisitsin D.V., Gavrilov K.V. Maximin problem of parameter estimation in conditions of point Bayesian contamination. Tomsk State University Journal of Control and Computer Science, 2023, no. 62, pp. 56–64. (in Russian). https://doi.org/10.17223/19988605/62/6
17. Lisitsin D.V., Gavrilov K.V. Estimation of distribution parameters of a bounded random variable robust to bound disturbance. Science Bulletin of the Novosibirsk State Technical University, 2016, vol. 63, no. 2, pp. 70–89. (in Russian). https://doi.org/10.17212/1814-1196-2016-2-70-89
18. Shevlyakov G., Morgenthaler S., Shurygin A. Redescending M-estimators. Journal of Statistical Planning and Inference, 2008, vol. 138, no. 10, pp. 2906–2917. https://doi.org/10.1016/j.jspi.2007.11.008
19. Shurygin A.M. New approach to optimization of stable estimation. Proc. of the First US/Japan Conference on the Frontiers of Statistical Modeling: An Informational Approach, 1994, vol. 3, pp. 315–340. https://doi.org/10.1007/978-94-011-0854-6_15
20. Nogin V.D. Decision Making in Multicriteria Environment: a Quantitative Approach. Moscow, Fizmatlit Publ., 2004, 176 p. (in Russian)
21. Podinovski V.V. Ideas and Methods of Criteria Importance Theory in Multicriteria Decision-Making Problems. Moscow, Nauka Publ., 2019, 103 p. (in Russian)
22. Li F., Lai L., Cui Sh. Machine Learning Algorithms: Adversarial Robustness in Signal Processing. Springer, 2022, 104 p. https://doi.org/10.1007/978-3-031-16375-3
23. Großhans M., Sawade C., Brückner M., Scheffer T. Bayesian games for adversarial regression problems. Proc. of the 30th International Conference on Machine Learning. International Conference on Machine Learning, 2013, vol. 28. pp. 55–63. https://doi.org/10.5555/3042817.3042943
24. Aivazyan S.A., Yenyukov I.S., Meshalkin, L.D. Applied Statistics: Bases of Modelling and Initial Data Processing. Moscow, Finansy i statistika, Publ., 1983, 471 p. (in Russian)
25. Haykin S. Neural Networks: A Comprehensive Foundation. Prentice Hall, 1998, 842 p.
Review
For citations:
Lisitsin D.V., Gavrilov K.V. On the properties of compromise M-estimators optimizing weighted L2-norm of the influence function. Scientific and Technical Journal of Information Technologies, Mechanics and Optics. 2025;25(3):527-535. (In Russ.) https://doi.org/10.17586/2226-1494-2025-25-3-527-535
Views:
7
Email this article 