Algebraic Analysis for Singular Statistical EstimationAuthor: Sumio Watanabe. Source: Lecture Notes in Artificial Intelligence Vol. 1720, 1999, 39 - 50. Abstract. This paper clarifies learning efficiency of a non-regular parametric model such as a neural network whose true parameter set is an analytic variety with singular points. By using Sato's b-function we rigorously prove that the free energy or the Bayesian stochastic complexity is asymptotically equal to _{1} log n - (m_{1} - 1) log log n + constant, where _{1} is a rational number, m_{1} s a natural number, and n is the number of training samples. Also we show an algorithm to calculate _{1} and m_{1} based on the resolution of singularity. In regular models, 2_{1} is equal to the number of parameters and m_{1} = 1, whereas in non-regular models such as neural networks, 2_{1} is smaller than the number of parameters and m_{1} 1. ©Copyright 1999 Springer-Verlag |