Algebraic Analysis for Singular Statistical Estimation

Author: Sumio Watanabe.

Source: Lecture Notes in Artificial Intelligence Vol. 1720, 1999, 39 - 50.

Abstract. This paper clarifies learning efficiency of a non-regular parametric model such as a neural network whose true parameter set is an analytic variety with singular points. By using Sato's b-function we rigorously prove that the free energy or the Bayesian stochastic complexity is asymptotically equal to lambda1 log n - (m1 - 1) log log n + constant, where lambda1 is a rational number, m1 s a natural number, and n is the number of training samples. Also we show an algorithm to calculate lambda1 and m1 based on the resolution of singularity. In regular models, 2lambda1 is equal to the number of parameters and m1 = 1, whereas in non-regular models such as neural networks, 2lambda1 is smaller than the number of parameters and m1 1.

©Copyright 1999 Springer-Verlag