## A Stochastic Gradient Descent Algorithm for Structural Risk Minimisation
When learning multicategory classification there are M pattern classes
with a priori probabilities p,
1 _{i}i
M.
Using the usual
representation for a multi-category classifier as M
individual boolean classifiers, the penalty becomes
_{}.
If the
m are given then the standard SRM trivially applies here
by minimizing the penalised empirical risk with respect to
_{i}k, _{i}i = 1, ..., M.
However, in situations where the total sample size
i = 1, ..., M. The
obvious problem is that the empirical risk can only be
defined after the subsamples (and hence their sizes) are
given (known).
Utilising an on-line stochastic gradient descent approach,
this paper overcomes this difficulty and introduces a
sample-querying algorithm which extends the standard
SRM principle. It minimises the penalised empirical risk
not only with respect to the m, _{i}i = 1, ..., M.
The challenge here is in defining a stochastic empirical
criterion which when minimised yields a sequence of
subsample-size vectors which asymptotically achieve the
Bayes' optimal error convergence rate.
©Copyright 2003 Springer |