Using feature transformation and selection with polynomial networks

Polynomial networks have proven successful in authentication applications such as speaker recognition. A drawback of these methods is that as the degree of the polynomial network is increased, the number of model terms increases rapidly. This rapid increase can result in overfitting and make the network difficult to use in real-world applications because of the large number of model terms. We propose and contrast two solutions to this problem. First, we show how random dimension reduction can be used to effectively control model complexity. We describe a novel method which allows quick reduction of the dimension using an FFT. Applying these methods to a speaker recognition problem shows an approximately linear relation between the log of the number of model parameters and the log of the error rate. Second, we apply several methods of feature selection to reduce both model complexity and computation. We survey several methods and show which method yields the best performance in a speaker recognition application.