WHAT CAN YOU DO WITH THE NICO TOOLKIT?

Although it is mainly intended for, and originally developed for speech recognition applications, the NICO toolkit is a general purpose toolkit for constructing artificial neural networks and training with the back-propagation learning algorithm. The network topology is very flexible. Units are organized in groups and the group is a hierarchical structure, so groups can have sub-groups or other objects as members. This makes it easy to specify multi-layer networks with arbitrary connection structure and to build modular networks.

Recurrency and time-delays are constrained only by the condition that the activation of a unit cannot be dependent on its own activity at the present or future times (this is checked automatically every time the user connects between two groups or units). A convenient feature is that the user can specify not only time-delay connections but also look-ahead connections. All connections are converted to time-delays when the network runs, but this is completely hidden from the user.

The NICO toolkit was developed to solve a particular problem (phoneme probability estimation for automatic speech recognition) and the focus was not on research in the field of artificial neural networks. There are no fancy graphical tools to monitor network behavior or characteristics . But if you want to solve a real problem with a large database (millions of samples) and possibly large networks (e.g. 500.000 connections or more), then this toolkit is probably still a good choice as it is optimized for fast training and evaluation on large networks.

The communication with external data is handled conveniently by stream objects in the network. A network may have multiple input and output streams. The streams perform the normalization of the data to values in the range suitable for ANN computation. Streams can read data in several different file formats.

The only training algorithm available is a fast implementation of the back-propagation learning algorithm. It is optimized for high performance back-propagation through time, i.e., for training of networks with recurrency and/or time-delay windows. The reason that none of the theoretically more advanced algorithms (like quasi-newton, conj. grad. etc.) are available is that the NICO toolkit was developed for training large networks with the training typically terminated before the minimum error is reached (to prevent over-training) and weight updating many times per epoch. For this type of problems, back-propagation with momentum is a good choice.

Units of several different types can be arbitrarily mixed in a network. Some of the unit types are:

Sigmoid units
Tanhyp units
Linear units
Multiplication units
Exponential units e^x
Inverter units 1/x
Environment units (takes the activation value from the operating system environment)
File filter units (boolean unit that checks the filename of the input data)

and other unit types with for example x², sigma-pi, softmax, and radial basis activation functions can be modeled by combining the primitive unit types above.

Many different error functions are supported and can be arbitrarily mixed in a network. Some of the error functions are:

Mean square error
Cross entropy
Absolute value

WHAT CAN'T YOU DO?

No graphics support for viewing the networks.
No support for other types of training than back-propagation .
Weight-sharing is not possible in the current implementation.
The tools provided compute the activities of the ANN. The hybrid HMM/ANN speech recognition paradigm also requires HMM software that is not included in this toolkit.

For more documentation, visit the online manual, or go back to the NICO home page.

(C) 1996 Nikko Strom, All rights reserved.
Permission to use, copy, or modify this documentation and the software itself for educational and research purposes only and without fee is hereby granted, provided that this copyright and permission notice appear on all copies and supporting documentation. For any other uses of this software, in original or modified form, including but not limited to distribution in whole or in part, specific prior permission from the copyright holder must be obtained. The copyright holder makes no representations about the suitability of this software for any purpose. It is provided "as is" without express or implied warranty.