Probabilistic Interpretation of Feedforward Network Outputs, With Relationships to Statistical Prediction of Ordinal Quantities

Several problems require the estimation of discrete random variables whose values can be put in a one-to-one ordered correspondence with a finite subset of the natural numbers. This happens whenever quantities are involved that represent integer items, or have been quantized on a fixed number of levels, or correspond to "graded" linguistic values. Here we propose a correct probabilistic approach to such kind of problems that fully exploits all the available prior knowledge about their own structure. In spite of the very stringent constraints induced in output space, the method can be directly applied to standard feed-forward networks while keeping local computation of both outputs and error signals. According to these guidelines, we devised a neural implementation of a complex image pre-processing algorithm by using very poor resolution on the computing elements in the network.