Universal approximation capacity of bidirectional recurrent neural networks with noisy sequential inputs

In this paper, we prove that, under certain reasonable assumptions, bidirectional recurrent neural networks (BRNNs) can well approximate a class of bidirectional dynamical systems. As a special case, we prove that the forward recursions of BRNNs are capable of accurately approximating any filtering dynamics, while the bidirectional recursions of BRNNs satisfactorily approximate the corresponding smoothing dynamics. We present explicit estimation bounds and utilize these theoretical results to construct a BRNN-based smoother, which offers a synthetic resolution to the smoothing problem. By training a BRNN on sequences of observations and their corresponding states produced by computer simulations or actual experiments, we demonstrate that the BRNN-based smoother can closely approximate the optimal smoother, including approximating the optimal filter as its forward recursion. Finally, we support our theoretical findings through a numerical example and compare our results numerically against the extended Kalman smoother and particle smoother.