How Temporal Correlations Shape Memory in Linear Recurrent Neural Networks

Research into linear mappings and Linear Recurrent Neural Networks (LRNN) reveals how memory functions in simple models. Linear mappings treated as associative memory show that training beyond capacity adds noise to old associations rather than erasing them, though recent examples are recalled better. Meanwhile, LRNN studies demonstrate that temporal correlations in input sequences increase the cost of retaining past information. This correlation reshapes learning dynamics, influencing whether a network maintains memory or reacts only to present inputs.