Feb. 17, 11:45, Wei Ji Ma. Optimality and Probabilistic Computation in Visual Categorization

Robert Jacobs

University of Rochester

Theoretical perspectives on visual short-term memory

We propose two new theoretical perspectives on visual short-term memory (VSTM). In Project 1, we conceptualize VSTM as an information channel for transmitting visual information from the past to the present. This information-theoretic perspective allows us to quantify the capacity and precision of VSTM in a rigorous fashion, and to understand the trade-offs between these two properties. According to this framework, one cannot simply talk about whether an agent has "good" or "poor" memory because memory performance is influenced by the statistical properties of the to-be-remembered items, and the agent's knowledge of these properties, thus establishing an important connection between learning and memory. In Project 2, we conceptualize VSTM as performing a multi-scale clustering of the to-be-remembered items based on the items' feature similarities. This framework accounts for several biases and dependencies often found in people's visual memories. Importantly, VSTM learns representations at multiple scales or granularities, and thus there is no need for researchers to pre-specify hierarchies to account for VSTM performances. Project 1 was done in collaboration with Chris Sims and David Knill. Project 2 was done in collaboration with Emin Orhan.