That’s the idea behind the boundary extension, a term which has become widely accepted in psychology classes, textbooks and test-prep flashcards.
But what if the concept isn’t quite accurate?
A University of Chicago psychologist has discovered new evidence that challenges the decades-old understanding of the memory error as a universal phenomenon.
Published in the journal Current Biology, the study proposes that boundary contraction may be just as common as boundary extension – and that whether something appears zoomed in or out depends on the properties of the image itself.
“In a way, we’re debunking this very strong claim that has been made in psychology over the last 30 years,” said Asst. Prof. Wilma Bainbridge, the study’s lead author and an expert on the perception and memorability of images.
The finding is important, she added, because boundary extension has been used to make other claims about the nature of the brain, such as the function of the hippocampus.
Bainbridge co-authored the study with Chris Baker, a principal investigator at the National Institute of Mental Health.
Testing 2,000 participants, they found that although images of objects caused boundary extension, images of full scenes were more likely to produce boundary contraction.
That is, a person may see a close-up photo of an apple and fill in details that were not actually present. But if they see a football field, they may be more likely to remove details – zooming in, or contracting, the actual image.
In a previous study, Bainbridge and Baker showed participants various images and asked them to draw copies.
They were “perplexed” when boundary extension did not occur as often as they had expected.
The study relied on two databases, Google Open Images (SOI) and the Scene Understanding Database (SUN), which categorized images using object- and scene-oriented words. The image is credited to Wilma Bainbridge.
To further investigate those results, they conducted an online experiment using a broad set of 1,000 images and 2,000 participants. Participants would see an image, see a scrambled image and then see the original image again.
Even though the final image was identical to the first, the researchers found that people would indicate it being farther or closer according to its visual properties (object-based vs. scene-based).
Bainbridge said the results highlight the need for psychologists to revisit even long-held assumptions, as well as the potential pitfalls of drawing larger inferences from limited data sets.
Past replications of boundary extension, she suggested, could have been skewed in part by narrow data sets that repeated the use of certain image types.
“Anecdotally, I’ve spoken with many people who have thought about looking at boundary extension – but then they aren’t able to replicate the effects, so they give up and they set aside the data,” she said.
Recent work in healthy individuals has found that certain images are intrinsically memorable or forgettable across observers [1], [2]; there are images of faces or scenes that most people remember or forget, regardless of their different individual experiences.
This memorability of an image can be quantified and predicts 50% of the variance in people’s performance on a memory test [2].
It is intrinsic to the image itself, stable across different image contexts [3], tasks [4], [5], and timing [6], [7]. Viewing memorable images automatically elicits specific neural signatures [8], [9], and the memorability score of an image can be predicted by computational models [10], [11].
However, image attributes such as esthetics, emotionality, typicality, or what people believe will be memorable do not fully predict memorability [2], [12], and memorability is an automatically processed image property that is resilient to the effects of attention [4].
This means that researchers can predict in advance what images a person is likely to remember or forget and use such information to create memorable educational materials or design well-balanced memory tests.
Although memorability has so far been characterized based on healthy participants’ memory behavior, it is unclear if memorability is also consistent in populations with memory impairments at increased risk for Alzheimer’s disease, such as mild cognitive impairment (MCI) or subjective cognitive decline (SCD) [13].
Consistent memorability in SCD and MCI would enable better prediction of what images are likely to be remembered or forgotten.
Furthermore, changes in memorability patterns across disease stages could improve cognitive staging and design of cognitive progression markers. By avoiding highly memorable images, cognitive tests could be made more time efficient and more sensitive. Understanding which stimulus features improve or impair memorability could provide insights into the cognitive processes that are impaired.
Furthermore, knowledge about memorability could aid in the design of memorable environments or allow clinicians to focus on aiding memory for forgettable items.
In the present study, we analyzed the performance of 394 individuals, including those with SCD, MCI, and healthy controls (HCs), on a visual recognition memory test in which each participant had to memorize a randomly selected subset of 88 photographs from a pool of 835.
This randomization afforded us the possibility to assess memorability unconfounded by systematic effects of stimulus-selection or stimulus-order effects. First, we find significant similarities across groups in the images they remember and forget, and similarities to a convolutional neural network (CNN) trained on memorability, allowing the precise prediction of memory performance for each group.
Second, we find a separate set of images that can reliably differentiate groups, with meaningful implications for diagnosis. Finally, using a large-scale online experiment to score the images, we analyze what image features might lead to the memorability and diagnosticity of different images.
Discussion
Although individuals with SCD and MCI have decreased memory performance in comparison to HC, there is a considerable overlap in the images that they remember and forget. Thus, there are images that are highly memorable and forgettable to everyone regardless of diagnosis.
These consistencies in memorability exist not only between impaired memory groups and healthy controls, where consistencies in memorability are already well-established for controls [1], [2], but also within the SCD and MCI groups themselves.
Our questionnaire-based assessment of image attributes revealed that this common memorability is not related to esthetics or spaciousness, but to being manmade scenes that contain more objects, and are subjectively more memorable and interesting.
Although previous work has reported that ratings of interestingness, subjective memorability, and esthetics are ultimately not predictive of scene memorability at a fine-grained scale for healthy populations [7], such attributes may be important for guiding the selection of images that are broadly memorable across population types. We also find that memorable images are not necessarily the most visually distinctive, as determined by a CNN trained on image classification.
In addition, we show that a publicly available convolutional neural network (MemNet [6]) trained to predict image memorability aligns with performance of HC as well as those with SCD and marginally with MCI.
This raises the possibility that computational methods may guide the selection of images for diagnostic or therapeutic tools on the basis of memorability. Such tools may assist in creating or adapting environments to ease memory burdens on patients by avoiding low memorability items, or focusing strategies on rehearsing particularly forgettable information.
Although memorability is generally consistent across HC, SCD, and MCI groups, we have also identified a specific set of images that significantly differ between groups. Namely, we find that there are images that are highly memorable to HC, yet highly forgettable to MCI and SCD individuals, and a certain subset of these images can be used to best determine if an individual is likely to be healthy or have MCI or SCD.
The images generalize across impairments; images that differentiate MCI also successfully differentiate SCD, indicating that SCD may show similar cognitive impairments to those developed in MCI. This image set results in as much as a 10% improvement in diagnostic performance in comparison to a poorly chosen set of images (e.g., images memorable to MCI but forgettable to healthy controls).
Furthermore, this optimized image set reaches peak diagnostic performance with as few as 18.3 images seen per participant, classifying as well as the original set with 88 images per participant.
This means that individuals with MCI or SCD can be identified with higher certainty, and in a quicker, easier test. In terms of content, these diagnostic images tended to be manmade, indoor scenes that contained people.
However, in contrast to memorable images, they tended to be less esthetic, be less interesting, and seem subjectively less memorable. Scenes containing people tend to be the most memorable [12]; however, it is perhaps the combination of memorable image content (e.g., people, manmade objects) yet lack of memorable qualities (e.g., interestingness, esthetics) that causes these images to be remembered by healthy controls but forgotten by SCD and MCI individuals.
Functional neuroimaging work with healthy individuals has found that viewing memorable images results in automatic, stereotyped activity patterns in the visual cortex and medial temporal lobe [8], [9].
In future work, investigating the neural fate of memorable and forgettable images in older individuals and those with SCD or MCI may aid in understanding how patients may differentially process images at different processing stages of perception and memory encoding.
In the DELCODE study, we have indeed obtained fMRI data alongside the behavioral data reported here [15] and will be able to address this question in the future.
A related question is how Alzheimer’s pathology is related to memorability. For instance, we have previously shown that increasing levels of CSF total tau are related to decreasing novelty responses in the amygdala and the hippocampus [15].
These functional consequences of tau pathology could influence memorability patterns in MCI or SCD. Indeed, activity in medial temporal lobe regions shows early and automatic sensitivity to the memorability of an image in healthy individuals [8].
Furthermore, older adults at risk for MCI first show volume decrease in the entorhinal cortex, resulting in impairments in object location memory [23], [24] and object discrimination [25]. The diagnostic images, with their higher scene complexity and several manmade objects, may be most affected by early object processing deficits.
Image diagnosticity as calculated in this study could also be related to the biomarker status of individuals, a possibility that we will be able to address in the future with larger sample sizes. It will also be paramount to better understand the visual, semantic, and statistical features of an image that drive it to be forgettable, memorable, or diagnostic.
Several studies are working to examine memorability with more varied image sets, in a variety of experimental image contexts, and using new computational methods ([26] for a review).
In addition, understanding the content that makes an image most sensitive to differences between groups will allow for better identification of early impairments. Using fine-grained confidence rating scales or an information-dense metric of recollection (such as drawing [27]) may provide a more nuanced understanding of the memory for these images.
While the current work uses a memorability CNN trained on healthy participant memory data to predict participant memory, as larger-scale data from individuals with SCD, MCI, and Alzheimer’s disease are collected, a CNN could learn to identify images that would be particularly effective in diagnosis.
Finally, although the present study does not find consistent diagnostic ability in images remembered by impaired individuals and forgotten by healthy controls, this set of images may be particularly interesting to investigate in future work.
In sum, we show the importance of images themselves in predicting what memory-impaired individuals are likely to remember and differentiating them from healthy individuals. Such insights will have a meaningful impact in how we design cognitive assessment tools and tests for early diagnosis of memory impairments and in understanding how and why we process and remember certain images over others in our complex, visual world.
Source:
University of Chicago
