2019
Fiser J., Koblinger Á. & Arató J. (2019) Uncertainty-based adjustment of internal model during perceptual sequential decision-making. SNF 2019 [Abstract]
Abstract: Repeated perceptual decision-making is typically investigated under the tacit assumption that each decision is an independent process or, at most, it is influenced by a few decisions made prior to it. We investigated human sequential 2-AFC decision-making under the condition, when more than one aspect of the context could vary during the experiment: both the level of noise added to the stimulus and the cumulative base rate of appearance (how often A vs. B appeared) followed various predefined patterns. In seven experiments, we established that long-term patterns in the context had very significant effects on human decisions. Despite being asked about only the identity of the present stimulus, participants’ decisions strongly reflected summary statistics of noise and base rates collected dozens to hundreds of trials before. In addition, these effects could not be described simply as cumulative statistics of earlier trials: for example, a significant step change in base rate (a change point) could induce the same effect as a prolonged shift, while a gradual change did not induce any effect. As standard decision making models cannot explain these results, we developed a hierarchical Bayesian model that simultaneously represented the priors over the base rates and a potentially non-uniform noise model over the different stimulus identities. Based on simulations with the model, we conducted additional experiments and found that when a change occurred in the context that could be captured equally well by adjusting one or another aspects of the model, humans chose adjusting the variable that was less reliable as defined by variability in the preceding extended set of trials. In general, regardless of the simplicity of a perceptual decision-making task, humans automatically develop a complex internal model, and in the light of a detected change, they adaptively alter the component of this model that is implicitly judged to be the least reliable one.
Koblinger Á., Zoltowski D., Fiser J. & Lengyel M. (2019) Noise or signal? Psychophysical evidence for the role of sensory variability. COSYNE 2019, Lisbon, Portugal [Abstract]
Stimulus-independent fluctuations in the responses of sensory neurons are traditionally considered as mere noise, and thus a source of perceptual ambiguity. In contrast, sampling-based models of perceptual inference suggest that the magnitude of this intrinsic variability acts as a signal: it conveys information about the uncertainty in low-level perceptual estimates. In both cases, to improve accuracy, downstream areas need to average sensory responses over time, as in classical models of evidence accumulation. However, due to the different roles that upstream sensory variability plays under the “noise” and “signal” hypotheses, the uncertainty about this average behaves in fundamentally different ways in them: it is respectively related to the standard error or the standard deviation of responses. In order to compare these hypotheses, we used a modified orientation estimation paradigm in which, on every trial, subjects simultaneously reported their best estimate of one of several briefly viewed, static line segments and their confidence about this estimate. We varied the difficulty of trials by changing the number of line segments, their contrast level, and the presentation time of the display. In general, we found that subjects’ confidence predicted their accuracy even when controlling for these experimentally manipulated stimulus parameters. This indicated that subjects had a well-calibrated trial-by-trial subjective measure of their uncertainty and did not only rely on extrinsic stimulus parameters to gauge the difficulty of a trial. Critically, while both models could account for changes in estimation performance with stimulus parameters, only the “signal” model predicted correctly the experimentally observed changes in confidence reports, and in the strength of correlation between confidence reports and actual accuracy. These results offer a new psychophysical window onto the role of sensory variability in perception and indicate that it conveys useful information about uncertainty.
Bernacchia A., Fiser J., Hennequin G. & Lengyel M. (2019) Adaptive erasure of spurious sequences in cortical circuits. COSYNE 2019, Lisbon, Portugal [Abstract]
The sequential activation of neurons, reflecting a previously experienced temporal sequence of stimuli, is be- lieved to be a hallmark of learning across cortical areas1, including the primary visual cortex2,3 (V1). While circuit mechanisms of sequence learning have been studied extensively4,5, the converse problem, that is equally important for robust performance, has so far received much less attention: how to avoid producing spurious sequential activity that does not reflect actual sequences in the input? Here, we developed a new measure of sequentiality for multivariate time series, and a theory that allowed us to predict the sequen- tiality of the output of a recurrent neural circuit. Our theory suggested that avoiding spurious sequential activity is non-trivial for neural circuits: e.g. even with a completely non-sequential input and perfectly sym- metric synaptic weights, the output of a neural circuit will still be sequential in general. We then show that the most celebrated principles of synaptic organization, those of Hebb and Dale, jointly act to effectively pre- vent spurious sequences. We tested the prediction that cortical circuits actively diminish sequential activity, in an experience-dependent way, in multielectrode recordings from awake ferret V1. We found that activity in response to natural stimuli, to which animals were continually adapted, was largely non-sequential. In contrast, when animals were shown entirely non-sequential artificial stimuli, to which they had not been adapted yet, neural activity was sequential at first, and then gradually became non-sequential within a few minutes of extended exposure. Furthermore, this difference between responses to natural and artificial stimuli was not present at eye opening but developed over several days. Our work identifies fundamental requirements for the reliable learning of temporal information, and reveals new functional roles for Dale’s principle and Hebbian experience-dependent plasticity in neural circuit self-organization.
Fiser J., Koblinger Á. & Arató J. (2019) Reliability-based arbitration between noise and event-based component of observers’ internal model during perceptual decision making. VSS 2019 [Abstract]
The effects of long-term history on sequentially performed perceptual decision making are typically investigated either under the simplest stationary condition or in the context of changing volatility of the event statistics defined by the generative process. We investigated the rules of human decision making in the more natural situation when changes in the external conditions could be explained away by multiple equally feasible adjustment of the internal model. In each of four experiments, observers performed 500 trials of 2AFC visual discrimination between two arbitrary shapes that could appear with different frequency across trials and were corrupted by various amount of Gaussian noise in each trial. Trials were split to practice and test, where at the transition between the two, the appearance probability of the shapes (AP) changed either abruptly or gradually, their relative noise characteristics (NOISE) were altered, and feedback stopped. Using hierarchical Bayesian modeling, we showed that in this setup, the same perceptual experience can be explained by assuming a change in either AP or NOISE, but the two alternatives induce opposite long-term biases and consequently, different behavior under uncertain conditions. Interestingly, we found that observers strongly preferred one of the two alternatives. However, by manipulating the nature of the AP and the NOISE transition, and the volatility of AP during training, observers’ behavioral biases and hence their implicit choice of explaining the situation changed toward the other alternative as predicted by the model based on the newly introduced uncertainty. This suggests that similarly to arbitration between habitual and model-based explicit learning, humans adjust their implicit internal model during perceptual decision making based on the reliability of the various components, which reliability is assessed across detected change points during the sequence of events.
Szabó T., Avargués-Weber A., Finke V., Nagy M., Dyer A. & Fiser J. (2019) Increasingly complex internal visual representations in honeybees, human infants and adults. VSS 2019 [Abstract]
Although some animals such as honeybees (Apis mellifera) are excellent visual learners, little is known about their spontaneously emerging internal representations of the visual environment. We investigated whether learning mechanisms and resulting internal representations are similar across different species by using the same modified visual statistical learning paradigm in honeybees and humans. Observers performed an unrelated discrimination task while being exposed to complex visual stimuli consisting of simple shapes with varying underlying statistical structures. Familiarity tests was used for assessing the emergent internal representation in three conditions exploiting whether each of three different statistics (single shape frequencies, co-occurrence probabilities and conditional probability between neighboring shapes) were sufficient for solving the familiarity task.
We found an increasingly complex representation of the visual environment as we moved from honeybees to human infant and to adults. Honeybees automatically learned the joint probabilities of the shapes after extended familiarization, but didn’t show sensitivity to the conditional probabilities and they didn’t learn concurrently the single-element frequencies. As we know from previous studies, infants implicitly learn joint- and conditional probabilities, but they aren’t sensitive to concurrent element frequencies either. Adult results in this study were in line with previous results showing that they spontaneously acquired all three statistics.
We found that these results could be reproduced by a progression of models: while honeybee behavior could be captured by a learning method based on a simple counting strategy, humans learned differently. Replicating infant’s behavior required a probabilistic chunk learner algorithm. The same model could also replicate the adult behavior, but only if it was further extended by co-representation of higher order chunk and low-level element representations.
In conclusion, we’ve found a progression of increasingly complex visual learning mechanisms that were necessary to account for the differences in the honeybee, human infant- and adult behavioral results.
2018
Avarguès-Weber A., Finke V., Nagy M., Szabó T., Dyer A. & Fiser J. (2018) Visual Statistical Learning in Honeybees. ECVP 2018 [Abstract]
The ability of developing complex internal representations of the visual environment is crucial to the emergence of humans’ higher cognitive functions. Yet it is an open question whether there is any fundamental difference in how humans and other good visual learner species naturally encode aspects of novel visual scenes. We investigated how honeybees encode instinctively various statistical properties of different visual scenes presented in sequence. While after limited exposure, bees became sensitive to statistics of only elemental features (e.g. frequency of A) of the scenes, with more experience, they shifted to relying on co-occurrence frequencies of elements (frequency of AB) and lost their sensitivity to elemental frequencies. However the bees failed to show sensitivity to conditional probabilities (if A then B) contrarily to humans. Thus, humans’ intrinsic sensitivity to predictive information might be a fundamental prerequisite of developing higher cognitive abilities.
Bex P., Christensen JH. & Fiser J. (2018) Optimal variance encoding of contours in naturalistic images. ECVP 2018 [Abstract]
We investigated feature ensemble encoding at the lowest level of visual processing by focusing on contour encoding in natural images. In such images, the mean contour is not a single value, but it varies locally with spatial position, and variability of the contour can be quantified by the noisiness of the contour segments. We used a novel image decomposition/recomposition method, three different classes of images (circular patterns, object and fractal images), and two types of noise (orientation and position noise) to generate stimuli for a 2-AFC pedestal noise discrimination task. We found that humans readily encoded variability of contour ensembles, this encoding systematically varied with image classes, and it was distinctively different for orientation versus position noise despite participants not being able to reliably distinguish between the two types of noise. Moreover, JND obtained with mixed orientation and position noise followed the optimal maximum likelihood estimate, supporting a probabilitic coding of contours in humans.
Lengyel G. & Fiser J. (2018) Can task irrelevant statistical structure enhance perceptual learning? ECVP 2018 [Abstract]
Statistical learning (ability to extract and store new structures) and perceptual learning (improve visual discrimination abilities) traditionally thought to deal with separate tasks at different levels of visual processing. To test this conviction, we investigated whether perceptual learning can be enhanced by the presence of a task irrelevant statistical structure. We trained two groups (N=16) of observers for 5-days to perform an orientation discrimination task. For one group, the background color of the scene changed across trials according to a fixed sequence, for the other, it changed randomly throughout the training. Overall, the fix group achieved a larger reduction in discrimination thresholds than the random group. Furthermore, there was a marked difference in performance of the two groups with different context. This suggests that task irrelevant statistical structure during perceptual learning is automatically and implicitly built in the developing internal representation.
Reguly H., Nagy M., Márkus B. & Fiser J. (2018) Prior experience of stimulus co-occurrence increases sensitivity to visual temporal asynchrony. ECVP 2018 [Abstract]
The temporal relationship between sensory events plays a crucial role in establishing causal link between them, for example inferring a common cause. In the current study, we manipulated the probability of co-occurrence of various visual stimuli pairs to see whether this manipulation would affect participants’ ability to separate the two elements of a pair in time, when presented asynchronously. We used a simultaneity judgment task, with a learning phase, in which participants (N=14) saw synchronously disappearing shape-pairs, and a test phase, in which three types of pairs (learned, newly combined, novel) were presented, while the asynchrony between the disappearance of the elements was manipulated. Contrary to earlier results with cross-modal stimuli, a lower proportion of simultaneity judgments, as quantified by shorted temporal binding windows was reported for the learned pairs than for the newly combined or novel visual pairs indicating an increased probability of unisensory binding.
Nagy M., Reguly H., Márkus B. & Fiser J. (2018) Effect of unceartainty in audio-visual cross-modal statistical learning. ECVP 2018 [Abstract]
We investigated visuo-auditory statistical learning by using four visual shape and four auditory sound pairs and creating strong and weak cross-modal quadruples through manipulating how reliably a visual and an auditory pair occurred together across a large number of audio-visual scenes. In Exp 1, only the weak and strong quads were used, while in Exp 2 additional individual shapes and sounds were mixed in to the same cross-modal structures. After passive exposure to such scenes, participants were tested in three familiarity tests: (T1) visual or auditory pairs against pairs of randomly combined elements unimodally, (T2) strong cross-modal quads against weak ones, and (T3) visual or auditory pairs from the strong and weak quads against each other, unimodally. Without noise (Exp 1), participants learned all structures, but performed at chance in T3. In Exp 2, while T1 auditory was at chance, in the auditory T3, participants preferred strong pairs, showing a strong cross-modal boost.
Fiser J. (2018) Integrated learning across different levels of statistical structures in orientation discrimination task. International Workshop on Perceptual Learning, French Polynesia [Abstract]
We explored the interaction between perceptual learning and statistical learning, two domains of sensory learning that are traditionally investigated separately. Using a standard perceptual learning protocol, we trained observers to improve their sensitivity to orientation of Gabor patches while differentially manipulating task irrelevant context of the training, such as the background color of the training scenes. Overall, we found that irrelevant context not only strongly influenced observed perceptual learning performance, but it also induced highly specific effects in the post-training test determined by the statistical structure of the context modulation. Our results suggest that the task irrelevant statistical structure present in perceptual tasks is automatically and implicitly built in the developing internal representation during learning. Thus perceptual and statistical learning processes are strongly related and create an integrated and complex internal representation even in the simplest perceptual learning tasks.
Fiser J. & Lengyel G. (2018) Task irrelevant statistical regularities modulate perceptual learning in orientation discrimination task. VSS 2018, Journal of Vision 18 (10), 261-261 [Abstract]
Perceptual learning is defined as the ability to improve one’s performance in basic discrimination tasks via extended practice. Is this process influenced by statistical regularities in the scene that have no relation to the discrimination task at hand? Using a 5-day standard perceptual training protocol, we trained two groups of observers to perform an orientation discrimination task with Gabor patches. For one group, the background color of the scene changed across trials according to a fixed sequence, while for the other group, the background color changed randomly throughout the training. Baseline and post training discrimination thresholds were assessed in three conditions: (1) with randomly changing background colors, (2) with backgrounds following the fixed color sequence, and (3) with gray background.
Overall, the group trained with fixed color sequence learnt more (had a larger reduction in orientation threshold by the end of the fifth day) than the group trained with randomly changing colors. Furthermore, while there was no difference across the baseline thresholds in the three conditions before training, after training, observers in the fixed sequence group showed the lowest threshold with fixed color sequence of the background, while their thresholds with random, and gray backgrounds were equally worse (higher). In contrast, observers in the random sequence group showed the lowest threshold during post test with the randomly changing background, intermediate thresholds with gray background, and the worst thresholds with fixed color sequence.
Our results suggest that task irrelevant statistical structure in perceptual tasks is automatically and implicitly built in the developing internal representation during learning, and it can differentially affect the learning process. Moreover, altering such irrelevant context after learning has a highly specific effect on performance arguing for the emergence of a complex internal representation even in the simplest perceptual learning tasks.
Reguly H., Nagy M. & Fiser J. (2018) Complex interactions across modalities in audio-visual cross-modal statistical learning. VSS 2018, Journal of Vision 18 (10), 1132-1132 [Abstract]
Statistical learning (SL) within modalities is an area of intensive research, but much less attention has been focused on how SL works across different modalities apart from demonstrating that learning can benefit from information provided in more than one modalities. We investigated visuo-auditory SL using the standard arrangement of SL paradigms. Four visual and four auditory pairs were created from 8-8 abstract shapes and distinctive sounds, respectively. Visual pairs consisted of two shapes always appearing together in a fixed relation, audio pairs were defined by two sounds always being heard at the same time. Next, strong and weak cross-modal quadruples were defined as one visual pair always occurring together with a particular auditory pair (strong) or appearing with one of two possible auditory pairs (weak). Using additional individual shapes and sounds, a large number of cross-modal six-element scenes were created with one visual pair, a single shape, one sound pair and a single sound. Adult participants were exposed to a succession such cross-modal scenes without any explicit task instruction during familiarization, and then tested in three familiarity tests: (1) visual or auditory pairs against pairs of randomly combined elements unimodally, (2) strong cross-modal quads against weak ones, and (3) visual or auditory pairs from the strong and weak quads against each other, again unimodally. We arranged relative difficulties so that in Test 1, the visual pairs were highly favored against random pairs, while choosing the auditory pairs against random sound pairs was at chance. Surprisingly, this setup caused participants choosing the weak quads significantly more often as familiar constructs in Test 2, and preferring equally strongly both the visual and auditory strong pairs over the corresponding weak pairs in Test 3. We interpreted this complex interaction through probabilistic explaining away effects occurring within the participants’ emerging internal model.
Koblinger A., Arato J. & Fiser J. (2018) Complex adaptive internal model subserves perceptual sequential decision making. COSYNE 2018, Salt Lake City, UT [Abstract]
Despite recent findings of sequential effects in perceptual serial decision making (SDM) (Chopin & Mamassian 2012; Fischer & Whitney 2014), SDM is typically investigated under the assumption that the decisions in the sequence are independent or at most, are influenced by a few previous trials. We set out to identify the true underlying internal model of event statistics that drives decision in SDM by investigating and modeling a set of novel sequential 2AFC visual discrimination tasks by humans and rats. Participants solved the same decision task across trials, but experienced one shift in baseline appearance probabilities of noisy stimuli during the experiment. We found non-trivial interactions between short- and equally strong long-term effects guiding evidence accumulation and decisions in such SDM. These interactions could elicit paradoxical and long-lasting net serial effects, for example, a counterintuitive negative decision bias towards the recently less frequent element. Our findings cannot be explained by previous models of SDM that either assume a sequential integration of prior evidence, presume an implicit compensation of discrepancies between recent and long-term summary statistics, or adjust learning rates of those statistics at change points. To provide a normative explanation for the empirical data, we developed a hierarchical Bayesian model that could simultaneously represent the priors over the appearance frequencies and a potentially non-uniform noise model over the different stimulus identities. The results of simulations with the model suggest that humans are more disposed to readjust their noise model instead of updating their priors on appearance probabilities when they observe sudden shifts in the input statistics of stimuli. In general, regardless of the simplicity of the decision task, humans automatically utilize a complex internal model during SDM and adaptively alter various components of this model when detecting sudden changes in the conditions of the task.
Arato J., Koblinger A. & Fiser J. (2018) Uncertainty-based adjustment of internal model during perceptual sequential decision-making. SFN 2018 [Abstract]
Repeated perceptual decision-making is typically investigated under the tacit assumption that each decision is an independent process or, at most, it is influenced by a few decisions made prior to it. We investigated human sequential 2-AFC decision-making under the condition, when more than one aspect of the context could vary during the experiment: both the level of noise added to the stimulus and the cumulative base rate of appearance (how often A vs. B appeared) followed various predefined patterns. In seven experiments, we established that long-term patterns in the context had very significant effects on human decisions. Despite being asked about only the identity of the present stimulus, participants’ decisions strongly reflected summary statistics of noise and base rates collected dozens to hundreds of trials before. In addition, these effects could not be described simply as cumulative statistics of earlier trials: for example, a significant step change in base rate (a change point) could induce the same effect as a prolonged shift, while a gradual change did not induce any effect. As standard decision making models cannot explain these results, we developed a hierarchical Bayesian model that simultaneously represented the priors over the base rates and a potentially non-uniform noise model over the different stimulus identities. Based on simulations with the model, we conducted additional experiments and found that when a change occurred in the context that could be captured equally well by adjusting one or another aspects of the model, humans chose adjusting the variable that was less reliable as defined by variability in the preceding extended set of trials. In general, regardless of the simplicity of a perceptual decision-making task, humans automatically develop a complex internal model, and in the light of a detected change, they adaptively alter the component of this model that is implicitly judged to be the least reliable one.
Fiser J. (2018) Sampling: a probabilistic approach to cortical computation, learning, and development. The Probabilistic Brain, Durham UK [Abstract]
I will present a framework and a combined empirical-computational program that explores what computation and cortical neural representation could underlie our intelligent behaviour. I will start by giving a brief summary of the fundamental logic of the framework and the main results we obtained earlier in support of the framework. Next, I will focus on three specific topics of the framework in line with the main theme of the meeting. First, exploring the limits of generalization of human statistical learning, I will show that humans readily transfer object knowledge over sensory modalities: receiving diagnostic information only in the visual or only in the haptic modality, they automatically formulate abstract object representations that generalize in the other modality. Second, I report that when comparing humans and honeybees in the same visual statistical learning task, we found striking differences despite the fact that honeybees are known to be very good visual learners. While bees internal representation underwent a marked transformation from being based on simple elementary vs. complex joint combination of visual features, they systematically failed to extract predictive information automatically from the input the way that humans naturally do. Third, in a serial visual decision making task, humans known to be influenced not only by the momentary sensory input of the trial, but also by events in the preceding trials. However, we show that a major factor of this effect is due to a long-term internal model participants develop involuntarily, which more complex than a simple evidence integrator and therefore, it modulates human decision making in a way that cannot be captured by classical models. Together these results provide a firm support to the probabilistic framework of human learning.
2017
Fiser J., Lengyel G., Lengyel M. & Wolpert D. (2017) Emergence of object representations through generalization between visual and haptic statistics. Conference on Interdisciplinary Advances in Statistical Learning Bilbao, Spain [Abstract]
The emergence of the concept defining a discrete object in the brain is a fundamental yet poorly understood process. In two statistical learning experiments, we show that humans can form these abstract concepts via purely visual statistics or physical interactions, which nevertheless will generalize across these two modalities. Participants saw a sequence of visual scenes composed of multiple objects. Each object consisted of abstract shapes, where object identity was only defined either by the shape co-occurrences across scenes (Exp. 1), or by the physical effort required to pull the scene apart (Exp. 2). In Experiment 1, observers learned the visual statistical contingencies across the scenes (measured with visual familiarity test), and this knowledge also generalized to their judgments as to how they would pull apart novel scenes (in pulling-apart-object test). In Experiment 2, participants learned haptic statistics across scenes: in the pulling-apart-object test they preferred easier pulling directions as defined by underlying object boundaries. Moreover, this haptic learning also biased participants' judgments in the purely visual familiarity test. Thus, objects can be extracted solely based on visual or haptic statistics while still retaining an integrated quality that allows generalization across modalities, which is a hallmark of object-like representations.
Fiser J., Lengyel G. & Nagy M. (2017) Visual statistical learning provides scaffolding for emerging object representations. VSS 2017, Journal of Vision 17 (10), 39-39 [Abstract]
Although an abundance of studies demonstrated human’s abilities for visual statistical learning (VSL), much fewer studies focused on the consequences of VSL. Recent papers reported that attention is biased toward detected statistical regularities, but this observation was restricted to spatial locations and provided no functional interpretation of the phenomenon. We tested the idea that statistical regularities identified by VSL constrain subsequent visual processing by coercing further processing to be compatible with those regularities.
Our paradigm used the well-documented fact that within-object processing has an advantage over across-object processing. We combined the standard VSL paradigm with a visual search task in order to assess whether participants detect a target better within a statistical chunk than across chunks. Participants (N=11) viewed 4-4 alternating blocks of “observation” and “search” trials. In both blocks, complex multi-shape visual scenes were presented, which unbeknownst to the participants, were built from pairs of abstract shapes without any clear segmentation cues. Thus, the visual chunks (pairs of shapes) generating the scenes could only be extracted by tracking the statistical contingencies of shapes across scenes. During “observation”, participants just passively observed the visual scenes, while during “search”, they performed a 3-AFC task deciding whether T letters appearing in the middle of the shapes formed a horizontal or vertical pairs. Despite identical distance between the target letters, participants performed significantly better in trials in which targets appeared within a visual chunk than across two chunks or across a chunk and a single shape.
These results suggest that similar to object-defined within/between relations, statistical contingencies learned implicitly by VSL facilitate visual processing of elements that belong to the same statistical chunk. This similarity between the effects of true objects and statistical chunks support the notion that VSL has a central role in the emergence of internal object representations.
Bernacchia A., Fiser J., Hennequin G. & Lengyel M. (2017) Dale’s principle preserves sequentiality in neural circuits. COSYNE 2017, Salt Lake City, UT [Abstract]
Cortical circuits obey Dale’s principle: each neuron either excites or inhibits all its postsynaptic targets. There is no known principled justification for why this must be so; in fact, Dale’s principle is considered – if at all – a mere constraint in neural network models. Here we provide a novel rationale for Dale’s principle: networks with separate excitatory (E) and inhibitory (I) populations preserve the temporal relationships between their inputs, thus preventing spurious temporal correlations that could mislead spike timing-dependent plasticity (STDP). To show this, we study a recurrent firing rate network model with arbitrary nonlinear response functions. We assume that, in line with known Hebbian mechanisms at both excitatory and inhibitory synapses, the magnitudes of recurrent synaptic weights are proportional to the covariance of pre- and postsynaptic rates, while their sign is determined by the E/I identity of the presynaptic cell. We show that this connectivity pattern is both necessary and sufficient to ensure that neural circuit output will be non-sequential, if the input has no specific temporal ordering of its elements. Conversely, if there is some specific temporal ordering of inputs to different neurons, then the neural circuit output will also have sequences that reproduce those of the input. Our theory predicts the relative degree of sequentiality
of V1 responses to visual stimuli with different statistics, which we confirmed in cortical recordings: stimuli that are similar in lacking temporal ordering evoke responses that differ in their sequentiality, depending on whether V1 has been adapted to them. Our results suggest a novel and unexpected connection between the ubiquitous Dale’s principle and STDP, namely that Dale’s principle acts as a control mechanism to guarantee that STDP will act only on input-driven temporal sequences, rather than on internally generated ones.
2016
Arato J. & Fiser J. (2016) Spatio-temporal probability integration during visual discrimination.ECVP 2016, Perception 45, 178-179 [Abstract]
How specific statistical priors do we maintain? While it is known that past stimulus statistics influences later perceptual decisions, it is unclear how such effects would interact and influence decisions across different spatial locations. To test this, we used a visual discrimination paradigm with two target locations, and two abstract shapes that could appear in varying levels of Gaussian noise. The stimulus appearance probabilities shifted from the training block (always balanced) to the test block (unbalanced). In each trial, participants reported via fixation which of the two locations the stimulus would likely to appear, and using a touchpad, they chose which shape was presented. In the test of Exp. 1, object A was equally more frequent at both locations. In the test of Exp. 2, object A was more frequent at one location, but less frequent at the other location. In Exp. 1, strong priors emerged after the shift, as if participants were compensating to maintain the overall training probabilities. In contrast, in Exp. 2, participants followed the shift in appearance probabilities with their responses. This suggests that only simple priors emerge automatically, with more complex statistics, averaging might cause visual decisions to follow local probabilities more accurately.
Fiser J., Christensen J. & Bex P. (2016) Encoding basic visual attributes of naturalistic complex stimuli. ECVP 2016, Perception 45, 340-340 [Abstract]
Despite numerous studies with simple stimuli, little is known about how low-level feature information of complex images is represented. We examined sensitivity to the orientation and position of Gabor patches constituting stimuli from three classes according to their image type:Familiar natural objects, Unfamiliar fractal patterns, and Simple circular patterns. All images were generated by re-synthesizing an equal number of Gabor patches, hence equating all low-level statistics across image types, but retaining the higher-order configuration of the original images. Just noticeable differences of perturbations in either the orientation or position of the Gabor patches were measured by 2-AFC on varying pedestal. We found that while sensitivity patterns resembled those reported earlier with simple, isolated Gabor patches, sensitivity exhibited a systematic stimulus-class dependency, which could not be accounted for by current feedforward computational accounts of vision. Furthermore, by directly comparig the effect of orientation and position perturbations, we demonstrated that these attributes are encoded very differently despite similar visual appearance. We explain our results in a Bayesian framework that relies on experience-based perceptual priors of the expected local feature information, and speculate that orientation processing is dominated by within- hyper-column computations, while position processing is based on aggregating information across hyper-columns.
Fiser J., Arato J., Khani A. & Rainer G. (2016) Change-related weighting of statistical information in visual decision making. VSS 2016, Journal of Vision 16 (12), 574-574 [Abstract]
There is a complex interaction between short- and long-term statistics of earlier percepts in modulating perceptual decisions, yet this interaction is not well understood. We conducted experiments, in which we independently manipulated the appearance probabilities (APs) of abstract shapes over short and long time ranges, and also tested the effect of dynamically changing these probabilities. We found that, instead of simply being primed by earlier APs, subject made decisions so that they reduced the discrepancy between recent and earlier APs. Paradoxically, this leads to favor the less frequent recent event if it was more frequent in the long past. Moreover, this compensatory mechanism did not take effect when the difference in APs between long past and recent times was introduced gradually rather than abruptly. This leads to a paradox false and lasting negative compensation with uniform APs after a momentary abrupt shift followed by a gradual return. We replicated our key human finding with behaving rats, de onstrating that these effects do not rely on explicit reasoning. Thus instead simply following the rule of gradually collected event statistics, perceptual decision making is influenced by a complex process in which statistics are weighted by significance due to detected changes in the environment.
Jellinek S. & Fiser J. (2016) Evidence for automatic generative learning in humans. ECVP 2016, Perception 45, 274-274 [Abstract]
It is commonly assumed that humans learn generative or discriminative representations of the sensory input depending on task context (e.g. Hsu & Griffiths, 2010). Following our earlier findings (Orban et al., 2008), we propose that humans always form generative models based on the statistics of the input. To test this proposal, we investigated whether a learned internal model of the visual input would automatically incorporate task-irrelevant dimensions, and whether a generative model is formed even when the task requires only a simpler, discriminative representation. Participants (N=30) were presented with circle ensembles of varying mean size and standard deviation (SD). Their task was to estimate one of these parameters throughout the experiment, making the other dimension task-irrelevant. Unbeknown to the participants, the input formed two implicit categories across trials, one with small means and large SDs, and the second with large means and small SDs. Participants showed the same significant regression to the mean bias in either dimension both during the estimation task and after a categorization along the task-irrelevant dimension. Thus, even in a restricted or discriminative context, humans implicitly form a generative model of the distribution of the data, which model automatically influences their subsequent decisions.
Lengyel G. & Fiser J. (2016) The relation between initial thresholds, learning, and generalization in three perceptual learning paradigms. VSS 2016, Journal of Vision 16 (12), 1104-1104 [Abstract]
It has been suggested recently that the extent of learning in perceptual tasks can be predicted well from the initial performance according to a Weber-like law. However, the exact relationship between initial thresholds and the amount of learning and the link between learning and generalization still remained unclear. In three perceptual learning paradigms, we tested (1) how initial thresholds influence learning, (2) how the amount of learning influences generalization, and (3) how general these relationships are across different paradigms of perceptual learning. Using a 5-day training protocol in each paradigm, separate groups of observers were trained to discriminate around two different reference values: at 73 or 30% in contrast, at 45 or 15 degrees in orientation, and at 88 or 33 dots in magnitude discrimination task. In each paradigm, initial thresholds were significantly higher at the high reference (73% contrast, 45 degrees, and 88 dots) than those at the low reference (ps< 0.05). Within conditions in each paradigm, we found strong correlations between subjects' initial threshold and their percent improvement, (rs=0.63-0.82, ps< 0.01), but their relationship did not conform the proposed Weber-like law. In contrast, across conditions in each paradigm, both the average absolute improvement and the mean percent improvement confirmed the Weber-like relationship showing no difference in percent improvement between the conditions (Bayes Factors= 2-2.3). Finally, generalization of learning was proportional to the amount of learning (linear regression slopes= 0.74-0.92, r2s= 0.45-0.83). This pattern of result suggests that (1) individual variations in perceptual learning are not related to the learning process but to other factors such as motivation, (2) regardless of individual differences and testing paradigms, the amount of perceptual learning conditioned on visual attributes is proportional to the initial thresholds following a Weber-like law, and (3) generalization is linearly proportional to the amount of learning within the task.
Christensen, J. H., Bex, P.J., & Fiser, J. (2016) Encoding of basic visual attributes in naturalistic-like human vision Annual Meeting of the European Mathematical Psychology Group, Copenhagen, Denmark [Abstract]
Coding of visual attributes in human vision has traditionally been researched with simple stimuli (e.g., Gabor patches), presented either in isolation or in simple lattice-like arrangements. Consequently, little is known about how low-level feature information is represented with complex and naturalistic images under natural-like viewing conditions. In severalexperiments, we examined coding of the orientation and position of Gabor patches constituting stimuli from three classes according to their image type: 1) Familiar natural objects,2) Unfamiliar fractal patterns, and 3) Simple circular patterns. Naturalistic-like stimuli were generated by decomposing images from each class with a bank of Gabor wavelets, which were then re-synthesized using an equal number of oriented Gabor patches, equating all low-level statistics across image types, but retaining the higher-order configuration of the original images. Using a 2AFC paradigm, we measured the justnoticeable difference of perturbations to either the orientation or position of the Gabor patches. We found that for both orientation and position noise, sensitivity across increasing levels of pedestal noise resembled that found with simple, isolated Gabor patches (Morgan et al., 2008;Li et al., 2004) validating our stimuli and method. However, we also found that sensitivity systematically depended on the familiarity and the complexity of the stimulus class, which could not be accounted for by current computational accounts of encoding. As an alternative, we propose a Bayesian framework that utilizes an experience-based perceptual prior of the expected local orientations and positions. Furthermore, using this unified method, we could directly compare orientation and position coding and show that they are encoded differently. We speculate that sensory processing of orientation is dominated within hyper-columns, well approximated by an intrinsic hard threshold operating among orientation columns to discount noise, while sensory processing of position is based on aggregating information across hyper-columns.
Stanciu O., Lengyel M., Wolpert D. & Fiser J. (2016) On optimal estimation from correlated samples. ECVP 2016, Perception 45, 228-229 [Abstract]
Optimal estimation from correlated, as opposed to uncorrelated, samples requires different strategies. Given the ubiquity of temporal correlations in the visual environment, if humans are to make decisions efficiently, they should exploit information about the correlational structure of sensory samples. We investigated whether participants were sensitive to the correlation structure of sequential visual samples and whether they could flexibly adapt to this structure in order to approach optimality in the estimation of summary statistics. In each trial, participants saw a sequence of ten dots presented at different locations on the screen which were either highly correlated (r = 0.7) or uncorrelated (in two separate blocks of 260 trials), and were asked to provide an estimate of the mean location of the dots. In the high correlation block, participants showed a trend towards overweighting the first and last samples of the sequence, in accordance with the optimal strategy given correlated data. In contrast, when exposed to uncorrelated inputs, the weights that participants assigned to each sample did not differ significantly from the optimal uniform allocation. Thus, it appears that humans are sensitive to the correlational structure of the data and can flexibly adapt to it so that their performance approximates optimality.
2015
Christensen JH., Lengyel M. & Fiser J. (2015) The balance between Evidence Integration and Probabilistic Sampling in human during perceptual decision-making behavior with noisy dynamic stimuli. COSYNE 2015, Salt Lake City, UT [Abstract]
Probabilistic models of perception posit that subjective uncertainty related to any perceptual decision is represented in the cortex via probability distributions that encode features in a task-relevant, distributed manner (e.g. Probabilistic Sampling PS, Fiser et. al 2010). According to PS, to achieve any decision, this posterior distribution needs to be sampled through time. However, traditional evidence integration (EI) models also assume sequential integration of external sensory information over time, and they can predict the accuracy in such a task. Which process shapes the trial-by-trial time course of human behavior? In a series of human behavioral experiments, we found that both processes are present in the time-course of a perceptual judgment and that their mutual influence on behavior is flexible.
We used an estimation-based variant of the classical random dot motion (RDM) task, where in each trial, participants (N=14) reported their best estimate of stimulus direction and their subjective uncertainty about their decision simultaneously. The objective uncertainty for each trial was chosen locally from two coherence values (30% vs. 55%), and globally across trials by either blocking or randomly interleaving the trial-by-trial coherence values. Stimuli were presented for varying length of time the RDM for durations between 100ms and 1.5sec.
Confirming our analytical derivations, we found a significant and positive overall correlation between error and subjective uncertainty beyond 300-500 msec as a function of time in all participants. As such positive correlations are the hallmark of PS and cannot be caused by EI, these results indicate that, indeed, probabilistic inference processes dominate the latter part of decision making. Specifically, by splitting up trials in RDM duration, we found a marked decrease in both error-uncertainty correlation and absolute error within the first 300-500ms, indicating EI, followed by a significant increase throughout the remaining time, indicating PS. Importantly, the transition between these segments shifted as a function of both local and global objective uncertainty.
Thus, we propose that in perceptual decision making based on dynamic stimuli with limited information perceptual process is not simply noisy evidence integration, but rather a probabilistic inference process. Moreover, this process in perceptual judgments follow a pattern that is related to that found during learning in an uncertain environment: When the global uncertainty is high, PS begins dominating earlier in time if local uncertainty is low compared to when local uncertainty is high. In contrast, when the global uncertainty is low – PS takes over at the same time regardless of the level of local signal uncertainty.
Fiser J., Koblinger A. & Lengyel M. (2015) Modeling information integration in sequential visual decision-making. VSS 2015, Journal of vision 15 (12), 90-90 [Abstract]
Current models of human visual decision making based on sequentially provided samples posit that people make their next decision by unconsciously compensating the statistical discrepancy between measures collected in the long past and those collected very recently. While this proposal is compelling, it does not qualify as a rigorous model of human decision making. In addition, recent empirical evidence suggests that human visual decision making not only balances long- and short-term summary statistics of sequences, but in parallel, it also encodes salient features, such as repetitions, and in addition, it relies on a generic assumption of non-discriminative flat prior of events in the environment. In this study, we developed a normative model that captures these characteristics. Specifically, we built a constrained Bayesian ideal observer with a generative model having features as follows. First, data is generated randomly but not necessarily independently depending on its parameter selection. Second, the system has a memory capacity denoted by a small window size = t, and a world representation denoted by a large window size = T, the latter reflecting the observer’s belief of the volatility of the world, i.e. the extent to which changes should be represented. Third, events can be described with pi appearance probability, which is not constant in time but changes according to a Markovian update, and it has an initial strong peak at 50%. Fourth, observations are noisy so that the observer can collect only limited amount of information (𝛾i) from each sample image. We implemented the above model and training on human data, we determined the optimal parameters for T, t, and inferred the evolving pi for each subject. Our model could capture the behavior of human observers, for example their deviation from binomial distribution based on T, and t, and the negative correlation between recent and past decisions.
Arató J. & Fiser J. (2015) Information integration in sequential visual decision-making. VSS 2015, Journal of Vision 15 (12), 94-94 [Abstract]
Although it is widely accepted that both summary statistics and salient patterns affect human decision making based on temporally varying visual input, the relative contributions and the exact nature of how these aspects determine human judgment are unclear and controversial, often discussed under the labels of priming, adaptation, or serial effects. To tease apart the role of the various factors influencing such decision making tasks, we conducted a series of 7 adult behavioral experiments. We asked subjects to perform a 2-AFC task of judging which of two possible visual shapes appeared on the screen in a randomly ordered sequence while we varied the long- and short-term probability of appearance, the level of Gaussian pixel noise added to the stimulus, and the ratio of repetitions vs. alternations. We found that the quality of the stimulus reliably and systematically influenced the strength of influence by each factor. However, instead of a simple interpolation between long-term probabilities and veridical choice, different pairings of short- and long-term appearance probabilities produced various characteristic under- and over-shootings in choice performances ruling out earlier models proposed for explaining human behavior. Independent control of base probabilities and repetition/alternation revealed that despite the two characteristics being correlated in general, repetition/alternation is a factor independently influencing human judgment. In addition, we found that human performance measured by correct answers and by reaction times (RT) yield opposing results under some conditions indicating that RT measures tap into motor rather than cognitive components of sequence coding. Our results can be captured by a model of human visual decision making that not only balances long- and short-term summary statistics of sequences, but in parallel also encodes salient features, such as repetitions, and in addition, relies on a generic assumption of non-discriminative flat prior of events in the environment.
Hofer M., Maloney L. & Fiser J. (2015) Detecting structure in visual sequences. VSS 2015, Journal of vision 15 (12), 333-333 [Abstract]
We investigated how well people discriminate between different statistical structures in letter sequences. Specifically, we asked to what extent do people rely on feature-based aspects vs. lower-level statistics of the input when it was generated by simple or by more hierarchical processes.
Using two symbols, we generated twelve-element sequences according to one of three different generative processes: a biased coin toss, a two-state Markov process, and a hierarchical Markov process, in which the states of the higher order model determine the parameters of the lower order model. Subjects performed sequence discrimination in a 2-AFC task. In each test trial they had to decide whether two sequences originated from the same process or from different ones. We analyzed stimulus properties of the three sets of strings and trained a machine learning algorithm to discriminate between the stimulus classes based either on the identity of the elements in the strings or by a feature vector derived for each string, which used 13 of the most common features split evenly between summary statistics (mean, variance, etc.) and feature-based descriptors (repetitions, alternations). The learning algorithm and subjects were trained and tested on the same sequences to identify the most significant features used by the machine and humans, and to compare the two rankings.
Not only there was a significant agreement between the ranks of features for machine and humans, but both used a mixture of feature-based and statistical descriptors. The two most important features for humans were ratio between relative frequencies of symbols and existence of repeating triples. We also found a consistent asymmetry between repetition and alternation as repetitions of length three or higher were consistently ranked higher than alternations of the same length. We found that, without further help, humans did not take into account the complexity of the generative processes.
Jellinek S., Maloney L. & Fiser J. (2015) Evidence of probabilistic representation in assessing visual summary statistics. VSS 2015, Journal of vision 15 (12), 946-946 [Abstract]
People rapidly and precisely extract summary statistics (e.g. mean and variance) of visually presented ensembles, and such statistics represent an essential part of their internal representation reflecting their environment. Recently, we reported that humans’ behavior in perceptual decision making task complies with the proposal that they handle simple visual attributes in a sampling-based probabilistic manner (Popovic et al. Cosyne 2013, VSS 2013; Fiser et al. ECVP 2013; Christensen et al. VSS 2014, ECVP 2014). In this study, we tested whether such probabilistic representations also may underlie the assessment of visual summary statistics.
In each trial, subjects saw a group of circles (N=2...10, randomly chosen) of varying sizes and had to estimate either the mean or variance of sizes of the ensemble or the size of one individual circle from the group specified after the figure with the group was taken off the screen. In addition, they also reported their subjective confidence about their decisions on a trial-by-trial basis. Trials from the three tasks were tested either intermixed or by presenting them in blocks, separately. Stimuli were also presented at nine different durations (50, 75, 100, 133, 167, 200, 300, 400, or 600 msec).
In accordance with previous results, participants could estimate correctly the mean, the variance and size of an element within the ensembles. Interestingly, mean estimation improved significantly as a function of the number of circles in the display (p < 0.001). More importantly, we found and increasing correlation between error and uncertainty as a function of presentation time, which is the hallmark of sampling-based probabilistic representation. Thus such probabilistic representation is not used exclusively for the simplest visual attributes, such as orientation, speed of small dots, but they also apply to representing more abstract kind of summary statistics.
Ledley J. & Fiser J. (2015) Generalizing visual rules in 3-dimensional perceptual space. Conference on Interdisciplinary Advances in Statistical Learning, San Sebastian, Spain [Abstract]
Although visual rule learning has been viewed as a potential candidate of how humans develop higher-order internal representations, typical rule-learning experiments have focused on the ability to extract abstract rules defined by element repetition rather than regularities based on feature dimensions. To link rule learning to the natural task of visual recognition, we investigated learning rules that were based on size regularities of visual objects perceived in a graphically generated three-dimensional layout. We found that adult subjects could extract the classical AAB type of rules based on size relations, and that the extracted rule was defined by the perceived 3-dimensional interpreted size of the objects rather than by the actual 2-dimensional extent of their images. Moreover, the extracted rule generalized not only to new displays with never-before-seen objects, but also to new 2-dimensional contexts where the original 3D perceptual constraints were not present at all. These results extend the generality of rule learning in vision supporting the view that the extracted rules are not purely semantic but incorporate mid-level perceptual information, yet at the same time, they are abstract enough to apply across wide range of contexts.
Arato J., Khani A., Rainer G. & Fiser J. (2015) Statistical determinants of sequential visual decision-making. ECVP 2015, Perception 44, 369-369 [Abstract]
Apart from the raw visual input, people’s perception of temporally varying ambiguous visual stimuli is strongly influenced by earlier and recent summary statistics of the sequence, by its repetition/alternation structures, and by the subject’s earlier decisions and internal biases. Surprisingly, neither a thorough exploration of these effects nor a framework relating those effects exist in the literature. To separate the main underlying factors, we ran a series of nine 2-AFC visual decision making experiments. Subjects identified serially appearing abstract shapes in varying level of Gaussian noise (uncertainty), appearance probabilities and repetition-alternation ratio. We found a) an orderly relationship between appearance probabilities on different time-scales, the ambiguity of stimuli and perceptual decisions; b) an independent repetition/alternation effect, and c) a separation of bias effects on RT and decision, suggesting that only the latter is appropriate for measuring cognitive effects. We confirmed our main human results with behaving rats making choices based on luminance between stimuli appearing at different locations. These findings are compatible with a probabilistic model of human and animal perceptual decision making, in which not only decisions are taken so that short-term summary statistics resemble long-term probabilities, but higher order salient structures of the stimulus sequence are also encoded.
Fiser J., Koblinger Á. & Arató J. (2015) The interplay between long-and short-term memory traces in sequential visual decision making. SFN 2015, Chicago, IL [Abstract]
Past experience strongly guides sensory processing and influences every perceptual decision. Yet, due to contradictory findings in the literature, the exact pattern of these effects is unclear and a convincing general computational framework underlying these effects is still missing. Even in the simplest version of the problem, making a forced choice between two hypotheses based on noisy sequential input, the field is divided over how basic statistics of the input (e.g. appearance frequencies) and various significant patterns (e.g. repetition) jointly determine the observer’s behavior.
We used the above model problem in 7 experiments to tease apart the relative contributions of each effect on human sequential decision making. Observers performed a 2-AFC decision making (“Which of the two shapes is seen?”), while we independently modulated the level of pixel-noise, the appearance frequency of the elements coming from the two classes at two different time scales, and the ratio of repetition/alternation in the sequence.
We found that the noise level of the stimulus systematically modulated the strength of each contributing factor to decision making. However, instead of a simple interpolation between long-term probabilities and veridical choice as it would be predicted by adaptation or priming, different pairings of short- and long-term appearance probabilities produced various characteristic under- and over-shootings in choice performances. This rules out a number of earlier models proposed for explaining human behavior in such tasks. We also found that human performance measured by correct answers and by reaction times yielded opposing results under some conditions indicating that RT measures tap into motor rather than cognitive components of sequence coding. By controlling the base-rate probabilities and repetitions/alternations independently, we also observed that despite the two measures being correlated in general, repetition/alternation is a factor independently influencing human judgment. To assess the generality of our findings, we run behavioral studies with adult rats asking them to choose between two full-field stimuli of different brightness. We found that rats replicated the striking results of humans, by choosing the frequent stimulus of recent past fewer times under high uncertainty after experience with particular long-term appearance statistics.
Through simulations, we confirmed that our results can be captured by a probabilistic model of human visual decision making that balances long- and short-term summary statistics of sequences, and in parallel, also encodes salient features, such as repetitions in the sequence.
Christensen JH., Bex PJ. & Fiser J. (2015) Prior implicit knowledge shapes human threshold for orientation noise. VSS 2015, Journal of vision 15 (9), 24-24 [Abstract]
Although orientation coding in the human visual system has been researched with simple stimuli, little is known about how orientation information is represented while viewing complex images. We show that, similar to findings with simple Gabor textures, the visual system involuntarily discounts orientation noise in a wide range of natural images, and that this discounting produces a dipper function in the sensitivity to orientation noise, with best sensitivity at intermediate levels of pedestal noise. However, the level of this discounting depends on the complexity and familiarity of the input image, resulting in an image-class-specific threshold that changes the shape and position of the dipper function according to image class. These findings do not fit a filter-based feed-forward view of orientation coding, but can be explained by a process that utilizes an experience-based perceptual prior of the expected local orientations and their noise. Thus, the visual system encodes orientation in a dynamic context by continuously combining sensory information with expectations derived from earlier experiences.
Fiser J. (2015) Factors that influence judging and guessing about probabilistic event sequences. SFX 2015, Pisa, Italy [Abstract]
Previous studies have reported several factors, including prior knowledge, past experience, immediately preceding events, and rate of event repetitions that influence humans’ ability to predict and perceive sequentially occurring probabilistic events. However, many of these factors are correlated and most earlier studies made little effort to disentangle their confounding effects. I will present a series of human behavioral experiments, in which we systematically inspected the separate and joint effects of these factors within a simple visual perceptual paradigm. We found that, rather than simply balancing past and present statistics, the best model describing human performance is probabilistic and it assumes a parallel working of several factors: a) reliance on prior statistical knowledge of the sequence as a function of stimulus uncertainty, b) a “regression to the mean” kind of effect that could reflect a general strategy of non-commitment, and c) an independent short-term repetition effect which influences performance asymmetrically.
2014
Fiser J. & White BL. (2014) Learning-based cross-modal suppression of ongoing activity in primary cortical areas of the awake rat. COSYNE 2014, Salt Lake City, UT [Abstract]
Ongoing activity is ubiquitous in the cortex and recently has been implied to play a significant functional role in shaping sensory-evoked responses by reflecting the momentary internal state of the brain together with its knowledge about the external world (Berkes et al., Science 2011). A wide variety of studies also reported that ongoing neural signal variability in many cortical areas gets reduced when a stimulus is presented (Churchland et al., Nat Neuro 2010), raising the possibility that this reduction is functionally linked to the decrease of uncertainty due to the stimulus onset (Orban et al., Cosyne 2011). This conjecture would imply that just as stimulus onset reduces uncertainty and hence neural response variability, ongoing signal variability should also decrease due to reduction of uncertainty when the animal can apply acquired long-term knowledge about the environment in a particular situation. To test this hypothesis, we recorded neural activity with multi-electrode arrays simultaneously from the primary visual and gustatory cortices of rats while they learned to associate visual cues with delayed water reinforcement. We found within-modality, cue-dependent suppression of variability of the evoked activity in the visual cortex in line with previous reports. However, we also found that, independent of firing rate changes, spike count variability in the absence of any sensory stimuli was significantly suppressed both during the delay periods in the primary visual cortex, and during the cue period in the ongoing activity of the gustatory cortex. Importantly, this suppression both within and across modalities occurred only in animals that learned the task. These findings demonstrate that not only domain-specific stimulus-evoked, but also experience-based, internally-generated cross-modal signals are capable of suppressing the variability of ongoing activity in the primary cortex supporting the proposal that this activity is not noise but rather it represents information about particular behaviorally-relevant conditions.
Haefner RM. & Fiser J. (2014) Good noise or bad noise? The role of correlated variability in a probabilistic inference framework. COSYNE 2014, Salt Lake City, UT [Abstract]
The responses of sensory neurons in cortex are variable, and this variability is often correlated [Cohen and Kohn, 2011]. While correlations were initially seen as primarily detrimental to the ability of neuronal populations to carry information about an external stimulus [Zohary et al., 1994], more recent studies have shown that they need not be information-limiting in populations of neurons with heterogenous tuning curves [Shamir and Sompolinsky, 2006, Ecker et al., 2011]. In general, information about some variable of interest, s, is only limited by correlations — sometimes called ’bad correlations’ (Pitkow et al., SfN 2013) — that are equivalent to correlations induced by external uncertainty in s. The greater the magnitude of these correlations, the less information about s can be represented by the population. We show in the context of a 2AFC task that correlations of the ’bad noise’ structure are induced by feedback connections in a sensory population of neurons involved in probabilistic inference. However, unlike in the traditional encoding/decoding framework, here their presence reflects the fact that the brain has learnt the task. In fact, perceptual learning will increase their magnitude such that stronger ’bad’ correlations are simply a side-effect of better psychophysical performance. We further show that increasing ’bad correlations’ entails a steeper relationship between choice probabilities (CP) and neurometric performance as has been observed empirically during perceptual learning [Law and Gold, 2008]. Finally, we derive the results of classic reverse correlation techniques as applied to a neural system performing probabilistic inference and relate them to the bottom-up and top-down information flow as predicted by the normative model. Interestingly, we find that despite the fact that CPs are primarily caused by the top-down influence of the decision on sensory responses, they can nonetheless be used to infer the influence of an individual neuron onto the subject’s decision.
Jellinek S. & Fiser J. (2014) Neural correlates of learning and uncertainty during the acquisition of novel categories. BCCCD 2014, Budapest, Hungary [Abstract]
In our study, we addressed the question of whether and how the learning process of a novel category and uncertainty in our knowledge about it is reflected in ERP responses and alpha-band suppression of the brain. To test these ideas, we built our experimental procedure on the oddball paradigm that is frequently used to investigate the nature of category representations in the human brain. Contrary to the majority of similar studies, we used continuous stimuli and recorded ERPs from the beginning of learning omitting any training before the actual data collection. Obtained results suggest that participants mapped correctly the proper statistical distribution of the input both in their implicit judgments and explicit decisions about the learnt boundary between the two acquired categories. Uncertainty followed this mapping as higher uncertainty reports were localized along the implicit category boundary. Neural signatures of the emerging internal representations allow us to track the formation of categories. Increased amplitude of P300 ERPs reflects the level of suddenness of an observed stimulus, and alpha-suppression corresponds to the learner's subjective confidence in their knowledge about certain exemplars of a category during learning. These results support the ideas that ERP and alpha-suppression are reliable signals of not only the already acquired structure of concepts, but they also allow for tracking the ongoing acquisition of categories.
Arató J. & Fiser J. (2014) Learning about the Structure of Probabilistic Visual Events. BCCCD 2014, Budapest, Hungary [Abstract]
There is increasing evidence suggesting, that people encode dynamic visual information probabilistically. However, the mechanism of this phenomena is unknown. Recently Kidd et al (2012) investigated, how predictability of varying visual stimuli influences attention and learning in infants. Their main finding was, that infants maintain attention longest for stimuli, that have intermediate predictability. Such a behavior could be explained by applying an optimal learning mechanism, where attention is allocated at the most informative stimuli. However, this prediction could not be explicitly tested by Kidd et al as their method did not have a separate measure of attention and learning. To explore this prediction, we investigated how people perceive and learn about probabilistic events, and tested how accurately their behavior could be captured by a probabilistic framework. In our set of experiments we measured how precisely people can estimate the probabilities of multiple, intertwined simple visual events. We found that increased variability, due to multiple shapes can lead to better estimation. This corroborates previous findings, that probabilistic processes are better captured by implicit than by explicit mechanisms. We also found a linear relationship between visual probabilities and participants' estimates. Moreover, the pattern of learning within and across blocks is well predicted by a probabilistic model, that includes the statistical structure of the input of the task. These results support the notion, that human learning of dynamic events is well captured by a framework that assumes probabilistic encoding.
Karuza EA., Emberson LL., Roser ME., Gazzaniga MS., Cole D., Aslin RN. & Fiser J. (2014) Dynamic shifts in connectivity between frontal, occipital, hippocampal and striatal regions characterize statistical learning of spatial patterns. VSS 2014, Journal of Vision 14 (10), 955-955 [Abstract]
Extensive behavioral evidence has revealed that humans automatically develop internal representations that are adapted to the temporal and spatial statistics of the environment. However, the neural systems underlying this statistical learning process are not fully understood. Recently, various neuroimaging methods have been employed to examine this topic, but these studies have focused exclusively on temporally ordered stimuli. Since spatial structure is a hallmark of object and scene perception in vision, the present functional magnetic resonance imaging (fMRI) study investigated the substrates and processes underlying complex spatial pattern learning. Neuroimaging data were obtained while 20 subjects passively viewed artificially created scenes with a pre-specified pair-based statistical structure. After three runs of exposure to 144 different 6-element scenes, subjects performed a yes/no task on base-pairs and cross-pairs. Using seed regions defined by relating magnitude of activation to this post-exposure behavioral learning performance, we examined changes in functional connectivity over the course of learning. In addition to a general increase in connectivity throughout exposure, we find a specific connectivity relationship between frontal, occipital, hippocampal and subcortical areas that was dynamically reconfigured as learning progressed. Specifically, we show that connectivity with frontal regions shifted from early visual areas to subcortical areas when comparing early and late phases of exposure. These results suggest that learning is not fully captured by a single, fixed “learning” network, but is reflected at least partially in dynamic shifts in connectivity across numerous cortical and subcortical areas.
Christensen JH., Lengyel M. & Fiser J. (2014) The temporal balance between evidence integration and probabilistic sampling in perceptual decision making. VSS 2014, Journal of Vision 14 (10), 836-836 [Abstract]
Models of evidence integration (EI) assume that the accumulation of external information alone is the dominant process during perceptual decision making until an overt response is made. In contrast, probabilistic sampling (PS) theories of the representation of uncertainty (Fiser at al. 2010) posit that time during perceptual decision making is primarily used for collecting samples from essentially static, internally represented distributions -- at least for briefly presented simple stimuli. While EI predicts a decreasing trend in the correlation between participants’ estimation error and uncertainty over a trial, PS predicts an increasing trend even long after error and uncertainty reach asymptotic values. The predictions of PS have recently been confirmed in experiments using static stimuli (Popovic et al. 2013). However, the precise relationship between EI and PS in the more general case of dynamic stimuli have remained unexplored. To dissect the contributions of EI and PS to perceptual decisions, we used a variant of the classical random dot motion task that required estimation (rather than discrimination judgment. In each trial, participants reported their best estimate for stimulus direction and their subjective uncertainty about it by the direction and length of a line drawn on a tablet. We controlled EI by varying the coherence of the signal (providing more or less evidence), and PS by varying the stimulus presentation time (allowing for the collection of more or less samples). In each participant, we found a marked decrease in error-uncertainty correlation in the first part of the trial, indicating EI, and a significant increase in the second part, indicating PS. Moreover, the transition between these segments shifted in accordance with the change in signal coherence. These results suggest that EI and PS during decision making work in parallel with EI taking the lead early but PS determining the later part of the process.
Arató J. & Fiser J. (2014) Short term and baseline effect in the estimation of probabilistic visual event sequences. VSS 2014,Journal of Vision 14 (10), 373-373 [Abstract]
To understand how people build probabilistic internal representations of their dynamic perceptual environment, it is essential to know how the statistical structures of event sequences are encoded in the brain. Previous attempts either characterized this coding by the structure of short-term repetition/alternations or while acknowledging the importance of long-term baseline probabilities, they failed to explore their effect by manipulating properly the baseline statistics.
We investigated how expectations about the probability of a visual event are affected by varying short-term and unbalanced baseline statistics. Participants (N=19) observed sequences of visual presence-absence events and reported about their beliefs by two means: by quickly pressing a key indicating whether or not an object appeared and by giving interspersed numerical estimates of the appearance probability of the event together with their confidence of their answer. Stimuli appeared at random with the baseline probabilities systematically manipulated throughout the experiment.
We found that reaction times (RTs) for visual events did not depend exclusively on short-term patterns but were reliably influenced by the baseline appearance probabilities independent of the local history. Error rates, RTs and explicit estimates were similarly influenced by the baseline: subjects were more accurate estimating the probability of very likely and very unlikely events. Furthermore, we found that subjects’ report of their confidence was systematically related to both the implicit and explicit accuracy measures. Finally, reaction times could be explained by a combined effect of short-term and baseline statistics of the observed events.
These results indicate that the perception of probabilistic visual events in a dynamic visual environment is influenced by short-term patterns as well as automatically extracted statistics acquired on the long run. Our findings lend support to proposals that explain behavioral changes in terms of relying on an internal probabilistic model rather than as a local adaptation mechanism.
Christensen JH., Lengyel M. & Fiser J. (2014) Factors determining the balance between evidence integration and probabilistic sampling in perceptual decision-making. ECVP 2014, Perception 43, 24-24 [Abstract]
We have reported that contrary to theories of Evidence Integration (EI), time in visual perceptual decision-making is needed not only for accumulating external sensory evidence, but also for collecting samples from static internally represented distributions as it is predicted by the theory of Probabilistic Sampling (PS) (Cosyne 2013; VSS 2013). However, the precise relationship between EI and PS in the general case of dynamic stimuli is still unknown. We used an estimation-based variant of the classical random dot motion task, where in each trial, participants reported their best estimates of stimulus direction and their subjective uncertainty about it.
Across trials, we varied the strength of the sensory evidence and the trial time, and across experiments, we varied the trial sequence volatility (intermixed versus blocked coherence levels). We found a marked decrease in error-uncertainty correlation within the first 300-500ms of the trial, indicating EI, followed by a significant increase throughout the trial, indicating PS. Importantly, the transition between these segments shifted as a function of signal coherence and volatility. Consequently, EI and PS during decision making work in parallel, with EI taking the lead early but PS dominating the later part of the probabilistic process.
Jellinek S. & Fiser J. (2014) Alpha ERD reflects learning and uncertainty during the acquisition of novel categories. ECVP 2014, Perception 43, 54-54 [Abstract]
How are the learning process of a novel category and uncertainty about the category reflected in the alpha-band suppression of the brain? We built an experimental procedure on the oddball paradigm frequently used to investigate the nature of category representations in humans, but contrary to similar studies, we used a continuous stimulus dimension and recorded EEG signals from the beginning of learning. Behaviorally, we found that participants mapped correctly the statistical distribution of the input in their implicit judgments with higher uncertainty reports along the implicit category boundary. Confirming earlier results, we found significant differences in alpha ERD elicited by frequent and infrequent stimuli [t(11)=2.66, p<.02].
Arató J. & Fiser J. (2014) Short term and baseline effect in the estimation of probabilistic visual event sequences. ECVP 2014, Perception 43, 54-54 [Abstract]
Humans’ reliance on expectations based on past experience to evaluate uncertain sensory events of their environment has been interpreted either as local adaptation or probabilistic implicit inference. However the exact interplay between immediate past and longer-term sensory experiences in influencing these expectations has not yet been experimentally explored. In a simple probabilistic visual 2-AFC task, we assessed how human judgments depended on unbalanced base-rate appearance statistics and the immediate history of possible events. Participants detected the appearance of visual shapes in blocked random visual sequences, where base-rate appearances were systematically manipulated throughout the blocks. Expectations were assessed implicitly by reaction times and explicitly by interspersed numerical estimates and confidence judgments. Implicit expectations were reliably influenced by both probabilities of the immediate past (F(3,14)=34.733,p <.000) and the base-rate (F(3,14)=28.674,p<.000) in an additive manner. In addition, implicit and explicit measures were consistent, as participants’ accuracy and confidence judgments in estimating the probability of likely and unlikely events followed the same pattern for the two measures. The results confirm previously untested assumptions about the interaction of base-rate statistics and short-term effects in forming visual expectations, and suggest that behavioral changes are based on internal probabilistic models not just on local adaptation mechanisms.
Jellinek S. & Fiser J. (2014) Dissociation of alpha ERD and P300 measures of categorization with continuously varying stimuli. SFN 2014, Washington, DC [Abstract]
EEG measures, especially alpha ERD and P3 ERP, has often been used as a tool for the investigation of the structure of categorical and conceptual representation of incoming stimuli. Studies of categorization that use alpha ERD and P3 signals typically have three characteristics: they use the oddball paradigm, the categories consist of discretely separable exemplars, and participants either have to categorize familiar objects or an intensive training precedes the test when EEG is recorded. We investigated whether alpha ERD and P3 are reliable indicators of category formation when instead of the structure of the already stable representation, the dynamics of learning of novel categories is investigated, and if so, whether they indicate the same underlying cognitive processes. To this end, we altered the usual oddball categorization paradigm in two aspects. First, participants had to categorize schematic human silhouettes ranging from slender to chunky in two categories (Thin and Fat), that is stimuli varied continuously along the parameter of width. Specifically, category exemplars were distributed as two Gaussians per category, one closer to the category boundary, the other at the extreme of the range. Second, we recorded neural responses with high-density (128-electrode) nets from the very beginning of the category formation. Apart from EEG recordings, we collected implicit and explicit behavioral responses (RT, mean and subjective uncertainty reports) to assess the correspondence between the two.
Confirming earlier results, we found significant differences in alpha ERD elicited by frequent and infrequent stimuli [t(11)=2.66, p<.02]. Importantly, learning was reflected in the difference of alpha-band suppression between neural responses collected early vs. late during the experiment, especially for exemplars at the extremes of stimulus range [t(11)=2.98, p = .012]. This difference was less articulate for stimuli sampled around the category boundary where confidence was also lower, suggesting that alpha ERD and subjective confidence in acquired knowledge are correlated. We found no significant differences in P3 amplitudes neither at the boundary nor at the extremes.
Thus, alpha-suppression is not only a reliable measure of the already acquired structure of concepts, but it also allows for tracking the ongoing acquisition of categories. In contrast, P3, seems to be a different measure of category representations, and it is highly sensitive to the discrete—continuous distinction of the stimuli.
Osik JJ., Roy A., Ritter NJ., Miller J., Wang Y., Fiser J. & Van Hooser SD. (2014) Spatiotemporally controlled optogenetic activation of developing ferret visual cortex. SFN 2014, Washington, DC [Abstract]
Realizing the full potential of optogenetic techniques will require concurrent development of light delivery technologies that improve spatial and temporal control of neuronal stimulation to facilitate increasingly informative electrophysiology recordings. Excitation (or inhibition) by light-activation of channelrhodopsin (ChR) and its many variants has quickly emerged as a preferred methodology in the neurosciences by reason of the cellular specificity achievable through conditional genetic expression. However, despite the variety of cell types targetable through the use of promoter-driven constructs, the spatial distribution of cells of any given classification can still be quite homogeneous, a fact that complicates the targeting of neural circuits that exhibit a very specific structure-function relationship, as in the circuits underlying functional maps. In studying the emergence of cortical maps in vivo where response variations are closely tied to spatial locations on the order of a few hundred microns in width, new light delivery techniques are needed to improve on the dispersive, on/off light control of implanted fiber optics that are commonly used to drive ChR stimulation. Two specific advances would enhance the utility of optogenetic stimulation in such applications: 1.) use of a controllable light source capable of projecting both stationary and dynamic light patterns; and 2.) improved spatial and temporal resolution while sustaining power sufficient to drive ChR activation at depth in the intact living brain.
We have developed an optogenetic stimulator based on epi-illumination microscope design and equipped with a high numerical aperture objective to stimulate and collect images through the same optical axis. The stimulator is coupled to an LCD-MLA projector light source to produce high-resolution spatiotemporally varying patterns in a very confined area. We evaluate the performance of this stimulator with respect to power, optical resolution and the quality of spatial and temporal control of single and multi-unit responses at a multichannel NeuroNexus electrode. We further assay the stimulator’s suitability for experimental paradigms calling for fine sequential control of horizontal cortical connections in vivo. Preliminary results indicate that retinotopically-constrained horizontal activation through ChR-mediated surface training in ferret V1 is sufficient to drive asymmetry in the orientation-tuning response to drifting gratings.
2013
Haefner RM., Berkes P. & Fiser J. (2013) Top-down influences on sensory processing during perceptual decision-making and attention. SFN 2013, San Diego, CA [Abstract]
Recent work has established the importance of top-down influences on early sensory processing. These influences are alternatively taken as representing task context, attention, expectation, working memory and motor commands (Gilbert & Li, 2013). At the same time, top-down influences have been recognized as essential for supporting probabilistic inference (Lee & Mumford, 2003). Here, we combine the latter idea with the recent hypothesis that the brain implements probabilistic inference using a neural sampling-based representation (Fiser et al 2010) and show that a normative account can indeed explain several disparate empirical observations on the effect of task context, expectation and attention on neuronal response gain and interneuronal response correlation. In particular, we show for a classic 2AFC task, that neural sampling can explain the task-dependent correlations seen by Cohen & Newsome (2008) and the choice probability time-course observed by Nienborg & Cumming (2010). Furthermore, we propose that top-down attention due to unequal rewards acts as a loss-calibration of the sampling approximation to the true posterior and show that this hypothesis entails an increase in response magnitude for neurons at the attended location as well as a decrease in noise correlations (Cohen & Maunsell, 2009, Mitchell & Reynold, 2009). Unlike previous accounts which assumed these effects to be the source for an improved psychophysical performance at the attended location, in our account, they are a consequence of probabilistic inference with changing constraints.
Popovic M., Van Hooser S. & Fiser J. (2013) Testing the functional significance of directional selectivity in the developing primary visual cortex SFN 2013, San Diego, CA [Abstract]
Directional selectivity (DS) is known to increase significantly in ferrets during two weeks after eye opening and it has been shown to strongly depend on visual experience (Li et al. 2008, Nature). Such selectivity to features of the input is traditionally assumed to indicate the functional maturity of the visual system, however, this assumption has not been confirmed directly. Recently, a measurement for assessing functional maturity has been put forth by Berkes et al. (2011, Science) based on the idea that a mature visual system is optimized for probabilistically encoding natural stimuli. They use Kulback-Leibler divergence (KL) to quantify dissimilarities in the statistical structure of multi-neuron firing patterns in V1 of awake ferrets acquired under different stimulus conditions. The distribution of firing patterns acquired in complete darkness (spontaneous activity) reflects the prior probability distribution over visual features that is unconstrained by visual input. According to the probabilistic approach, the more the distribution of spontaneous activity is similar to the distribution of average activity evoked by naturalistic stimuli, the more the visual system is adapted to the statistics of the visual environment. Berkes et al. (2011) have shown that the dissimilarity of these two distributions monotonically decreases with animal age, supporting the notion of gradual maturation of the system. However, due to constraints of the experimental design it was not possible to determine the role that visual experience plays in this optimization process. In the present study, we explored both the role of experience in this maturation, and the relationship between KL and the more commonly used measure of functional maturity based on direction selectivity.
Specifically, we acquired measurements of spontaneous and visually evoked activity in awake, and directional selectivity tuning in anesthetized visually naïve ferrets (mean age p30) using extracellular 16 channel microwire array electrodes chronically implanted into V1. Immediately after acquiring these measurements, the animals were randomly assigned to one of three experimental conditions: (1) visual training with drifting gratings while anesthetized, (2) visual training with naturalistic movies while anesthetized or (3) no visual training while awake. After 12 hours of training in the assigned experimental condition, both measurements were acquired again. This experimental design allowed us to correlate the two measurements of visual maturity, and explore the role of visual experience in the process of optimizing an immature visual system to the statistics of the visual environment.
Fiser J. (2013) B10: Bayesian methods and generative models. ECVP 2013, Perception 42, 4-5 [Abstract]
In the last two decades, a quiet revolution took place in vision research, in which Bayesian methods replaced the once-dominant signal detection framework as the most suitable approach to modeling visual perception and learning. This tutorial will review the most important aspects of this new framework from the point of view of vision scientists. We will start with a motivation as to why Bayes, then continue with a quick overview of the basic concepts (uncertainty and probabilistic representations, basic equations), moving on to the main logic and ingredients of generative models including Bayesian estimation, typical generative models, belief propagation, and sampling methods. Next we will go over in detail of some celebrated examples of Bayesian modeling to see the argument and implementation of the probabilistic framework in action. Finally, we will have an outlook as to what the potential of the generative framework is to capture vision, and what the new challenges are to be resolved by the next generation of modelers.
Fiser J., Popovic M., Haefner RM. & Lengyel M. (2013) Time and making perceptual decisions. ECVP 2013, Perception 42, 237-237 [Abstract]
In models of perceptual decision making within the classical signal processing framework (e.g. integration-to-bound), time is used to accumulate evidence. In probabilistic, sampling-based frameworks, time is necessary to collect samples from subjective posterior distributions for the decision. Which role is dominant during perceptual decisions? We have analytically derived the progression of the error and subjective uncertainty in time for these two models of decision making, and found that they show a very differently evolving pattern of the correlation between subjects’ error and their subjective uncertainty. Under sampling, after a brief initial period, the correlation always increases monotonically to an asymptote with this increase continuing long after the error itself has reached its asymptote. In contrast, integration-to-bound shows increasing or decreasing changes in correlation depending on the posterior’s kurtosis, and with additive behavioral noise, the correlation decreases. We conducted a decision making study where subjects had to perform time-limited orientation matching and report their uncertainty about their decisions, and found that the results confirmed both predictions of the sampling-based model. Thus, under typical conditions, time in decision making is mostly used for assessing what we really know and not for gathering more information.
Fiser J., Savin C., Berkes P., Chiu C. & Lengyel M. (2013) Experience-based development of internal probabilistic representations in the primary visual cortex. VSS 2013, Journal of Vision 13 (9), 600-600 [Abstract]
The developmental increase in similarity between spontaneous (SA) and average stimulus-evoked activity (EA) in the primary visual cortex has been suggested to reflect a progressive adaptation of the animal’s internal model to the statistics of the environment, a hallmark of probabilistic computation in the cortex (Berkes et al, 2011). Still, this gradual adaptation could be due to genetically controlled developmental processes that have little to do with the animal's visual experience. To clarify this issue, we disrupted normal visual experience of N=16 ferrets of different ages (P30-P120) so that the animals perceived only diffuse light through their eyelids up to the moment of data collection. We measured neural activity from the superficial layers of V1 and compared SA and EA to those in normally reared controls. Furthermore, we extended the original analysis using maximum entropy models that control not only for the effects of single unit firing rates, but also for the population firing rate distribution which could confound measures of functional connectivity that we use as a measure of learning. The general statistics of V1 activity in lid-sutured animals developed very similarly to controls confirming that withholding natural visual experience does not abolish the general development of the visual system. However, while in the control animals SA was completely similar to EA evoked by natural stimuli and significantly less similar to EA evoked by noise, in lid-sutured animals this specificity to natural inputs disappeared, and the match between SA and EA for natural inputs became incomplete. Our novel analysis further confirmed that learning drives the increase of similarity between SA and EA in the oldest control adults. These results suggest that while intrinsic development of visual circuitry is controlled by developmental factors, learning from visual experience is crucial for the emergence of a complete match between SA and EA.
Popovic M., Lengyel M. & Fiser J. (2013) The role of time in human decision-making. VSS 2013, Journal of Vision 13 (9), 305-305 [Abstract]
The effects of time on human decision-making are well known, yet, the precise mechanisms underlying these effects remain unclear. Under the classic signal processing framework (e.g. integration-to-bound) the passing of time allows for accumulation of evidence, parametric models of probabilistic neural representations (e.g. PPC) hold that time is used for averaging internal noise for a better estimate of firing rates, while non-parametric, sampling-based models posit that time influences the collection of samples from subjective posterior distributions. These models provide different predictions about the nature and temporal evolution of subjects’ errors and the correlation between their error and their subjective uncertainty. We have analytically derived the progression of error and subjective uncertainty in time for the three models under a decision-making scenario, and found characteristic differences in their behavior. Under sampling, after a possible transient decrease depending on the kurtosis of the posterior, the correlation always increases monotonically to an asymptote. Importantly, this increase continues long after the error itself has reached its asymptote. In contrast, both integration-to-bound and PPC models can show increasing or decreasing changes in correlation depending on the posterior’s kurtosis, and when noise corrupts the posterior, this correlation decreases. We conducted a decision-making study in which subjects performed time-limited orientation matching and reported their uncertainty about their decisions, and found that the results confirmed both predictions of the sampling-based model. As these characteristics are not present in parametric and integration-to-bound models, the present results lend strong support to a novel use of time in decision-making: collecting samples from otherwise static internally represented distributions.
Savin C., Berkes P., Chiu C., Fiser J. & Lengyel M. (2013) Similarity between spontaneous and sensory-evoked activity does suggest learning in the cortex. COSYNE 2013, Salt Lake City, UT [Abstract]
The developmental increase in similarity between spontaneous (SA) and average stimulus-evoked activity (EA) in the primary visual cortex has been suggested to reflect a progressive adaptation of the animal’s internal model to the statistics of the environment (Berkes et al., Science 2011). However, it is unknown how much of this adaptation is due to learning or simple developmental programmes. If learning plays a role, it makes two predictions: changes in the functional connectivity between neurons should underlie the changes seen during development, and these developmental changes should be experience-dependent. Neither of the two has been satisfyingly tested, if at all, in previous work. Here we address the issue of functional coupling by novel analyses with maximum entropy models (Schneidman et al., Nature 2006) that control not only for the effects of single unit firing rates, but also for the population firing rate distribution which could otherwise confound measures of functional connectivity (Okun et al., SfN, 2011). We show that functional connectivity plays an increasing role during development in shaping both SA and EA, and in particular that it significantly contributes to the similarity of SA and EA. Moreover, we directly asses the role of experience by comparing neural activities recoded in animals reared with their lids sutured (LS) to those recorded in normally developing controls. Neural activity in LS animals was qualitatively similar to that in controls, confirming that withholding natural visual experience does not abolish the general development of the visual system. However, there were some key differences: the match between SA and EA remained incomplete, and the specificity of this match for natural images was significantly reduced in LS animals. Taken together, these results strongly suggest that learning in the cortex crucially contributes to the similarity between SA and EA.
Popovic M., Haefner R. M., Lengyel M. & Fiser J. (2013) Psychophysical evidence for a sampling-based representation of uncertainty in low-level vision. COSYNE 2013, Salt Lake City, UT [Abstract]
Human and animal studies suggest that human perception can be interpreted as probabilistic inference that relies on representations of uncertainty about sensory stimuli suitable for statistically optimal decision-making and learning. It has been proposed recently that the way the brain implements probabilistic inference is by drawing samples from the posterior probability distribution, where each sample consists of instantaneous activity of a population of neurons (Fiser et al, 2010). However, there is no experimental evidence thus far, showing that an internal representation of uncertainty can extend to low-level sensory attributes, nor that humans use sampling-based representations in perceptual judgment tasks. To address these questions, we created an orientation-matching task in which we measured both subjects’ performance and their level of uncertainty as they matched orientation of a randomly chosen element of the previously presented stimulus. Stimuli consisted of 2-7 differently oriented line segments shown spaced evenly on a circle extending 2 degrees of the visual field. In response to the first question, we found that subjects’ performance and subjective report of uncertainty were significantly correlated (r=0.37, p<.001) and that this correlation was independent of the number of oriented line segments shown. To address the second question, we varied the stimulus presentation time trial-to-trial to influence the number of samples available before making a judgment. Since samples are drawn sequentially, the prediction of the sampling-based representations is that precision of representing uncertainty will depend on the time available independent of the recorded performance. We found that decreasing the presentation time results in a significant decrease of the error-uncertainty correlation (p<0.05) while the performance levels remain constant. Thus, limiting the presentation time influences the reliability of uncertainty representation specifically, in agreement with sampling-based representations of uncertainty in the cortex, and in contrast with the predictions of other probabilistic representations.
Haefner RM., Berkes P. & Fiser J. (2013) Perceptual decision-making in a sampling-based neural representation. COSYNE 2013, Salt Lake City, UT [Abstract]
Most computational models of the responses of sensory neurons are based on the information in external stimuli and their feed-forward processing. Extrasensory information and top-down connections are usually incorporated on a post-hoc basis only, e.g. by postulating attentional modulations to account for features of the data that feed-forward models cannot explain. To provide a more parsimonious account of perceptual decision-making, we combine the proposal that bottom-up and top-down connections subserve Bayesian inference as the central task of the visual system (Lee & Mumford 2003) with the recent hypothesis that the brain solves this inference problem by implementing a sampling-based representation and computation (Fiser et al 2010). Since the sampling hypothesis interprets variable neuronal responses as stochastic samples from the probability distribution that the neurons represent, it leads to the strong prediction that dependencies in the internal probabilistic model that the brain has learnt will translate into observable correlated neuronal variability. We have tested this prediction by implementing a sampling-based model of a 2AFC perceptual decision-making task and directly comparing the correlation structure among its units to two sets of recently published data. In agreement with the neurophysiological data, we found that: a) noise correlations between sensory neurons dependent on the task in a specific way (Cohen & Newsome 2008); and b) that choice probabilities in sensory neurons are sustained over time, even as the psychophysical kernel decreases (Nienborg & Cumming 2009). Since our model is normative, its predictions depend primarily on the task structure, not on assumptions about the brain or any additional postulated processes. Hence we could derive additional experimentally testable predictions for neuronal correlations, variability and performance as the task changes (e. g. to fine discrimination or dynamic task switching) or due to perceptual learning during decision-making.
Orbán G., Aslin RN. & Fiser J. (2013) Statistical optimal effects of uncertainty in scene segmentation on human learning. BCCCD 2013, Budapest, Hungary [Abstract]
In contrast with the traditional deterministic view of perception, a number of recent studies have argued that it is best captured by probabilistic computations. A crucial aspect of real-world scenes is that conflicting cues render stimuli ambiguous which results in multiple hypotheses being compatible with the stimuli. Although the effects of perceptual uncertainty have been well-characterized on perceptual decisions, the effects on learning have not been studied. Statistically optimal learning requires combining evidence from all alternative hypotheses weighted by their respective certainties, not only from the most probable interpretation. We tested whether human observers can learn about and make inferences in such situations. We used an unsupervised visual learning paradigm, in which ecologically relevant but conflicting cues gave rise to alternative hypotheses as to how unknown complex multi-shape visual scenes should be segmented. The strength of conflicting segmentation cues, “high-level” statistically learned and “low-level” grouping features of the input, were systematically manipulated in a series of experiments, and human performance was compared to Bayesian model averaging. We found that humans weighted and combined alternative hypotheses of scene description according to their reliability, demonstrating an optimal treatment of uncertainty in learning. These results capture not only the way adults learn to segment new visual scenes, but also the qualitative shift in learning performance from 8-month-old infants to adults. Our results suggest that models of perceptual learning that evaluate a single hypothesis with the “best explanatory power” instead of model averaging, are not sufficient for characterizing human visual learning
2012
Haefner RM., Berkes P. & Fiser, J. (2012) Decision-making and attention in a sampling-based neural representations. COSYNE 2012, Frontiers in Neuroscience Conference Abstract: Neural Coding, Decision-Making & Integration in Time. [Abstract]
According to the sampling hypothesis, the activity of sensory cortex can be interpreted as drawing samples from the probability distribution over features that it implicitly represents. Perceptual inference is performed by assuming that the samples are drawn from an internal model that the brain has built of the external world (Fiser et al 2010).
We explore the implications of this hypothesis in the context of a perceptual decision-making task and present three findings:
(1) Because the simple generative model for typical experimental stimuli does not match the rich internal model of the brain, the psychophysical performance is below what is theoretically possible based on the sensory neurons' responses. This can explain why previous studies have found that surprisingly few sensory neurons are required to match the performance of the animal, and why traditional decoding models need to invoke ad-hoc “decision noise” (Shadlen et al 1996) when pooling the responses of all relevant sensory neurons.
(2) We show that in the sampling framework typical 2AFC tasks induce higher correlations between neuron pairs supporting the same choice, than between those contributing to different choices - as has previously been observed empirically (Cohen & Newsome 2008).
(3) We demonstrate that, given the limited number of samples in a trial and a reward structure that is strongly concentrated on particular parts of the sampling space, expected reward is maximized by sampling from a probability distribution other than the veridical posterior (for a related, but parametric, idea see Lacoste-Julien et al 2011).
Based on these findings we propose that the brain actively adapts the posterior distribution to account for (1) and (3), and that this adaptation is closely related to the cognitive concept of attention. Using this interpretation of attention, we replicate existing neurophysiological findings and make new predictions.
Popovic M., Van Hooser S. & Fiser J. (2012) Comparing functional measures of maturity in the developing ferret’s primary visual cortex. SFN 2012, New Orleans, LA [Abstract]
Traditional assessments of the development of the visual system typically use change in selectivity to particular features such as orientation or direction to determine the maturity of visual cortical circuits. Directional selectivity tuning (DS) was shown not only to increase rapidly and significantly in the two weeks after eye opening but also to exhibit clear experience-dependent maturation (Li et al. 2008, Nature). However, the usefulness of this measurement depends on the assumption that selectivity to a feature such as direction is a direct and precise indicator of how well suited the visual system is for normal functioning. We have developed an alternative measurement based on the statistical similarity of spontaneous and visually evoked activity patterns in the visual cortex as measured by Kullback-Leibler divergence (KL). This measure assesses the optimality of information encoding given the statistics of the visual input, and under the assumption that the visual system encodes information probabilistically (Berkes et al. Science 2011). We showed previously that information encoding becomes more optimal with age; however, it remains unclear whether this change is driven by visual experience or purely developmental factors. In the present work, we explore the relationship between traditional selectivity measurements and our measurement of development in animals at the same stage of development, but with varying exposure to visual experience.
We recorded extracellularly in V1 of ferrets at P30 using implanted line arrays of microwire electrodes. At this age, ferrets’ eyes are closed limiting their visual experience prior to the experiment. First we conducted a pretest to obtain measurements of neural activity in the awake animal in response to drifting gratings, natural movies, as well as spontaneous activity. This enabled us to obtain measurements of both DS and the KL between the activity patterns during presentation of different types of stimuli in the same animal and at the same stage of visual development. Next, animals underwent 10-15 hours of training consisting of passively viewing full-field gratings drifting in different directions under isoflurane anesthesia. After training, we conducted a posttest containing the same stimulus conditions as pretest thus allowing us to directly assess the role of visual experience in development as measured by both DS and KL divergence between distributions of neural activity patterns under different stimulus conditions. Furthermore, this allowed us to elucidate the relationship between the two measures and the differences in their trends across different exposures to visual experience.
Fiser J., Savin C., Berkes P., Chiu C. & Lengyel, M. (2012) Visual experience drives the increase in similarity between spontaneous and stimulus evoked activity in V1. SFN 2012, New Orleans, LA [Abstract]
Despite ample behavioral evidence for probabilistic learning in the brain, the neural underpinnings of this process remain unclear. It has been recently hypothesized that spontaneous activity in primary sensory areas could be a marker for this learning process (Fiser et al, 2010). In this view, the increase in similarity between spontaneous and stimulus-evoked activity over development reflects a progressive adaptation of the animal’s internal model to the statistics of the environment. Indeed, recordings in the developing visual system of the ferret confirmed that the structure of spontaneous activity gradually adapts with age to that of visually evoked activity and that this adaptation is specific to natural stimuli (Berkes et al, 2011). However, this finding leaves open the option that the gradual adaptation is due to genetically controlled developmental processes that has little to do with the animal's visual experience. One test of the theory is to disrupt visual experience and measure whether this intervention reduces the similarity between spontaneous and stimulus evoked activity for naturalist stimuli.
To perform such a test, we collected neural activity of the primary visual cortex of N=14 lid-sutured awake behaving ferrets at three different age groups of P44, P80 and P120. The animals had no visual experience only diffuse light through their eye lids up to the moment of data collection. Extracellular multi-unit activity was collected with a 16 channel line electrode from the superficial layers of V1. We analyzed the spontaneous and stimulus evoked activity and compared them to normally reared controls. The analysis revealed that, while in control animals in the oldest age group spontaneous activity is most similar to activity evoked by natural stimulation, in lid-sutured animals responses to natural scenes show the same degree of similarity as those evoked by noise. The effect is specific to this distinction, as the general statistics of V1 activity in lid-sutured animals develop very similarly to controls. Overall, these results confirm that while intrinsic development of visual circuitry is strongly controlled by developmental factors, learning is one of the driving forces behind the observed increases in similarity between the spontaneous and stimulus evoked activity.
Savin C., Berkes P., Chiu C., Fiser J. & Lengyel M. (2012) Similarity between spontaneous and sensory-evoked activity does suggest learning in the cortex. SFN 2012, New Orleans, LA [Abstract]
Over development, spontaneous activity (SA) in primary visual cortex (V1) becomes increasingly similar to stimulus evoked activity (EA) (Berkes et al, Science 2011). This increasing similarity has been taken to reflect a progressive adaptation of the animal's internal models to the statistics of the environment (Berkes et al, 2011). An alternative interpretation (Okun et al, Soc Neurosci Abstr 2011) suggests these effects can be obtained with a simple model that only matches the mean firing rate on each electrode and the distribution of population rates, but not detailed patterns of pairwise correlations between neurons. Hence, the increased similarity between SA and EA in adult animals could be simply a reflection of changes in these network properties, without necessarily implying the learning of a model of the environment (Okun et al, 2011).
To test this hypothesis directly, we created surrogate data by fitting maximum entropy models to the original data recorded in ferret V1 (Berkes et al, 2011) matching the mean firing rates of all individual channels and the distribution of population rates, as suggested by Okun et al. We then compared these surrogate data to the original. In line with the analysis of Okun et al (2011), we found that similar results to those described by Berkes et al (2011) could be obtained if the true response distributions were only constrained by single neuron firing rates and population rate distributions. However, in contrast to the prediction of Okun et al (2011), the true response distributions were found to become increasingly dissimilar from their surrogate versions, and this difference was important in matching SA and EA. In a parallel submission (Fiser et al) we also report that in lid sutured animals the specificity of the match between SA and EA to natural scenes is disrupted, suggesting a direct role of visual experience in its development. Overall, our results support the view that learning is indeed one of the factors driving the increase in similarity between spontaneous and evoked activity in primary visual cortex.
Popovic M., Lengyel M. & Fiser J. (2012) Decision-making under time constraints supports sampling-based representation of uncertainty in vision. ECVP 2012, Perception 41, 58-59 [Abstract]
Increasing body of psychophysical evidence supports the view of human perception as probabilistic inference that relies on representations of uncertainty about sensory stimuli and that is appropriate for statistically optimal decision making and learning. A recent proposal concerning the neural bases of these representations posits that instantaneous neural activity corresponds to samples from the probability distribution it represents. Since these samples are drawn sequentially, a crucial implication of such sampling-based representations is that precision of representing uncertainty will depend on the time available. To test this implication we created an orientation- matching task in which the subjects were presented several differently oriented line segments. We measured both subjects’ performance and their level of uncertainty as they matched the orientation of a randomly chosen element from the previously presented stimulus. We varied the stimulus presentation time trial-to-trial to influence the number of samples available before making a decision. We found that subjects’ performance and uncertainty judgment were correlated. Importantly, with decreasing the presentation time this correlation decreased significantly while the performance levels did not change. Thus, limiting the available time specifically influences the reliability of uncertainty representation, in agreement with sampling-based representations of uncertainty in the cortex
Fiser J., Orbán G., Berkes P. & Lengyel M. (2012) Explaining neural variability in the visual cortex through sampling-based neural representations. AREADNE 2012: Research in Encoding and Decoding of Neural Ensembles, Santorini, Greece [Abstract]
It is well-documented that neural responses in sensory cortices are highly variable: the same stimulus can evoke a different response on each presentation. Traditionally, this variability has been considered as noise and eliminated by using trial-averaged responses. Such averaged responses have been used almost exclusively for characterizing neural responses and mapping receptive fields with tuning curves, and accordingly, most computational theories of cortical representations have neglected or focus on unstructured Poisson-like aspects of neural variability. However, the large magnitude, characteristic spatio-temporal patterns and systematic, stimulus-dependent changes of neural variability suggest it may play a major role in sensory processing. We propose that sensory processing and learning in humans and other animals is probabilistic following the principles of Bayesian inference, and neural activity patterns represent statistical samples from a probability distribution over visual features. In this representational scheme, the set of responses at any time in a population of neurons in V1 represents a possible combination of visual features. Variability in responses arises from the dynamics that evokes population patterns with relative frequencies equal to the probability of the corresponding combination of features under the probability distribution that needs to be represented. Consequently, the average and variability of responses encode different and complementary aspects of a probability distribution: average responses encode the mean, while variability and co-variability encode higher order moments, such as variances and covariances, of the distribution. We developed a model derived from this sampling-based representational framework and showed how it can account for the most prominent hitherto unexplained features of neural variability in V1 related to changing variability and the pattern of correlations without necessarily changing mean responses. Besides providing the traditional mean responses and ting curves, the model replicates a wide range of experimental observations on systematic variations of response variability in V1 reported in the literature. These include the quenching of variability at stimulus onset measured either by membrane potential variability or by the Fano factor of spike counts, contrast-dependent and orientation-independent variability of cell responses, contrast-dependent correlations, and the close correspondence between spontaneous and evoked response distributions in the primary visual cortex. Crucially, current theories of cortical computations do not account for any of these non-trivial aspects of neural variability. The framework also makes a number of key predictions related to the time-dependent nature of the sampling-based representation. These results suggest that representations based on samples of probability distributions provide a biologically feasible new alternative to support probabilistic inferential computations of environmental features in the brain based on isy and ambiguous inputs.
MacKenzie KJ., McDevitt EA., Fiser J. & Mednick SC. (2012) The differing effects of REM and non-REM sleep on performance in visual statistical learning. VSS 2012, Journal of Vision 12 (9), 283-283 [Abstract]
Although visual statistical learning (VSL) has been established as a method for testing implicit knowledge gained through observation, little is known about the mechanisms underlying this type of learning. We examined the role of sleep in stabilization and consolidation of learning in a typical VSL task, where subjects observed scenes composed of simple shape combinations according to specific rules, and then demonstrated their gained familiarity of the underlying statistical regularities. We asked 1) whether there would be interference between learning regularities in multiple VSL tasks within close spatial and temporal proximity even if the shapes used in the tasks were different, and 2) whether sleep between interfering conditions could stabilize and consolidate the learning rules, improving performance. Subjects completed four separate VSL learning blocks, each containing scenes composed of different shapes: Learning A and B were presented sequentially, Learning B and C were separated temporally by two hours, and Learning C and D were separated by a period of similar length in which subjects either took a nap which included or excluded REM sleep, or remained awake, either quietly or actively. Familiarity tests with the four structures were conducted following Learning D. If sleep blocks interference, we would expect to see interference between Learning A and B, and not between Learning C and D. If sleep increases learning, performance would be best on the test of Learning D. We found indications of interference between Learning A and B, but only in the non-REM nap group. Also, a significantly improved performance on the Learning D familiarity test was found, but only in the REM nap group. Thus, knowledge gained by VSL does interfere despite segregation in shape identity across tasks, a period of stabilization can eliminate this effect, and REM sleep enhances acquisition of new learning.
Haefner RM., Berkes P. & Fiser J. (2012) The relation of decision-making and endogenous covert attention to sampling-based neural representations. VSS 2012, Journal of Vision 12 (9), 159-159 [Abstract]
Empirical evidence suggests that the brain during perception and decision-making has access to both point estimates of any external stimulus and to the certainty about this estimate. This requires a neural representation of entire probability distributions in the brain. Two alternatives for neural codes supporting such representations are probabilistic population codes (PPC) and sampling-based representations (SBR). We examined the consequences of an SBR and its implications in the context of classical psychophysics. We derive analytical expressions for the implied psychophysical performance curves depending on the number of samples collected (i.e. the stimulus presentation time), which constitute the theoretical limit for optimal performance. This time-dependence allows us to contrast SBR with PPC, in which probability distributions are represented explicitly and near-instantaneously as opposed to successively over time as for sampling. We compared our predictions with empirical data for a simple two-alternative-choice task distinguishing between a horizontal and a vertical Gabor pattern embedded in Gaussian noise. Recasting the decision-making process in the sampling framework also allows us to propose a new computational theory for endogenous covert attention. We suggest that the brain actively reshapes its representation of the posterior belief about the outside world in order to collect more samples in parts of the stimulus space that is of greatest behavioral relevance (i.e. entails rewards or costs). We show that compared to using the veridical posterior, the benefit of such a mechanism is greatest under time pressure - exactly when the largest effects due to endogenous attention have traditionally been seen. We present experimental data for a task in which attention has been manipulated by varying the behavioral relevance of two stimuli concurrently on the screen, but not their probabilities as traditionally done.
Yang C., Lisitsyn D. & Fiser J. (2012) Testing the nature of the representation for binocular rivalry. VSS 2012, Journal of Vision 12 (9), 204-204 [Abstract]
Recently, several studies proposed a probabilistic framework for explaining the phenomenon of binocular rivalry, as an alternative to the classic bottom-up or eye-dominant interpretation of it. According to this framework, perception is generated from the observer’s internal model of the visual world, based on sampling-based probabilistic representations and computations in the cortex. To test the validity of this proposal, we trained participants with repeated four patterns of two-Gabor patches corresponding to the four possible perceptions in binocular rivalry settings, with a particular probability distribution of appearance (10%, 40%, 15% and 35%). We also tested participants’ prior and posterior distributions of these four perceptions in both binocular rivalry and non-rivalry situations, where they either made judgments by what was perceived in rivalry or guessed what could possibly be the answers of Gabor orientation pairs when they saw only non-rivalry Gaussian noise. Kullback–Leibler divergence and resampling methods were used to compare the pretest and posttest distributions from each individual participant. For the non-rivalry inference, three out of four participants showed significant difference (ps<0.05) between pre and post distributions of the four possible answers. Compared with the pretest, the post-test distribution shifted towards the target distribution manipulated in the training session. In contrast, for binocular rivalry, none of the participants showed change in the distributions of four perceptions overall from pretest to posttest, suggesting no learning effect transferred from non-rivalry training. Further analysis on the relationship between perception duration time and distribution changes in rivalry showed that with longer perception duration it was more likely to find pre-test and post-test distribution differences. However, the transition from pretest to posttest did not necessarily follow the target distribution from training. These results provided no decisive evidence that binocular rivalry is a visual process based on probabilistic representation.
Popovic M., Lisitsyn D., Lengyel M. & Fiser J. (2012) Time to decide: sampling based representation of uncertainty in human vision. VSS 2012, Journal of Vision 12 (9), 616-616 [Abstract]
Growing behavioral evidence suggests that animals and humans represent uncertainty about both high and low-level sensory stimuli in the brain for probabilistic inference and learning. One proposal about the nature of the neural basis of this representation of uncertainty suggests that instantaneous membrane potentials of cortical sensory neurons correspond to statistical samples from a probability distribution over possible features those neurons represent. In this framework, the quality of the representation critically depends on the number of samples drawn, and hence on the time available to perform a task. This implies a strong link between the available time and the reliability of the representation. We tested this hypothesis in an orientation matching experiment with two distinct types of stimuli: circles consisting of 1-4 Voronoi patches, each filled in with a number of gray-scale Gabor wavelets with their orientations sampled from a Gaussian distribution with a different mean orientation; and 2-10 differently oriented line segments spaced evenly on a circle. After 2 seconds of stimulus presentation subjects were asked to match the orientation of one of the patches or lines in the stimulus, and indicate their certainty about the correctness of the orientation match. To test our predictions, we manipulated the number of samples on a trial-to-trial basis by varying the time available to respond. Without time constraints, subjects’ performance and certainty judgment were significantly correlated independent of the number of patches or lines the stimuli consisted of. With a decrease in available time, subjects’ orientation and certainty judgments followed the theoretically predicted trends. Importantly, a decrease in response time lead to a decrease in correlation between performance and uncertainty, even though the performance remained unchanged. Therefore, limiting the response time, and consequently the number of samples drawn, significantly influences the quality of uncertainty representation in accord with the sampling hypothesis.
Ledley J., MacKenzie K. & Fiser J. (2012) Coding object size based rules in 3D visual scenes. VSS 2012, Journal of Vision 12 (9), 806-806 [Abstract]
Learning abstract rules in the auditory and visual domains is customarily investigated with the AAB vs. ABB paradigm where each scene contains three auditory events or visual objects and either identity or an attribute of these items, such as the size of the objects, follows a "same-same-different" (i.e. AAB) pattern during a training period. In a subsequent test session, never seen before items are used and subjects’ preference to judge the AAB over ABB arrangements as familiar is taken as evidence for acquiring the abstract rule. We asked whether 2D retinal or 3D perceptual size is the basis of this judgment in case of visual rule learning of size arrangements. We used three triplets of 3D computer graphic colored objects arranged in perspective so that by physical extent on the screen they followed a large-large-small (AAb) template, but due to perspective their perceptual appearance was (aBB). After 2 minutes of random sequential presentation of the triplets for 2 sec each without any explicit task, two tests were administered with two versions of instruction. In the first test (No Context), context and perspective were taken away, and triplets were presented horizontally on white background, in the second (Context), exactly the same context was used as during the practice. The instructions were either "choose the more familiar scene" (Naïve) or "considering size, chose the more familiar scene" (Cued). In the Naïve-No Context condition, subject showed no preference between AAb and aBB, which changed in the Cued condition to significant aBB preference. In the Cued-Context condition, subjects showed a strong aBB preference. However, in the Naïve-Context condition, they switched to significant AAb preference. Thus size-rule coding seems to utilize high-level perceptual coding of size when directed explicitly, but in implicit familiarity tasks the more veridical retinal coding has a stronger influence.
Lisitsyn D., Galperin H. & Fiser J. (2012) Linking eye fixation strategies to experience in visual statistical learning. VSS 2012, Journal of Vision 12 (9), 1005-1005 [Abstract]
Linking eye-movement to visual perception or to learning has been notoriously difficult due to the fact that the visual stimulus is either too simplified providing no insights to the true nature of learning or with too rich input, the process of learning becomes intractable. Visual statistical learning (VSL) provides an ideal framework for such studies since it uses stimuli with precisely controlled statistics and regular spatial layout. We used the classical VSL paradigm combined with eye tracking and asked whether this controlled implicit learning paradigm allows following the contribution and development of eye movements during the learning process. Stimuli were based on 12 simple shapes combined into six base-pairs. From this alphabet, each scene was composed by randomly selecting three of the base-pairs and juxtaposing them on a grid to generate over 140 scenes that were shown sequentially for 3 sec each on a large 4*3 feet screen while the subjects’ eye movements were monitored. Subjects had no task beyond attentively observing the scenes. Post practice, subjects were given a test with multiple trials, where they had to choose between true building base-pairs and random combination of pairs based on their judgment of familiarity. Subjects typically became familiar with the base-pairs to a different degree, showing a wide variation of success in choosing the true base-pair over a foil. This distribution of percent correct values was correlated with various measures of eye-movement. We found a correlation between the amount of eye-fixations and the total fixation time on the shapes of the highly learned pairs versus the pairs that weren’t learned. These results provide a first indication that not only in highly explicit cognitive tasks, but even in implicit observational tasks, eye movements have a tight link to the acquired knowledge of the visual scenes.
Janacsek K, Fiser J, & Nemeth D. (2012). What is the best time to acquire new skills: age-related differences in implicit sequence learning across lifespan. BCCCD 2012, Budapest, Hungary [Abstract]
Implicit skill learning underlies not only motor, but also cognitive and social skills. Nevertheless, the ontogenetic changes in humansʼ implicit learning abilities have not yet been comprehensively characterized. We investigated such learning across the life span, between 4-85 years of age with an implicit probabilistic sequence learning task, and we found that the difference in implicitly learning strongly vs. weakly predictable events exhibited a characteristic and rapid decrement around age of 12. These lifelong learning efficiency measurements support an extension of the traditional 2-stage lifespan skill acquisition model: in addition to a decline above the age 60 reported in previous studies, sensitivity to raw probabilities and, therefore, acquiring fundamentally new skills is significantly more effective until early adolescence than later in life. These results suggest that due to developmental changes in early adolescence, implicit skill learning processes undergo a marked shift in weighting raw probabilities vs. more complex interpretations of events, which, with appropriate timing, prove to be an optimal strategy for human skill learning.
MacKenzie K., Aslin RN. & Fiser J. (2012) Statistical learning of hierarchical visual structures by human infants. BCCCD 2012, Budapest, Hungary [Abstract]
Brandeis University University of Rochester Brandeis University, USA Human infants are known to implicitly learn statistical regularities of their sensory environment in various perceptual domains. Visual statistical leaning studies with adults have illustrated that this learning is highly sophisticated and well approximated by optimal probabilistic chunking of the unfamiliar hierarchical input into statistically stable segments that can be interpreted as meaningful perceptual units. However, the emergence and use of such perceptual chunks at an early age and their relation to stimulus complexity have not been investigated. In three experiments, we found that 9-month-old infants can extract and encode statistical relationships within complex, hierarchically structured visual scenes, but they are not able to identify and handle these chunks as individual structures in the same manner as adults. These results suggest that as stimulus complexity increases, infantsʼ ability to extract chunks becomes limited, even though they are perfectly able to encode the structure of the scene. Apparently, the ability to use embedded features in more general and complex contexts emerges developmentally after encoding itself is operational.
2011
Popovic M., Lisitsyn D., Lengyel M. & Fiser J. (2011) Simultaneous representation of uncertainty about multiple low-level visual elements. EVCP 2011, Perception 40, 190-190 [Abstract]
Recent findings suggest that humans represent uncertainty for statistically optimal decision making and learning. However, it is unknown whether such representations of uncertainty extend to multiple low-level elements of visual stimuli, although this would be crucial for optimal probabilistic representations. We examined how subjects’ subjective assessment of uncertainty about the orientations of multiple elements in a visual scene and their performance in a perceptual task are related. Stimuli consisted of 1–4 Voronoi patches within a circular 2º wide area, each patch filled with gray-scale Gabor wavelets drawn from distributions with different mean orientations. After a 2 s presentation, the stimulus disappeared and the subjects had to select the overall orientation around a randomly specified location within the area of the stimulus, and report their confidence in their choice. We found that subjects’ performance, as measured by the accuracy of the selected orientation, and their uncertainty judgment were strongly correlated (p<0.00001) even if multiple different orientations were present in the stimulus, and independently of the number of patches. These results suggest that humans not only represent low-level orientation uncertainty, but that this representation goes beyond capturing a general mean and variance of the entire scene.
Fiser J., Berkes P., Orban G. & Lengyel M. (2011) Probabilistic computation: a possible functional role for spontaneous activity in the cortex. ECVP 2011, Perception 40, 53-53 [Abstract]
Although a number of recent behavioral studies implied that the brain maintains probabilistic internal models of the environment for perception, motor control, and higher order cognition, the neural correlates of such models has not been characterized so far. To address this issue, we introduce a new framework with two key ingredients: the “sampling hypothesis” and spontaneous activity as a computational factor. The sampling hypothesis proposes that the cortex represent and compute with probability distributions through sampling from these distributions and neural activity reflect these samples. The second part of the proposal posits that spontaneous activity represents the prior knowledge of the cortex based on internal representations about the outside world and internal states. First, I describe the reasoning behind the proposals, the evidence supporting them, and will derive a number of empirically testable predictions based on the framework. Next, I provide some new results that confirm these predictions in both the visual and the auditory cortices. Finally, I show how this framework can handle previously reported observations about trial-to-trial variability and contrast-independent coding. These results provide a general functional interpretation of the surprisingly high spontaneous activity in the sensory cortex.
MacKenzie KJ., Aslin RN. & Fiser J. (2011) The interaction between chunking and stimulus complexity in infant visual statistical learning. VSS 2011, Journal of Vision 11 (11), 459-459 [Abstract]
Human infants are known to learn statistical regularities of the sensory environment implicitly in various perceptual domains. Visual statistical leaning studies have illustrated that this learning is highly sophisticated and well_approximated by optimal probabilistic chunking of the unfamiliar input. However, the emergence and unitization of such perceptual chunks at an early age and their relation to stimulus complexity have not been investigated before. This study examines how 8_month_old infants can extract statistical relationships within more complex, hierarchically structured visual scenes and how unitization of chunks is linked to familiarity performance. In the first experiment, infants were habituated to quadruplet scenes composed of a triplet of elements always appearing in the same relative spatial arrangement and one noise element connected to the triplet in various ways. After meeting a criterion of habituation, in each of several test trials, infants saw one original triplet and a new triplet containing a rearrangement of familiar elements. Contrary to earlier results obtained with pairs rather than triplets, infants did not show a significant preference for either test stimulus (N = 20, p > 0.9). In a second experiment, infants were habituated using the same quadruplet scenes, but during the test, they saw one of the habituation quadruplets, and a second quadruplet in which the associated noise element was switched with a noise element from another triplet. Infants that habituated (N = 13) to the familiar quadruplet looked longer to the novel quadruplets, indicating they can recognize a change of one single element (p = 0.026), whereas non_habituating infants (N = 9) showed no preference (p > 0.9). These results suggest that as stimulus complexity increases, infants’ ability to learn and unitize chunks becomes limited, even though they are perfectly able to encode the structure of the scene. Apparently, unitization and the ability to use embedded features in more general contexts emerge after encoding itself is already operational.
Popovic M., Lisitsyn D., Berkes P., Lengyel M. & Fiser J. (2011) Uncertainty representation of low-level visual attributes. VSS 2011, Journal of Vision 11 (11), 807-807 [Abstract]
There is increasing behavioral evidence that humans represent uncertainty about sensory stimuli in a way that it is suitable for decision making and learning in a statistically optimal manner. Do such representations of uncertainty exist for low-level visual stimuli, and furthermore, are they probabilistic in nature? We tested whether subjective assessment of the orientation uncertainty of a stimulus consisting of a fixed number of Gabor wavelets of different orientations reflects the true distribution of orientation uncertainty of the stimulus. Textured gray-scale stimuli were created by superimposing Gabor wavelets of three spatial frequency bands with their orientation randomly sampled from a bimodal Gaussian distribution. After 2 seconds of stimulus presentation, two oriented lines were displayed and subjects were asked to indicate the overall orientation of the stimulus by choosing one of the lines, or to opt not to respond if they were uncertain about the orientation. The orientation of the two lines matched the mean orientation of the stimulus orientation distribution and one of the modes. On average, subjects strongly preferred the mode (65%) over the mean (20%) and only rarely chose not to respond (15%) when the distribution of the orientations had two prominent modes. Increasing the variance of each mode led to a gradual reversal of ratios between the “mode” and “uncertain” responses. When the increase of variances changed the shape of the distribution to unimodal, subjects chose the mode 25%, the mean 15%, and the “uncertain” option 60% of the time. Results suggest that uncertainty associated with low level visual stimuli is explicitly represented as a probability distribution at a level of precision that goes beyond that of a simple parametric representation.
Fiser J., Orbán G. & Lengyel M. (2011) Uncertainty in scene segmentation: statistically optimal effects on learning visual representations. VSS 2011, Journal of Vision 11 (11), 994-994 [Abstract]
A number of recent psychophysical studies have argued that human behavioral processing of sensory inputs is best captured by probabilistic computations. Due to conflicting cues, real scenes are ambiguous and support multiple hypotheses of scene interpretation, which require handling uncertainty. The effects of this inherent perceptual uncertainty have been well-characterized on immediate perceptual decisions, but the effects on learning (beyond non-specific slowing down) have not been studied. Although it is known that statistically optimal learning requires combining evidence from all alternative hypotheses weighted by their respective certainties, it is still an open question whether humans learn this way. In this study, we tested whether human observers can learn about and make inferences in situations where multiple interpretations compete for each stimulus. We used an unsupervised visual learning paradigm, in which ecologically relevant but conflicting cues gave rise to alternative hypotheses as to how unknown complex multi-shape visual scenes should be segmented. The strength of conflicting segmentation cues, “high-level” statistically learned chunks and “low-level” grouping features of the input based on connectedness, were systematically manipulated in a series of experiments, and human performance was compared to Bayesian model averaging. We found that humans weighted and combined alternative hypotheses of scene description according to their reliability, demonstrating an optimal treatment of uncertainty in learning. These results capture not only the way adults learn to segment new visual scenes, but also the qualitative shift in learning performance from 8-month-old infants to adults. Our results suggest that perceptual learning models based on point estimates, which instead of model averaging evaluate a single hypothesis with the “best explanatory power” only, are not sufficient for characterizing human visual learning of complex sensory inputs.
Selig G., Lisitsyn D., Bex P. & Fiser J. (2011) The diagnostic features used for recognizing faces under natural conditions. VSS 2011, Journal of Vision 11 (11), 614-614 [Abstract]
Classical studies of face perception have used stimulus sets with standardized pose, feature locations and extremely impoverished information content. It is unclear how the results of these studies translate to natural perception, where faces are typically encountered in a wide variety of viewpoints and conditions. To address this issue, we used a 2-AFC coherence paradigm, a novel method of image generation and photographs of real faces presented at multiple viewpoints in natural context. A library of portraits, with 10–15 images of each person in various positions, was collected and in each image the prominent features (eye, mouth, ear, etc.) were labeled. Images were decomposed using a bank of Gaussian-derivative filters that gave the local orientation, contrast and spatial frequency, then reconstructed using a subset of these elements. Noise was added by altering the proportion of filter elements in their correct, signal location or a random noise location. On each trial, subjects first viewed a noiseless image, followed by noisy versions of a different exemplar of the same face and of a different face, and had to identify which image matched the person in the source image. The proportion of elements in the correct location on each trial was varied using a staircase procedure to maintain 78% correct responses. Each labeled feature was then analyzed independently by reverse correlation based on correct and incorrect trials. For correct identification, a significantly higher proportion of signal elements were necessary in the hair, forehead and nose regions, whereas elements in regions indicated as diagnostic in classical studies based on standardized stimulus sets, such as eye and mouth, were significantly less influential. These results are in odds with earlier findings and suggest that under natural conditions humans use a more extended and different set of features for correct face identification.
Berkes P., Turner R. & Fiser J. (2011) The army of one (sample): the characteristics of sampling-based probabilistic neural representations. COSYNE 2011, Nature Precedings (2011) [Abstract]
There is growing evidence that humans and animals represent the uncertainty associated with sensory stimuli and utilize this uncertainty during planning and decision making in a statistically optimal way. Recently, a nonparametric framework for representing probabilistic information has been proposed whereby neural activity encodes samples from the distribution over external variables. Although such sample-based probabilistic representations have strong empirical and theoretical support, two major issues need to be clarified before they can be considered as viable candidate theories of cortical computation. First, in a fluctuating natural environment, can neural dynamics provide sufficient samples to accurately estimate a stimulus? Second, can such a code support accurate learning over biologically plausible time-scales? Although it is well known that sampling is statistically optimal if the number of samples is unlimited, biological constraints mean that estimation and learning in cortex must be supported by a relatively small number of possibly dependent samples. We explored these issues in a cue combination task by comparing a neural circuit that employed a sampling-based representation to an optimal estimator. For static stimuli, we found that a single sample is sufficient to obtain an estimator with less than twice the optimal variance, and that performance improves with the inverse square root of the number of samples. For dynamic stimuli, with linear-Gaussian evolution, we found that the efficiency of the estimation improves significantly as temporal information stabilizes the estimate, and because sampling does not require a burn-in phase. Finally, we found that using a single sample, the dynamic model can accurately learn the parameters of the input neural populations up to a general scaling factor, which disappears for modest sample size. These results suggest that sample-based representations can support estimation and learning using a relatively small number of samples and are therefore highly feasible alternatives for performing probabilistic cortical computations.
Turner RE., Berkes P. & Fiser J. (2011) Learning complex tasks with probabilistic population codes. COSYNE 2011, Nature Precedings (2011) [Abstract]
Recent psychophysical experiments imply that the brain employs a neural representation of the uncertainty in sensory stimuli and that probabilistic computations are supported by the cortex. Several candidate neural codes for uncertainty have been posited including Probabilistic Population Codes (PPCs). PPCs support various versions of probabilistic inference and marginalisation in a neurally plausible manner. However, in order to establish whether PPCs can be of general use, three important limitations must be addressed. First, it is critical that PPCs support learning. For example, during cue combination, subjects are able to learn the uncertainties associated with the sensory cues as well as the prior distribution over the stimulus. However, previous modelling work with PPCs requires these parameters to be carefully set by hand. Second, PPCs must be able to support inference in non-linear models. Previous work has focused on linear models and it is not clear whether non-linear models can be implemented in a neurally plausible manner. Third, PPCs must be shown to scale to high-dimensional problems with many variables. This contribution addresses these three limitations of PPCs by establishing a connection with variational Expectation Maximisation (vEM). In particular, we show that the usual PPC update for cue combination can be interpreted as the E-Step of a vEM algorithm. The corresponding M-Step then automatically provides a method for learning the parameters of the model by adapting the connection strengths in the PPC network in an unsupervised manner. Using a version of sparse coding as an example, we show that the vEM interpretation of PPC can be extended to non-linear and multi-dimensional models and we show how the approach scales with the dimensionality of the problem. Our results provide a rigorous assessment of the ability of PPCs to capture the probabilistic computations performed in the cortex.
Orbán G., Lengyel M. & Fiser J. (2011) Statistically optimal effects of uncertainty in scene segmentation on human learning. COSYNE 2011, Nature Precedings (2011) [Abstract]
High-throughput neuroscience presents unique challenges for exploratory data analysis. Clustering often helps experimenters make sense of data, but model-based clustering techniques, including Dirichlet-process mixture models, have difficulty when differing subsets of dimensions are best explained by differing clusterings. As a result, they can be misled by irrelevant dimensions, they easily miss structure that dimensionality reduction methods find, and they often predict less accurately than discriminative alternatives. We introduce cross-categorization, a modeling technique for heterogeneous, high-dimensional tabular data that addresses these limitations. Based on an efficient MCMC inference scheme for a novel nonparametric Bayesian model, cross-categorization infers which groups of dimensions share a common generative history and are therefore mutually predictive.
2010
Berkes P., White BL. & Fiser J. (2010) Sparseness is not actively optimized in V1. COSYNE 2010, Salt Lake City, UT [Abstract]
Sparse coding is a powerful idea in computational neuroscience referring to the general principle that the cortex exploits the benefits of representing every stimulus by a small subset of neurons. Advantages of sparse coding include reduced dependencies, improved detection of co-activation of neurons, and a more efficient encoding of visual information. Computational models based on this principle have reproduced the main characteristics of simple cell receptive fields in the primary visual cortex (V1) when applied to natural images.
However, direct tests on neural data of whether sparse coding is an optimization principle actively implemented in the brain have been inconclusive so far. Although a number of electrophysiological studies have reported high levels of sparseness in V1, these measurements were made in absolute terms and thus it is an open question whether the observed high sparseness indicates optimality or simply high stimulus selectivity. Moreover, most of the recordings have been performed in anesthetized animals, but it is not clear how these results generalize to the cell responses in the awake condition.
To address this issue, we have focused on relative changes in sparseness. We analyzed neural data from ferret and rat V1 to verify two basic predictions of sparse coding: 1) Over learning, neural responses should become increasingly sparse, as the visual system adapts to the statistics of the environment. 2) An optimal sparse representation requires active competition between neurons that is realized by recurrent connections. Thus, as animals go from awake state to deep anesthesia, which is known to eliminate recurrent and top-down inputs, neural responses should become less sparse, since the neural interactions that support active sparsification of responses are disrupted.
To test the first prediction empirically, we measured the sparseness of neural responses in awake ferret V1 to natural movies at various stages of development, from eye opening to adulthood. Contrary t