Recent Findings

Prof. Ehud Zohary (Hebrew University, Jerusalem, Israel)

The children that have been included in our research have all suffered from bilateral cataracts since birth or a few months afterwards. Cataract-removal surgeries are done in Hawassa referral hospital since May 2013. Surgery was done on 108 children from 4 to 19 years of age. The children who are traceable are being followed up till this day. Cataract surgery involving the implantation of artificial lenses, allow for a focused image on the retina. This is a obviously necessary but insufficient in itself to assure functional vision. The operated kids are prone to amblyopia, a condition of reduced vision which cannot be explained by optical factors (as those are corrected by surgical treatment). Amblyopia occurs when the quality of the visual image (from one or both eyes) is very poor during early childhood. See more in the section "vision development and learning".

Overall summary

Generally, we can see clear signs of improvement of visual capabilities with time, though there is considerable variance in the children's performance. Despite sustaining a long deprivation period and a substantial image blur, following surgery the patients' functional vision typically improves dramatically: They are able to recognize objects, and manipulate them; they seamlessly navigate around obstacles, and they no longer behave like blind people. They also improve in some challenging tasks, such as judging the size of an object using information from both vision and haptics or recovering the shape of an object moving behind a narrow slit such that only a tiny fraction of it is visible at any instant. Collectively, these results indicate that vision restoration after prolonged early-onset blindness is feasible to a greater extent than previously thought.

On the other hand, the newly-sighted patients still have some deficiencies, in understanding and responding to social cues: One key example is their failure to automatically follow the gaze of others. Gaze understanding—a suggested precursor for understanding others’ intentions—requires recovery of gaze direction from the observed person's head and eye position. Despite acquiring sufficient resolution to extract eye position after surgery, the newly sighted do not show automatic eye gaze following, probably due to the lack of detailed early visual experience.

To conclude, the emerging picture from these studies is a mixed one. It is clear that surgery at relatively late age for vision restoration is advantageous, but there still are some limits to the capabilities that can be attained. It appears that some visual skills acquired in infancy will be difficult to be learnt later, even when sufficient image resolution for the task is restored. This creates fundamental barriers to spontaneous vision recovery following prolonged deprivation in early age, but it is possible that guided perceptual learning may allow bypassing some of these bottlenecks.

Below is a summary of their performance in some basic tests.

Visual acuity and the Contrast Sensitivity Function

Optometrists and ophthalmologists have their technique for assessing visual acuity. When testing people with very poor vision, the standard Snellen chart is irrelevant. Typically, patients are categorized to ones having only light perception (LP), being able to sense hand motion (HM), or counting fingers from X meters (FC, x). These tests are generally only qualitative and may vary considerably upon the optometrist's specific technique, the lighting conditions in the room and other factors.

We use the contrast sensitivity test: a more controlled technique to assess spatial acuity. We present sinusoidal gratings at different spatial frequencies and contrasts and find the threshold contrast per spatial frequency in which the participant can still discriminate between a vertical and horizontal grating (at 75% of the cases). The results are presented as the contrast sensitivity function (CSF). The CSF is commonly used in vision science and allows us to follow the children's acuity as they improve following surgery. We extract the cutoff frequency: the highest spatial frequency at which the the viewer can still discriminate between the gratings when the contrast is maximal (100%).

The figure below depicts the CSF of 3 patients, and their development of visual acuity following surgery. Typically, the patients are unable to see high spatial frequencies. This results in a blurry image perception. The image of the lion below schematically illustrates the image which is seen depending on the cutoff frequency of the CSF.

In the vast majority of patients, visual acuity improves substantially after cataract surgery, but it does not reach normal levels. Typically, the cutoff frequency reaches an asymptote within the first 6-12 months from surgical treatment. The graphs below show this development in two extreme cases. The scatterplot on the right shows the post-operative acuity of each patient as a function of the patient's pre-operative acuity. Red and cyan circles around specific points denote the patients whose improvement of acuity following surgery was depicted in the left graphs. Points above the main diagonal indicate improvement in acuity. Note the great variability in the extent to which the patients improve, and their final level of acuity. One of the major goals of project eye-opener is to understand the factors contributing to this variability, as post-operative acuity is key to attaining functional vision in many domains (see for example "extraction of face information" below).

Size Constancy is generally intact

One key feature of visual perception is object constancy: The ability to perceive an object as it is, despite radical changes in its physical appearance due to variation in the level of illumination, shading, viewpoint, physical distance, etc. For example: we do not perceive an object as getting smaller when we move away from it, even though its projection on our retina (e.g. its retinal size) is becoming smaller. Apparently, our brain makes use of the pictorial information (such as perspective and other 3D cues) to derive the true size of the object. This capability is termed Size Constancy. Typically-developing children are able to assess the true size of an object irrespective of its distance by 9 years of age.

To study size constancy in the newly-sighted we placed two balls of various sizes in two distances (close and far) from the subjects and asked them to estimate which ball was bigger. If the visual system of the newly-sighted cannot take the distance of the object into account one would expect subjects to estimate the further ball as being smaller than the closer one (following retinal-size-based perception), even when the further ball is actually physically bigger. Somewhat surprisingly, our newly-sighted group were able to compare the objects' physical size quite accurately in such conditions often even before surgery (they actually tended to over-compensate for distance) and most achieved normal performance within a year from surgery (see figure below). Thus, the brain mechanisms supporting size constancy can develop even in patients with highly blurred vision and very poor shape recognition. In hindsight, this may not be that surprising since a highly blurred retinal image is still inversely scaled with distance, and object size can be confirmed by haptics, at least for objects within hand reach. For further information, see the paper size constancy following long-term visual deprivation in our "publications" section

Global motion judgment is also preserved

Motion cues are central for visual perception and action. In primates, motion signals are initially encoded in primary visual cortex (V1) by neurons with small receptive fields. To overcome the aperture problem and estimate the veridical motion of a large object, the visual system must integrate velocity information across space and time in a weighted fashion (using neural population codes). The sensitivity to the direction of global motion develops already in infancy, and in general does not necessitate fine visual acuity. To appreciate this try to assess the global direction of the dot array shown below. Despite the obvious image blur, this is relatively trivial. Thus, we hypothesized that some residual global motion capabilities will be spared in the late-treated cataract patients even prior to surgery, but this might be enhanced following surgery. The motion on the left in which all dots move coherently in one direction. The motion on the right is the case where there is a wide distribution of dot directions, in which we typically perceive the mean direction, but this becomes more difficult when the directions distribution is wider. See more below.

To study motion integration capabilities in the newly-sighted we utilized two tasks that required judging the global direction of dots moving in various directions that were distributed evenly and symmetrically around the central (mean) direction. Note that the greater the bandwidth of direction, the more difficult it is to judge the mean direction of motion. The mean directions differed by either 60°. All patients were capable of extracting the mean direction of the dots even when the directional bandwidth was very wide. Remarkably, the patients' bandwidth threshold was no different than that of controls (the latter viewed blurred stimuli, matching the visual acuity of the newly-sighted).

We also used other versions of the motion discrimination task requiring discrimination of fully coherent motion varying at a fine angle scale (few degrees relative to vertical) or using shorter stimulus presentation time (0.5 sec compared with the initial 3 sec). In both cases, performance of the cataract-treated was worse than of controls.

We propose a model of motion discrimination to better understand whether the above discrimination deficiency is related to a poorer directional selectivity of motion sensitive neurons, sub-optimal decoding of precise directional information from neural population codes, or a slower rate of accumulation of sensory information for perceptual decisions. The overall data suggest that, in general, the newly-sighted do not have problems with pooling motion signals across space, time and directions. Thus, spatiotemporal integration capabilities can develop in early childhood in the absence of patterned vision, despite extreme image blur.

Extraction of Face information depends on post-op visual acuity

We seamlessly and accurately judge a person's gender, age or emotion from just a brief view of his or her face. The newly-sighted children must learn this from scratch after regaining sufficient image resolution following surgery. The example pictures on the left show the original images presented to the patients and the blurred version seen by controls, respectively. Note that you, who have had lifelong experience with faces, have no trouble discriminating between the younger and older face, despite the substantial image blur (cutoff ~1cpd). This was generally true for all controls in all tasks. The scatter plots on the right show individual performance on various tests involving face information extraction: making comparative judgments regarding the Gender, Age, and Emotion of the seen faces. Our results indicate that the patient performance in each of these tasks is strongly dependent on the level of visual acuity attained following cataract surgery.

Gaze following requires early visual experience

Joint attention – realizing the object of another individual's gaze and switching attention to that object – is a key component of behavior with clear evolutionary advantages. But can this strategy develop despite years of extremely poor vision from infancy? We studied 15 congenital bilateral-cataract patients, surgically treated only in late childhood, to test the effect of viewed gaze-direction (indicated by either head turns or eye-movements) on target localization. Performance was compared to that of age-matched controls (viewing a blurry version of the images to account for the poorer visual acuity of the late-treated patients even after surgery). A third group consisted of age-matched early-treated patients (having surgery before 1 year of age). The task required touching the balloon as soon as it appeared. On average, the late-treated participants were faster to point to a target matching a gaze-cue indicated by head-direction (cue-compatible case) than when the target appeared at the opposite side (incompatible case). But they failed to respond faster when gaze was indicated by eye-direction, despite having acquired sufficient resolution post-surgery to discriminate between left and right eye positions. In contrast, early-treated cataract patients (and controls) automatically followed both head- and eye-gaze of others, and performed faster in the cue-compatible cases.

We also tracked the eye-movement patterns elicited by patients and controls when viewing head or eye gaze-oriented actions. The image below shows the average fixation map of controls (left column; n = 31) and late-treated participants (middle column; n = 9) during observation of an actor gazing at a target object (depicted, for illustration only, by a red arrow) indicated by head orientation (upper row) or eye position (lower row). Hotter colors indicate larger fixation dwell times. Predefined interest areas are depicted by red ellipses (right column). The bar plots on the right depict the mean cued-object preference index ([cue congruent – incongruent]/[congruent + incongruent] fixation dwell times) for the control (white) and the late-treated (yellow) groups. Positive values indicate a fixation preference for the cued object.

Again: The late-treated patients fail to automatically follow the eye-gaze of others, but are able to do so when gaze is indicated by head-direction. Our computational modeling suggests that a this late stage in childhood, their learning is severely limited due to reduced availability of internal self-supervision mechanisms, which guide learning in normal development. For more information see our paper "Gaze following requires early visual experience" in the publications section.

The same is true for automatic imitation behavior

Much of our behavior is mediated by a functioning visual system. One example of a visually guided action is our tendency to automatically imitate the motor actions of someone else (as in the “Simon says” game). In this study, we wanted to test whether viewing human gestures elicits automatic mimicry behavior in the newly-sighted children as in controls. To that end, we only tested children who could understand human gestures and presented them with short videos of hands tapping on a table. They were asked to tap with their right or left hand on the basis of the viewed hands’ color (red: left tap; blue: right tap). Crucially, the viewed hands were seen in a similar tapping action using either a spatially-corresponding (i.e. compatible) hand or the other (incompatible) hand. Sighted subjects typically respond faster when the seen action is compatible with required response and slower in incompatible trials. But this does not occur if they watch balls moving in the same path rather than hand action. This hand-specific automatic imitation effect suggests a direct mapping between action-viewing and motor-performance, a key notion of the mirror neuron system hypothesis. Crucially, the newly-sighted were almost unaffected by the viewed hands’ action, showing a much smaller automatic imitation effect. These results suggest that newly-sighted may have a dysfunctional mirror neuron system. This deficit could influence the newly sighted abilities to learn new motor skills or to understand the intentions of others.

The ability to interpret others' social interactions from seeing them is also impaired

Throughout early childhood, humans learn to interpret the social interactions of others from simply watching them. But what if you were deprived of pattern vision throughout your early childhood as in our case. We hypothesized that social action-understanding, if attainable in these children, would rely heavily on biological-motion signals as they are relatively immune to image blur. We tested this by using full-body videos, still images, and point-light-display movies depicting social interactions between two people (See examples below). Our participants categorized these as friendly or aggressive.

In comparison to typically-developing peers, the newly-sighted performed worse in all conditions, yet still better than chance. Surprisingly, form and motion cues had similar contributions to categorization success. Further experiments revealed that the newly-sighted had poorer sensitivity when discriminating between walking directions or body-postures. Our data suggests that development of social vision skills is prone to a sensitive period, limiting their full acquisition.

Discrimination of social interactions, based on motion and form (body-posture) information. (a) Example of the stimuli used in each format. The video-camera icon indicates that we show here a single frame from the short video that was presented in the experiment. (b) Distributions of the individual results (filled circles) in the two groups (light and darker grey: Ethiopian and Israeli controls, red: newly-sighted) for full-body animations (FBA), static snapshots (ST) or point light displays (PLD). Group averages (large open circles) and the 95% confidence interval (error bars) are also depicted (for the controls – data of both sub-groups, combined). Chance level (50%) indicated by the horizontal line.

Learning does occur for other sophisticated tasks such as shape intergration

We typically recognize visual objects, by utilizing the spatial layout of their parts, simultaneously present on our retina. However, a retinotopic image of an object is not a necessary condition for perception: When an object moves behind a
narrow slit, only a tiny fraction of the object is visible at any instant, activating the same place on the retina. Yet, we can recognize the object in such conditions (see example video on right). Here, shape can only be reconstructed by an integration of successive shape-views over time, using a non-retinotopic reference frame. This impressive visual inference capacity normally develops by early childhood. But what if you have had very limited vision during the early years of life?

We studied whether the newly-sighted are able to acquire this capacity. On each trial, a shape moved behind a horizontal or vertical slit (orthogonal to the slit orientation). In such conditions, one must assess the velocity of the moving object to correctly piece together its fragments into a coherent whole. Therefore, we first tested whether the newly-sighted are able to recognize the direction of shape motion. In another task they were asked to recognize the slit-viewed shape (see left image). The newly-sighted children ("pat") judged motion direction as good as their normally sight-developing peers ("cont"), but initially they failed to judge shape configuration (see Panels A & B). However, with further visual experience, they accomplished the task as typically developing peers (see panel C. right image)