Perception
My dissertation work was in visual perception, though I have also worked on projects examining user experiences, cognition, emotion, health, and movement. Below is a general introduction to the focus of my dissertation, followed by a detailed look at two specific projects that went into it.
How do we perceive objects?
One of the most important functions of the human visual system is to construct coherent representations of the 3-dimensional surfaces and volumes that make up our physical world. Constructing these representations requires overcoming some complicated problems.
Much of my work has examined the fragmentation problem in vision. That is, the input our eyes receive is something like a fragmented patchwork of color splotches and contour segments - object and surface bits and pieces. Adjacent regions of the retinal image reveal color and edge patches that do not necessarily belong to the same object or surface. Likewise, different parts of the same object or surface often project to disconnected, spatially separate regions of the retina.
The fragmentation problem results from the fact that most objects in the world are opaque. Nearer objects are, therefore, almost always occluding further away ones. As a result, much of the information necessary for complete representations of objects is missing, and what information is in the input is often scattered across disconnected, spatially separate regions of the retina.
In the simple image at left, a sparrow is seen through some foliage. Due to partial occlusion by the leaves and the branch, the sparrow projects to the retina in disconnected, spatially separated patches.
How do we perceive wholes when only disconnected parts are visible?
Perceiving wholes when only parts are visible requires first identifying which parts belong to the same object and then filling in the gaps in the input our eyes receive.
Determining what parts of an image are revealing pieces of the same object would be a relatively simple problem if the pieces of a single object or surface all had the same luminance and color. However, it is almost never the case that the boundaries of an object are defined by homogenous color patches on the retina. In the image above, for example, the boundaries of the sparrow cannot be defined by any particular pixel value.
Instead of relying solely on surface characteristics like luminance, color, and texture, our visual system utilizes information about edges in order to determine which regions of an image belong to the same object.
Interpolating across gaps in the input our eyes receive
Interpolation processes in visual perception unify spatially disconnected regions of the retinal image and fill in edges and surfaces across gaps in the input. They convert the fragmentary input that lands on the retina into representations of complete, coherent objects.
There are two kinds of complementary interpolation processes: one for contours (object edges or boundaries) and one for surfaces. The contour interpolation process is the primary interpolation process in the sense that it defines an object’s boundaries. Surfaces can then spread within these interpolated (or “filled-in”) boundaries.
Contour interpolation
Contour interpolation depends on the geometric relationships among edges terminating in junctions (Kellman & Shipley, 1991). A junction is the intersection of two edges. One kind of junction, a T-junction (so named because the edges intersect in the shape of T) is illustrated in the figure below.
When two edges both terminate in a junction, an interpolated connection can be created between them if they are relatable. Two edges are relatable if they can be connected by a smooth, monotonic curve (Kellman & Shipley, 1991).
In the image at right, a brown branch is partially occluded by a leaf. The two branch parts, Edges A and B, can be connected by an interpolated edge because they are relatable - that is, they can be connected by a smooth, monotonic curve that bends through no more than 90 degrees. At far right are illustrated a subset of other orientations that Edge A could take on and still be relatable with B. At left, the T-junction in which Edge A terminates is highlighted.
Promiscuous Interpolation & Pruning
In fact, an interpolated connection will be created any time that two edges are relatable. These promiscuous interpolations are produced in a relatively early stage of visual processing. Because some relatable edges actually do not belong to the same object, the visual system takes other kinds of information into account, in a second stage of processing, to determine which interpolations are good ones (and should thus be maintained) and which interpolations are bad ones (and should thus be deleted). Because there is always some degree of uncertainty, this process is less of a binary decision process than I have made it sound - each interpolation actually receives some strength or weight depending on the degree to which the other information in the scene supports it.
This promiscuous interpolation and pruning has been the subject of my dissertation. Below are two projects that are a part of that dissertation. Both have contributed to our understanding of the contour interpolation process. The first reveals the early, promiscuous stage of interpolation, in which interpolated connections are made across all pairs of relatable edges. The second project takes a look at some kinds of information that are taken into account in a later stage of processing in order to determine the final strengths of interpolated edges.
Research Projects
Stage 1: Promiscuous Interpolation
In a relatively early stage of visual processing, interpolations are created promiscuously across all pairs of relatable edges. This project reveals the intermediate (i.e., not final) output of this early stage of processing.
Stage 2: Pruning promiscuous interpolations
In a later stage of visual processing, the visual system takes multiple kinds of information into account in order to determine the final strength of interpolations produced in the first stage. This project examines several kinds of information that play a role in determining the strengths of interpolations in the second stage.
References
Kellman, P. J., & Shipley, T. F. (1991). A theory of visual interpolation in object perception. Cognitive Psychology, 23, 141-221.