This paper presents a new computational theory of how humans integrate successive views to form a perceptual map. Traditionally, this problem has been thought of as a straightforward integration problem whereby position of objects in one view is transformed to the next and combined. However, this step creates a paradoxical situation in human perceptual mapping. On the one hand, the method requires errors to be corrected and the map to be constantly updated, and yet, on the other hand, human perception and memory show a high tolerance for errors and little integration of successive views. A new theory is presented which argues that our perceptual map is computed by combining views only at their limiting points. To do so, one must be able to recognize and track familiar objects across views. The theory has been tested successfully on mobile robots and the lessons learned are discussed.