The Galactic Center black hole Sagittarius A* (Sgr A*) is a prime observing target for the Event Horizon Telescope (EHT), which can resolve the 1.3 mm emission from this source on angular scales comparable to that of the general relativistic shadow. Previous EHT observations have used visibility amplitudes to infer the morphology of the millimeter-wavelength emission. Potentially much richer source information is contained in the phases. We report on 1.3 mm phase information on Sgr A* obtained with the EHT on a total of 13 observing nights over 4 years. Closure phases, the sum of visibility phases along a closed triangle of interferometer baselines, are used because they are robust against phase corruptions introduced by instrumentation and the rapidly variable atmosphere. The median closure phase on a triangle including telescopes in California, Hawaii, and Arizona is nonzero. This result conclusively demonstrates that the millimeter emission is asymmetric on scales of a few Schwarzschild radii and can be used to break 180-degree rotational ambiguities inherent from amplitude data alone. The stability of the sign of the closure phase over most observing nights indicates persistent asymmetry in the image of Sgr A* that is not obscured by refraction due to interstellar electrons along the line of sight.