Acoustic properties of residential building are often neglected by designers, developers, contractors, and even home buyers. Noises from both the internal and external environments affect occupants' daily lives. This motivates the current study which aims to identify all types of audible noises in Malaysian residential buildings and to determine the physical and psychological impact of noise loudness and annoyance on occupants in various types of residential buildings. A questionnaire was conducted covering 19 types of noises in residential buildings with participants assessing the loudness and annoyance level. There were 171 valid forms collected from around Malaysia for analysis from one thousand posted forms, responding rate of 17.1%. Traffic noises were deemed as the most undesired type of noise, followed by the noises from neighbors and animals. More interestingly, the annoyance experienced in relation to noise from traffic and neighbors were inter-correlated. While many animal noises such as birds chirping, rooster calls, and insect noise cause little irritation to occupants, dogs barking are significantly more annoying to the occupants. Job stress and occupants' sensitivity to noise are highly correlated with many types of noises. Generally, the influences of humanrelated factors are more significant relative to the housing-related factors on the perceived noise loudness and annoyance.