In unmagnetized materials the domains are randomly aligned in different directions and cancel each other out. In a magnet all the domains are oriented in the same direction. In the case of a nail, the domains can be aligned in the same direction causing the nail to become magnetic.
