As soon as the neighbours start annoying you, just pick up your house and move somewhere else!