Damn, wiss beat me to it!
Yeah, "always" is a bit silly. You shouldn't usually go below unity gain (Av = 1) - some op amps aren't stable - but there's no reason you can't go to unity gain, or below it on a decent op amp (only possible in the inverting configuration).