Yeah, while old-school non-low-drop linear regulators such as 7805 are still fairly OK, avoid most old low-drop regulators, as they tend to have strict requirements for the exact types and values of output capacitors.
Market is full of regulators specified to be stable with ceramic output capacitors; use them whenever possible. Also try to find one which is happy with lower capacitance, allowing smaller solution area and possibly lower total cost. Some are OK with 1uF while others require 10uF.
Using specific types and values of output capacitors in your BOM tends to eat away the price benefit you get from a cheap, old-tech regulator.