What are you reading the switch with? Are you working with a low power device?
The values seems off, 10k and 10µF is slow, what you are seeing is a fast discharge (charge actually, the cap is discharged when going high and charged once the switch is released) till you are under the LED conduction voltage and then it keeps discharging with the 10k alone, that's why the shape isn't a simple time constant exponential discharge.
You can get away using a smaller cap, or a smaller resistor, but that resistor will chew more energy while the button is pressed, so if you are working with a low power device a smaller cap is better, but the LED could charge it too fast. You want the charge to be slow enough so the bouncing is faster than the discharge of the cap. I've got away with 4.7nF last week but it only connected a reset pin on a µC, with the LED that's likely to be too small.
Get a closer picture from your scope without the cap so you see the actual bouncing times, to get an idea, the longer one we call 'bounce time'. Then choose a time constant, 5 times longer than the 'bounce time' could be a ball park. Then choose a cap so your LED resistor/cap time constant match that. To finis the design pick a resistor that ends that discharge fast enough so you are happy with it, say 10 times the other resistor so you get 50 'bounce times' to 99% discharge.
JS