Yes, but do keep in mind those are for 200mA.
200mA is kinda big current draw for the zinc chemistry, but they may behave better at 10-50mA compared to this graph. Alkaline batteries may not do much better.
Either way, you should be able to find LEDs with forward voltage less than 2.5v, which should allow you to use just a couple of AA batteries. Otherwise, yeah, I guess 4 AA or AAA batteries would definitely do the trick.