The problem with a tagline like "Father of the computer" is vagueness. What do you call a computer?
During the Manhattan Project, calculations were carried out by hundreds of women sitting at mechanical tabulator machines. The machines weren't computers, but the operators were. The use of the word "computer" to describe these people predates its use to describe machines. The dream of "automatic calculators" that could evaluate complex formulae faster than a human operator was around since the 1830s; Babbage's "engines" were computers of that sort. The dream next awoke in the 1930s, when several engineers, working independently, created electromechanical machines that were designed to solve specific problems. Zuse and Aiken read instructions from paper tape to perform math on accumulators; Stibitz read complex numbers from a telegraph and did complex arithmetic; Atanasoff solved linear equations stored on a drum.
Despite these forerunners, the ENIAC was the first computer that could be programmed for more or less any purpose. Maybe "programming" deserves scare quotes: it was not a series of instructions, but a maze of patch cables connecting different components. The "programmers" needed to understand the entire machine in minute detail, connecting gates and registers in a manner that could solve a problem. They were really doing hardware design without a HDL, since everything happened in parallel.
http://eniacinaction.com/After ENIAC was built, Eckert and Mauchly applied the concept of an instruction sequence (as used by Aiken) to be stored in electronic memory, to their work on the EDVAC. Later on, their names were deleted from the Draft Report On The EDVAC, written by von Neumann. That is how the stored program concept came to be called the von Neumann Architecture.