Born in Vienna as the son of a teacher, Gulda began learning to play the piano from Felix Pazofsky at the Wiener Volkskonservatorium, aged 7. In 1942, he entered the Vienna Music Academy, where he studied piano and musical theory under Bruno Seidlhofer and Joseph Marx.
He won first prize at the Geneva International Music Competition in 1946. Initially, the jury preferred the Belgian pianist Lode Backx, but when the final vote was taken, Gulda was the winner. One of the jurors, Eileen Joyce, who favoured Backx, stormed out and claimed the other jurors were unfairly influenced by Gulda's supporters. Gulda began to play concerts worldwide. He made his Carnegie Hall debut in 1950. Together with Jörg Demus and Paul Badura-Skoda, Gulda formed what became known as the Viennese troika.