Barretto was born on 29 April 1929, in the Brooklyn borough of New York City, of Puerto Rican parents. Noted for his many years as a prominent Latin bandleader, his music career actually began as a studio performer on the conga for jazz recording sessions.
He was raised in the Latin ghettos of East Harlem and the Bronx, in an environment filled with music of Puerto Rico but with a love for the swing bands of Ellington, Basie and Goodman. He escaped the ghetto by joining the United States Army when he was 17 years old, but he did not escape the music.
Influenced by a record of Dizzy Gillespie, “Manteca”, with conguero Chano Pozo. He was hooked and he knew then that his calling was was to become a professional musician. Barretto sat in on jam sessions held at the Orlando, a GI jazz club in Munich, Germany. After military service in 1949, he returned to Harlem and taught himself how to play the drums.
Barretto’s first regular job was with Eddie Bonnemere’s Latin Jazz Combo. He then went on to play for four years with Cuban bandleader/pianist Jose Curbelo. In 1957, Barretto then replaced Mongo Santamaria in Tito Puente‘s band, with which he recorded nis first album: Dance Mania. After four years with Puente, he was one of the most sought-after percussionists in New York City’s thriving music scene. He attended jam sessions with notable artists such as Max Roach, Charlie Parker, Art Blakey and other jazz giants. He also recorded with Sonny Stitt, Lou Donaldson, Red Garland, Gene Ammons, Eddie “Lockjaw” Davis, Cannonball Adderley , Freddie Hubbard, Cal Tjader, Dizzy Gillespie, and others.