Johanson came up in the R&B world and began drumming at an early age, often accompanied by friend Lamar Williams on bass. Johanson backed soul singers, including a membership in Otis Redding's touring band in 1966, and afterward touring with the acclaimed soul duo, Sam & Dave. After joining up with Duane Allman in February 1969, he quickly became the first recruit into Allman's new group, soon joined by bassist Berry Oakley, fellow drummer Butch Trucks, guitarist Dickey Betts and lastly Allman's younger brother, singer, organist and pianist Gregg Allman. The group, quickly named after the brothers Allman, began recording demos that April in Macon, Georgia, which became the group's spiritual home.
The band's mixture of blues, jazz and rock, spearheaded by the dual lead guitars of Betts and Allman, and the double-drums of Trucks and Jaimoe, were unique at that time, and they rapidly became known as an act that you had to see live. Their first two albums, their eponymous debut (November 1969) and Idlewild South (September 1970) brought positive critical reviews but only limited commercial success. Their third album, however, recorded live in March 1971 at one of their favorite concert halls, Bill Graham's Fillmore East in New York City in March 1971, made them one of the biggest rock acts in America. At Fillmore East became a RIAA certified gold album in late October 1971, finally bringing the group the chart success that had eluded them. The band quickly suffered tragedy, however. Duane Allman was killed in a motorcycle accident a few days later. Shaken by the loss of Allman, the group soldiered on and released Eat A Peach, which reached #4 in the Billboard charts in 1972, a hybrid studio and live album, with outtakes from the Fillmore East concerts and studio cuts both with and without their original leader.