Melvin Rhyne was born in Indianapolis in 1936 and started playing the piano shortly thereafter. At 19 years old, Rhyne started playing piano with then-unknown tenor saxophonist Rahsaan Roland Kirk but quickly switched over to the instrument that would make him famous: the Hammond B3 organ. Rhyne's piano skills translated to the organ fluently and before long he was backing famous blues players like B.B. King and T-Bone Walker. In 1959 when he was asked to join fellow Indianapolis musician Wes Montgomery's newly formed trio.
Rhyne then moved to Wisconsin and largely kept to himself for the next two decades. In 1991, however, Rhyne returned to the jazz scene, playing on Herb Ellis' album Roll Call, Brian Lynch's At the Main Event, and his own comeback The Legend. Rhyne continued to be prolific in the years to come, releasing eight more solo albums on the Criss Cross jazz label.