The fourth of seven children, John Royce Mathis was born on September 30, 1935 in Gilmer, Texas to Clem and Mildred Mathis. As a small boy, the family moved to Post Street in San Francisco. It was there that he learned an appreciation of music from his father who taught him his first song, “My Blue Heaven”. At age eight, his father purchased an old upright piano for $25. When he brought it home, it wouldn't fit through the front door. So that evening, Johnny stayed up all night to watch his father dismantle the piano, get it into the small living room of their basement apartment and then reassemble it. Clem Mathis, who worked briefly as a musician back in Texas playing the piano and singing on stage, would continue to teach his son many songs and routines. Johnny had proven to be the most eager of the children to learn all about music. He sang in the church choir, school functions, community events, for visitors in their home as well as amateur shows in the San Francisco area.
Johnny was 13 years old when Clem took him to see Connie Cox, a Bay Area voice teacher, who agreed to take on the youngster in exchange for his doing odd jobs around her house. Johnny studied with Connie for six years learning vocal scales and exercises, voice production, classical and operatic skills.
At George Washington High School, Johnny was known not only for his singing ability but his athleticism as well. He became a star athlete on the track and field team as a high jumper and hurdler and played on the basketball team.