Bailey was born in Sheffield, England. A third generation musician, he began playing the guitar at the age of ten, going on to study with John Duarte among others. As an adult he found work as a guitarist and session musician in clubs, radio, dance hall bands, and so on, playing with many performers including Gracie Fields, Bob Monkhouse and Kathy Kirby, and on television programs such as 'Opportunity Knocks'. Bailey was also part of a Sheffield based trio founded in 1963 with Tony Oxley and Gavin Bryars called 'Joseph Holbrooke' (named after the composer, whose work they never actually played). Although originally performing relatively 'conventional' jazz this group became increasingly free in direction.
Bailey moved to London in 1966, frequenting the Little Theatre Club run by drummer John Stevens. Here he met many other like-minded musicians, such as saxophonist Evan Parker, trumpet player Kenny Wheeler and double bass player Dave Holland. These players often collaborated under the umbrella name of the Spontaneous Music Ensemble, recording the seminal album Karyobin for Island Records in 1968. In this year Bailey also formed the Music Improvisation Company with Parker, percussionist Jamie Muir and Hugh Davies on homemade electronics, a project that continued until 1971. He was also a member of the Jazz Composers Orchestra and Iskra 1903, a trio with double bass player Barry Guy and trombone player Paul Rutherford that was named after a newspaper published by the Russian revolutionary Vladimir Lenin.