The Rolling Stones’ secret weapon throughout their golden period looks back on his time with the band