Their fourth album, a prophetic masterpiece and best-selling work, made them stars – but its creation wasn’t easy