Councillors refused to back plans to demolish the old cinema building in Hobsons Street in order to make way for a new office building.