He blames the fight against the press as a ‘central piece’ in destroying his relationship with his family