Prince William reportedly asked some sheep farmers if they were ‘concerned about Brexit’ in a break from royal tradition