"It would only make matters worse if the king refused to see her," a source said of Markle.