According to a senior royal source, the meeting was "very emotional", with Prince Charles and Camilla delighted to spend time with Harry and Meghan and their two children.
The couple travelled to London from their California home just before Lili's first birthday, enjoying some of the Queen's Platinum Jubilee celebrations as well as spending private time with their family.
WATCH: Prince Harry discusses his and Meghan Markle's visit with the Queen
Speaking at the briefing of Clarence House's annual review, the source said of the pair: "It was fantastic to see them. It was wonderful to have them back in Britain. The Prince and the Duchess were absolutely thrilled to see them.
"The Prince, of course, hasn’t seen his grandson Archie for a bit of time and so it was very, very, very special to have some time with him.
Little Lili turned one during her family's trip to the UK
"He hadn’t met Lili, his granddaughter, and so to meet her was very emotional, a very, very wonderful thing."
Harry and Meghan appeared at the Jubilee service of thanksgiving in St Paul's Cathedral and were also photographed watching Trooping the Colour the previous day.
Harry and Meghan leaving the Jubilee service of thanksgiving
They did not, however, appear with other royals on the Buckingham Palace balcony, as that honour was reserved for senior working members of the royal family.
Prior to the event, a spokesman for Buckingham Palace said: "After careful consideration, The Queen has decided this year's traditional Trooping the Colour balcony appearance on Thursday 2 June will be limited to Her Majesty and those members of the Royal Family who are currently undertaking official public duties on behalf of the Queen."
Harry and Meghan stepped back from official duties in early 2020.
Make sure you never miss a ROYAL story! Sign up to our newsletter to get all of our celebrity and royal news delivered directly to your inbox.