The 33-year-old, who has two children with husband Joel Madden, a daughter named Harlow and a son called Sparrow, has admitted that she is happy with the current size of her family.
Nicole Richie with her children Harlow and Sparrow
"If it happens, it happens," she told Sirius radio. "Of course, I'll be so excited, but I'm not necessarily planning. I'm so happy with what I have and I had more of course I'd be so happy. But right now, I'm good."
Nicole also revealed that she recently visited a friend and realised that she had forgotten the difference between looking after a baby compared to her growing children.
"My kids are five and six, and I held by friend's baby and the head was wobbling around," she said. "I remembered what it was like, 'Oh I have to watch this kid that I'm holding. I have to watch its neck!'"
"I gave it back to my friend," she added. "I was like, 'Take this baby!' I am not in the right head space to be taking care of a baby right now."
Nicole has been married to Joel Madden since 2010
The stunning socialite has previously spoken about parenthood, revealing that she believes that "kids want to be told the truth" in an interview with Oprah Winfrey on Oprah: Where Are They Now?
"I have to always come from a place of love and compassion with them," said Nicole. "My kids operate best when I let them have a voice and I really explain things to them. Because, really, kids want to be told the truth. They just want you to be truthful."
The mother-of-two has made no secret of her struggles growing up in a famous family. Singer Lionel became her guardian when she was three and officially adopted her when she was nine.
Although Nicole was involved in run-ins with the law during her early years in the spotlight, she has transformed her image since becoming a mother.