Rather than thrive on being a celebrity offspring, the Nashville actor went as far as to say he despised aspects of it.
WATCH: Oliver Hudson looks like Steve Jobs in hilarious new video
"I hated it when people would come to the table and ask for their autograph," he said of fans requesting to meet Goldie and Kurt while they were out and about. "It made me angry. Leave us alone!"
He continued: "Or when we would walk through the airport, and at the time people would jump out and take our photos. I hated it."
His younger sister, Kate, however, was the total opposite, according to Oliver. "Whereas Kate was posing," he joked. "She was like 'oh my god they're here!"
Oliver and Kate were raised by mum Goldie and her partner Kurt Russell
Goldie and Kurt never married but have been together for 37 years and their family includes her two children, Oliver, 44, and Kate, 41, from her first marriage to Bill Hudson, Kurt's son, Boston, 40, from a previous relationship, and son, Wyatt, 34, who they had together.
They are an incredibly close-knit family and Kate previously opened up about the first time Kurt met her and Oliver following a date with Goldie. What's more, the way Kurt cared for Kate and Oliver impressed Goldie – and won her heart as a result.
The Almost Famous actress was only three at the time and said: "Mom said that's when she fell in love with Kurt, right away was because she brought him back to the house, and we had already been asleep.
The entire family are close
"And we shared a room and we were sleeping, and Mom said he walked in and he saw us and she unexpectedly was just going to peek in and see them, but he sat down next to each of us. He sat down next to you first and watched you sleep and then came by."
Kate added, "I loved that story ‘cause she realized then, as Mom says, that values and kids and family values was everything to Kurt."
Read more HELLO! US stories here
Like this story? Sign up to our newsletter to get other stories like this delivered straight to your inbox.