He was born via a scheduled C-section on 29 August and weighed in at 7 pounds and 10 ounces, their rep told PEOPLE.
It is thought that the baby's unsual name has been inspired by Guns N' Roses rocker Axl Rose.
The new mum, 38, is a big fan of the band and even performed Sweet Child O' Mine with guitarist Slash during the Black Eyed Peas' Super Bowl halftime show in 2011.
In July, the 38-year-old celebrated her pregnancy with a fabulous 'gayby shower', so called because the party was organised by four of her gay friends – her agent Sean Patterson, designers Dean and Dan Caten and hairstylist Andy LeCompte.
Also there feting the expectant mama was Kelly Osbourne, accompanied by her fiancé Matthew Mosshart.
Back in May the Black Eyed Peas star spoke about what starting a family meant to her and Josh.
"We're thrilled," she said. "When that moment occurs to you, there's going to be a human being growing inside of you... it's reality. It's amazing ; we're thrilled."
This summer, Fergie, whose real name is Stacy Ferguson, applied to change her name legally to Fergie Duhamel. That way she and her baby will have the same surname.
Love blossomed for the pair on the set of his TV series Las Vegas when she had a cameo appearance. They married in 2009.
Josh, 40, has been just as excited as his wife. When the actor appeared on US chat show LIVE with Kelly and Michael a few weeks ago he was presented with two babygros – one pink and one blue – asked to pick one.
After playing coy for a few seconds, he grabbed the blue one, revealing the sex of their child.
The Tinseltown favourite is longing to be a dad and is trying to reorganise his schedule around their son.
"I plan on taking as much time as I can off before and as much time as possible after. I can't imagine having a kid and then not being able to be there — that would just drive me crazy," Josh said ahead of the birth. "My head would be at home with mum and baby.
"We want to work it out so that we can be together; whether I am working, they can come with me, or if she is working I can go with them. I can't see myself ever wanting to be away from my wife and kid ."