Khloé Kardashian Finally Opens Up About the Emotional Struggle of Growing Up with Kris Jenner as Both Mom and Manager — Why It Was So “Hard” to Differentiate Between Maternal Love and Business Pressure, How It Shaped Her Identity in the Spotlight, and the Hidden Toll of Mixing Family with Fame. What Did She Sacrifice? What Did She Learn? And Why Is She Choosing to Speak Out Now After Years of Staying Silent? This Candid Confession Pulls Back the Curtain on the Kardashian Empire Like Never Before — Prepare to See a Side of Khloé You’ve Never Seen Until Now. 

Khloe Kardashian/Instagram

Khloé Kardashian and Kris Jenner

Khloé Kardashian is admitting that she didn’t always love having her mom as her manager.

On this week’s episode of Khloé in Wonder Land, the Good American co-founder, 40, had her mom Kris Jenner, 69, on, and the two spoke about how their relationship has shifted in the many years their family has been in the public eye. As Jenner shares that she never could have predicted having her grandkids on their TV show, The Kardashians, Khloé notes that even their working relationship is now different.

“I guess I could also never have imagined it to even include another generation, which is so amazing. Now my grandchildren get to shoot with us, or film with us,” Jenner says. “Or even if they don’t want to film — be in the room. It’s the kind of thing where we can all work together.”

“I also think our dynamic at the beginning, it was challenging — I know more so for Kourt and I for you to be our manager at first. Because we were like, ‘What are you talking about? You’re not going to boss us around. We don’t even want to do this!’ ” Khloé remembers.

Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer​​, from celebrity news to compelling human interest stories.

Related: Khloé Kardashian’s Daughter True, 6, Thinks She and Tristan Thompson ‘Are Married’ — and She Doesn’t Want to Change That

“At first when you and Kim were like, ‘We’re going to film a show and you guys have to be on it,’ Kourt and I were like, ‘No we’re not. We don’t want to be on this show.’ You guys were like, ‘No, you’re going to be on the show,’ ” she explains of Jenner and sister Kim Kardashian.

Jenner nods her head, saying that she remembers telling Khloé and sister Kourtney Kardashian that they were going to do the show. Khloé says it just “sort of happened” and that they were told it’s only going to be one season, so to just do it.

“But then when you were like our manager and telling us what to do and where to go, it was hard for us to differentiate ‘mom’ and then a manager. It was hard for, I would say, the first few years,” Khloé candidly admits.

“To get the rhythm,” adds Jenner.

“To be like, okay, she’s ordering me around or telling me to do this, but not as my mom. Because we would be defensive or like, ‘What are you doing?’ It was, I think, Kim and you always had a very easy rapport and respect. And I had respect for you as my mom, but I was like, ‘You are not my boss,’ ” says Khloé.

“That was my attitude towards you until I had to realize, okay, this is manager Kris. I had to put you in two different categories. I think our relationship really became very seamless once I had kids. And it was always a great relationship, but the management/mom role was always very hard for me.”

Khloé notes that sometimes when she’d try to talk to her mom about manager stuff, Jenner would get defensive — and other times, Jenner would want to talk about work at the dinner table, which Khloé did not want.

“We definitely had to figure it out,” affirms Jenner. “The transition of that was definitely…”

“And when to turn it off,” adds Khloé. “That was always hard for me.”