Damon Wayans Jr. and his father remember the days when Jim Carrey would jump onto their car windshield

At that time, Carrey was "just wild," recalled Damon Jr., 41, appearing alongside his dad on a recent episode of SiriusXM's program The Spotlight With Jessica Shaw.

When Damon Sr., 64, quipped that his kids "thought Jim Carrey was funnier than me," his son joked that the Liar Liar star was "just more desperate," remembering a specific anecdote.

Related: One Last Thing with Damon Wayans: The Iconic Costume He Kept from In Living Color (Exclusive)

"There was one time we were coming home," Damon Jr. continued, "and I think I was like 5, 6 years old, maybe 7, and we were waiting for the gate to open. And Jim Carrey drops out of the tree onto the windshield of our car and slides down, like his face is just smeared down."

Laughing he added, "So that's why he was so funny 'cause he was just wild... That was one of my earliest memories of him."

As for how Damon Sr. reacted to his colleague appearing out of nowhere onto the family car, Damon Jr. added, "You turned on the windshield wiper."

Never miss a story -- sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from celebrity news to compelling human interest stories.

Related: Hollywood's Famous Acting Families, From the Hemsworths to the Hudsons

Carey starred alongside Damon Sr. on In Living Color from 1990 to 1994 (Damon Sr. and siblings Kim, Shawn, and Marlon left midway through season 4 due to disputes with the Fox network over censorship). Before that, the two played furry aliens alongside Jeff Goldblum in Earth Girls Are Easy, which also starred Geena Davis.

Damon Sr. and Damon Jr. star together as a father-son duo in Poppa's House, which premiered its first episode on CBS Wednesday, Oct. 21. The sitcom was co-created by Damon Sr. and Kevin Hench.