Paul McCartney is looking back on his incredible career.

On Wednesday, the former Beatles singer sat down with Howard Stern on his SiriusXM show to talk about his career, and admitted to having had a testy relationship with the late John Lennon

“That’s true, but I’d swap it all out for him to be alive,” he said. “Sometimes you’d get annoyed back, but not often … You’d just go, ‘That’s John — what a d**k.’”

He continued, “After the breakup of the Beatles, there were some very sad moments for me in there. You would get really down and, you know, I’m sure I cried a few times going, ‘What the f**k?’ … ‘Why did we have to get like this?’”

READ MORE: Paul McCartney Describes How He And John Lennon Rebuilt Their Friendship After Bitter Breakup Of The Beatles

Remembering one of their final conversations, McCartney said the two reconnected over the bread shortage that had been going on in the U.K. in the ’70s.

“I was baking bread and got quite good at it, so when I heard John was doing it, it was great… we could just talk about something so ordinary,” he recalled. “It was really nice, and I was so glad that we got back to that relationship that we always had when we were kids.”

McCartney also revealed that he considers “I Saw Her Standing There” to be one of, if not the, best song he ever wrote.

Looking back, he remembers the original version of the lyric “She was just 17 / You know what I mean” was actually “Never been a beauty queen,” but he realized it didn’t work after playing the song for Lennon.

“I was like ‘Oops, this is not good,’” McCartney said. “Years later, I was getting a star on the Hollywood Walk of Fame and Neil Young was there … and I told him that story. He was playing that night … and he did the song and he used that line, of course. He was the only one to ever use that line, I think.”

READ MORE: Paul McCartney Joins Foo Fighters On The Beatles’ Get Back’ At Rock & Roll Hall Of Fame Ceremony

Also during the interview, McCartney explains why he was originally hesitant about bringing “Yesterday” to The Beatles.

“It was a little bit embarrassing. I didn’t want to be the guy who’s out on the stage on his own because there’s a lot of comfort being in a group of mates,” he said. “When I brought it in it was just me, solo guitar, and that was it. The guys just said, ‘Well, we can’t put drums on that… and the one guitar is doing enough.’ [We] said, ‘No we can’t put that out, we’re a rock group.’ We liked it and stuff, but it wasn’t a big feature of our stage act.”