After discussing local issues in a late-night-style monologue, doing shots with Michelle and Kaye, and trying Jeff’s viral Colbert Questionert sandwich, the episode ended with Stephen, Jeff, and Jack destroying the show’s set and tossing everything into a dumpster. Before lighting up the bin on fire, the trio got the all-clear from “fire marshal” Eminem (real name Marshall Mathers).

While Stephen’s appearance on TV less than a day after the final Late Show episode came as a surprise, it shouldn’t have. After all, the comedian himself told his viewers they might find him on the local program.

“Tonight is our final broadcast from the Ed Sullivan Theater,” he told his audience during his last monologue on the May 21 show. “We were lucky enough to be here for the last 11 years, all right? Can’t take this for granted.”

He continued, “Though technically our first show in July of 2015 was from a public access station in Monroe, Michigan, for an audience of 12 people.”