Occasionally, between 1982 and 2001, Zevon filled in for Paul Shaffer as bandleader on Late Night with David Letterman and later The Late Show with David Letterman. One example was in 1998 when Shaffer traveled to Canada to film his cameo in Blues Brothers 2000.
On October 30, 2002, after Zevon's terminal mesothelioma diagnosis, he was featured on The Late Show with David Letterman as the only guest for the entire hour. The band played "I'll Sleep When I'm Dead" as his introduction. Zevon performed several songs and spoke at length about his illness. Zevon was a frequent guest and occasional substitute bandleader on Letterman's television shows since Late Night first aired in 1982. He noted, "I may have made a tactical error in not going to a physician for 20 years". It was during this broadcast that Zevon first offered his oft-quoted insight on facing death: "Enjoy every sandwich". He also took time to thank Letterman for his years of support, calling him "the best friend my music's ever had". For his final song of the evening, and his final public performance, Zevon performed "Roland the Headless Thompson Gunner" at Letterman's request. In the green room after the show, Zevon presented Letterman with the guitar that he always used on the show, with a single request: "Here, I want you to have this, take good care of it".
On April 18, 2008 Jordan Zevon appeared on The Late Show with David Letterman and again on June 7, 2008.