Can't remember which song was playing when Da Gang strode out to the car in Two Cathedrals? Want to know the lyrics CJ lip syncs when she does The Jackal? Welcome to Ryo's Land of Popular Music!

Songlist by Episode
The Pilot Moment of Weakness
Post Hoc, Ergo Proptor Hoc She's Lost Control
Five Votes Down Happy Days Are Here Again
The Crackpots and These Women Ave Maria (Ellens Gesang III)
The State Dinner Hail to the Chief
Mr. Willis of Ohio Learn to Fly
In Excelsis Deo Little Drummer Boy
Take This Sabbath Day Hashkiveinu
Six Meetings Before Lunch The Jackal
Put on a Happy Face

In the Shadow of Two Gunmen, Part II Celebration
And It's Surely to Their Credit He Is an Englishman
Shibboleth We Gather Together
Noel Joy to the World
God Rest Ye Merry Gentlemen
What Child Is This
Praeludium, from the Suite #1 in G Major
Carol of the Bells

Bartlet's Third State of the Union Blame It on the Bossa Nova
Somebody's Going to Emergency, Somebody's Going to Jail New York Minute
Bad Moon Rising The Wells Fargo Wagon
18th & Potomac Suite for Orchestra No 3 in D Major (Air on a G String)
Two Cathedrals Brothers in Arms
Isaac & Ishmael For What It's Worth
Manchester, Part I Rock the Boat
Instead
Evolution Revolution Love
Manchester, Part II Columbia, the Gem of the Ocean
War Crimes You Make Me Feel So Young
Gone Quiet I'm Too Sexy
H.Con. 172 Cool
100,000 Airplanes The Surrey with the Fringe on Top
The Two Bartlets Red, Red Wine
Hartsfield's Landing Destiny
Dead Irish Writers O Canada
The U.S. Poet Laureate Makin' Whoopee
Enemies Foreign & Domestic Hooray for Captain Spaulding
The Black Vera Wang Don't Know Why
We Killed Yamamoto Caravan
Moondance
Posse Comitatus Hallelujah
Patriotic Song
20 Hours in America, Part I National Emblem March
Battle Hymn of the Republic
20 Hours in America, Part II I Don't Like Mondays
The Wanderer
College Kids One Week
Shed a Little Light
The Red Mass Gloria
Debate Camp Gaudeamus Igitur
Election Night Won't You Come Home, Bill Bailey
The Times They Are A-Changin'
Process Stories Love Me, My Love
Someone to Watch over Me
The House of the Rising Sun
Holy Night Silver Bells
Bye, Bye Blackbird
Girl from Ipanema
O Holy Night
Inauguration: Over There Heroes
Rock Me to Sleep
March of the Toys
Commencement Pomp & Circumstance
Angel
7A WF 83429 Sanvean
Abu El Banat We Three Kings
The Warfare of Genghis Khan Dark Was the Night, Cold Was the Ground
An Khe My Country 'Tis of Thee
The Supremes American Pie

Side A
American Pie
Don MacLean - American Pie, 1971
After getting Josh drunk, Senator Pierce moves onto CJ Cregg, who joins him in a drunken rendition of American Pie [5.17].
After a bittersweet meeting with Charlie at the National Arboretum, Zoey  goes to a graduation celebration at a local techno club.  What she thinks is lightheadedness due to champagne and mixed drinks turns out to be the ecstasy that Jean-Paul slipped into her drink.  As Josh rambles on about taking his pants off, Amy questions Donna's relationship with Josh, and Danny and CJ discuss the possibility of retaliation from terrorists for Abdul Shareef's death, a disoriented Zoey disappears to use the restroom.   A few minutes, her Secret Service Agent, Wesley Davis, finds Zoey's panic button and a dead agent in the alleyway, all the while, Massive Attack's eerie Angel plays in the background [4.22]. 
Ave Maria (Ellens Gesang III)
Composed by Franz Schubert, performed by Marian Anderson - Marian Anderson, 1939
Deputy Chief of Staff Josh Lyman is presented with a small card that will -- in case of a catastrophic event -- inform him how to get to safety. When he learns, though, that the card is just for him, not his staff, it triggers his survivor's guilt. He tells CJ Cregg that his sister, Joanie, used to listen to Ave Maria over and over again in her room. Joanie died in a fire when Josh was a boy, and he feels terribly guilty because he managed to get out of the burning house. While deciding to give the card back, Josh holes up in his office to brood and listen to Ave Maria. When CJ finds him obsessing, she talks some sense into him, telling him he can be "very sweet sometimes" [1.5].
Battle Hymn of the Republic
After the President leaves the stage at the Naval Warfare Center, a group of Navy men sing the Battle Hymn of the Republic [4.1].
After an exhilarating turn on Capital Beat, Ainsley retreats to her office, fixes herself a Pink Squirrel, and cranks up this tune from the early 1960s. When Sam asks her what she's doing, she tells him that she's blaming it on the bossa nova, and beseeches him to dance with her. He refuses, and Ainsley's impromptu song and dance number is cut short when the President arrives unexpectedly, remarking that he never knew they had nightclub in the building [2.13].
After the chaos of the President's disclosure of his illness and the sudden, tragic loss of Mrs. Landingham, the staffers are noticeably sober as they ready themselves for the press conference, where they expect the President to announce that he does not plan to run for re-election. They meet up in the hallways of the West Wing, trench coats on, mouths set with grim determination as they walk out into the rainstorm and climb into the motorcade [2.22].
Bye, Bye Blackbird
Written by Mort Dixon and Ray Henderson, performed by the Whiffenpoofs - The Natural, 2002
The Whiffenpoofs, an a capella bunch of Yalies, performed the standard Bye, Bye Blackbird in the Mural Room on the day before the day before Christmas of 2002.  CJ thinks that she'd like it if they took her home.  Just to, you know, sing to her [4.11].
While Josh Lyman talks to Donna Moss on the phone, Amy Gardner cranks up Van Morrison and dances around her apartment, grooving to the music [3.21].
Carol of the Bells
Steve Banks - Winter Peace, 1999
After an intense day with therapist Stanley Keyworth, Josh Lyman and Donna Moss emerge from the White House and are confronted with a troupe of carolers singing Carol of the Bells.  Josh, whose PTSD episodes are triggered by music -- which he equates with sirens -- is momentarily entranced, watching the carolers until Donna gets his attention again, and the two head for the hospital to get Josh's hand patched up [2.10].  In an interesting note, this same song was performed by what I'm told is the same troupe (in this case, they were playing the recovery staff) in the sequel (A Miracle Happens Here) to the acclaimed E.R. episode Love's Labour Lost in which Bradley Whitford appeared.  Though Whitford is in Miracle, his appearance is brief and has nothing to do with the song.  Drat.
Though Josh Lyman called for the Doobie Brothers, it was Kool & the Gang's most famous song that blasted throughout the campaign suite after Jed Bartlet won the 1998 Illinois primary. In response, CJ Cregg and Sam Seaborn begin to dance (rather badly) together, lapsing into a very strange version of the robot, while in the foreground, Donna Moss has the grim task of informing the ecstatic Josh that his father has died [2.2].
Columbia, the Gem of the Ocean
Composed by Thomas Beckett, performed by RCA Victor Sympathy Orchestra - God Bless America, 2002
Before the rally for President Bartlet that's meant to kick of his re-election campaign, the Columbia band practices Columbia, the Gem of the Ocean.  Bruno is not amused that the Bartlet staffers actually negotiated with the band and wants them to play songs of the campaign's choosing [3.2].
Cool
Music by Leonard Bernstein, lyrics by Stephen Sondheim, performed by the Original Cast - West Side Story, 1961
When Sam Seaborn is up in arms over a soon-to-be-published book that purported to expose the Bartlet White House as undisciplined and faltering, CJ Cregg sings a few bars of the song Cool to him [3.10].
After stargazing with an attractive NASA administrator, Josh tells Donna the tragic story of Blind Willie Johnson, who died homeless, but whose song, Dark Was the Night (Cold Was the Ground), is included in Voyager, the space probe that just exited our solar system.  The song played as Josh took his brand new telescope out for a test drive, while President Bartlet brooded and rewatched satellite footage of an atomic bomb explosion in the Situation Room [5.13].
Not long after the scandal over Jed Bartlet's revelation that he has M.S., Donna Moss meets an old friend for a drink and is quite unexpectedly offered a job at CapitolScoop.com, his upstart website.  Her friend Casey considers her time working for Josh Lyman better than a Master's degree.  Interestingly, though the song playing in the background is called Destiny, Donna apparently turns down the job [3.14].
CJ Cregg is the cool aunt.  She takes her niece, Hogan, shopping for a prom dress at Barney's, and is herself entranced by a black Vera Wang.  Hogan and Simon Donovan discuss Rosslyn, all the while, Norah Jones jazzy Don't Know Why plays in the background, a song which is about a missed rendezvous with a lover.  Foreshadowing, perhaps? [3.20]
Evolution Revolution Love
Tricky with Ed Kowalczyk - Blowback, 2001
The trip-hop of Tricky plays as Toby Ziegler mocks Charlie Young's presumed inability to play pool ("I understand if you're scared chicken.") Charlie accepts the challenge, and after snarking Toby a little about the speech he's supposed to be writing, says "You give me the break, you're not going to need that stick." Next time we see these two, a dour Toby remarks "Nice game. It was fun to watch" [3.1].
After the horrific events of September 11, 2001, Aaron Sorkin hastily wrote and produced a special episode, which was referred to in its introduction (voiced by the actors) as a one-act play. Essentially, Sorkin worked through some of his confusion by using the familiar characters of The West Wing.  The set up was simple: a group of high school kids visiting the West Wing end up stuck inside when the secret service has to lock down the White House; to kill time, Josh Lyman starts a discussion which centers on terrorism.  Every major character shows up to weigh in, reflecting the contemporaneous real-life dialogue in newspapers and talk shows.  Over the credits sequence at the end, Sorkin chose For What It's Worth. Interestingly -- as Isaac & Ishmael was produced and aired before the United States began its bombing campaign in Afghanistan -- this song is largely about pacificism and protest against governmental militarism, with lyrics like There's battle lines being drawn/Nobody's right if everybody's wrong [3.0].
While trapped in Faith, North Carolina at Debate Camp, Sam, Larry, and some other former campers entertained their colleagues with a rousing rendition of Gaudeamus Igitur, a gem of the 18th century.  Luckily for the guys, they sang it in Latin, so CJ didn't have to kick their asses for singing lyrics like Long live all maidens/Easy and beautiful/Long live mature women also/Tender and loveable/And full of good labor [4.5].
Girl from Ipanema
Written by Antonio Carlos Jobim, performed by the Whiffenpoofs - Hi-Fi, 2001
When Donna requested the Whiffenpoofs sing her a song somehow related to the ambience of the Washington Inn, where she is set to spend the holidays with Jack Reese, they choose (and Charlie independently suggests) Girl from Ipanema. [4.11].
After Jed Bartlet dismisses Charlie's valid question about the Red Mass -- an apparently Catholic service held in honor of the Supreme Court Justices the night before they start a new term -- he and several of his staffers attend the service.  The glorious music playing during the processional is Vivaldi's Gloria [4.4].
God Rest Ye Merry Gentlemen
Performed by the Pittsburgh Symphony Brass - Christmas Concert, 2000
During Toby Ziegler's quest to avoid being called names for not displaying the appropriate holiday spirit, he enlisted a brass quintet to play carols in the Northwest Lobby.  As Donna Moss warns Josh Lyman not to be a yutz, the quintet plays God Rest Ye Merry Gentlemen in the background [2.10].
Hail to the Chief
Hail to the Chief, 1996
I was right. Hail to the Chief, the official theme song of the United States Presidency, has played at least once on the show.  Props to Dan and Kasey (Yay!  Inadvertent Sports Night reference!) for letting me know that just before Mandy Hampton runs off to toss her cookies, she's asking Josh if there's any news on a hostage situation in Idaho.  Josh gets distracted by the arrival of the President and First Lady (along with the President and First Lady of Indonesia) scored, of course, to Hail to the Chief [1.7].  Now the fun, interactive part:  Anyone who can tell me another episode in which this song was played will get virtual cupcakes.
Hallelujah
Written by Leonard Cohen, performed by Jeff Buckley - Grace, 1994
After CJ Cregg's mysterious stalker is caught, she and her erstwhile Secret Service agent, Simon Donovan, decide to explore the attraction between them.  While she attends an excessively long play with the president, Simon goes to a nearby convenience store for a candy bar and flowers for CJ.  He ends up in the middle of a robbery, and is shot and killed as Jeff Buckley's transcendent Hallelujah plays [3.22].
Mandy Hampton, while she is still relatively new working at the White House, allows the President to choose the music he'd like to be played as he left the stage after a rousing speech. The problem, as Sam Seaborn points out, is that the speech reaches its climax with the words, "Kids are dead! Kids are dead!" which makes for a rather stark contrast with the upbeat song [1.4].
Hashkiveinu
Arranged by Max Helfman (source:  W.G. Snuffy Walden) - Rossi: The Songs of Solomon, Volume I, 2000
While Toby Ziegler speaks with his Rabbi about the death penalty, a cantor practices Hashkiveinu in the background.  Rabbi Glassman admits she didn't need to practice just then, and calls her his 'communications director' [1.14].
After a running thread through And It's Surely to Their Credit about Gilbert and Sullivan, Sam Seaborn, CJ Cregg, Josh Lyman, and Toby Ziegler gather in Ainsley Hayes' basement office to serenade her with Englishman. Ainsley's experience on her first day culminated in two staffers leaving dead flowers on her desk with a note reading "BITCH" (they were fired immediately thereafter by Sam Seaborn), and so the Senior Staffers take it upon themselves to hang several posters along the basement hallway leading to her office and greet her in song. Well, Toby reorganizes her highlighters while the others sing, but it's the thought that counts [2.5].
Sometimes Josh, CJ, and Toby head over to a club called Iota in Arlington because they usually have good artists.  Two days before the Inauguration, they needed to get out of the White House, but they couldn't get past the subject:  genocide.  As Jill Sobule played her wickedly appropriate song (Why are all our heroes so imperfect.../FDR didn't save the Jews), CJ argued with Toby, making the memorable point that we don't want to send somebody's kids into danger halfway around the world, but we should, because the people being slaughtered are somebody's kids too [4.15].
While discussing which nations -- Slovenia, Estonia, Latvia, and Lithuania, Sam suggests -- should be admitted to NATO, CJ makes a joke about Fredonia.  After no one laughs, CJ sings a line from Hooray for Captain Spaulding [3.19].
The House of the Rising Sun
The Animals - The House of the Rising Sun, 2002
After Sam likens CJ's lava-lamp office decor to a brothel, the gathered staffers including Carol, Ed, Larry, and Ginger -- presumably having downed some celebratory champagne after Bartlet's re-election -- end up singing a The House of the Rising Sun.  Until CJ shushes them [4.8]. 
Much thanks to Jo March, Margeurite, Kat, Lisa R., Kimberly, Brian, Dana, Ginny, Anne, Stargirl, Julia, Marina, KDTB, Abby Dan, and Kasey for providing timely answers, song titles, and information.

The wwwhores have a page of music from The West Wing as well.