Alexander hailed from the Midwest; his father played violin, and the young drummer studied music at the Ray Knapp School in Chicago and privately in New York City with Sam Ulano. In the late '40s he began gigging professionally with Jimmy McPartland, moving over to the band of that leader's wife, pianist Marian McPartland, in 1952. The fascinating recordings of Sauter-Finegan from the mid-'50s often feature Alexander, but he really hit his stride collaborating with guitarist Smith on a series of small-combo dates around the same time. In 1956 he joined the Goodman band for a year, getting in on a Far East tour, where he worked up the gimmick of packing a miniature, battery-powered clip-on light just in case the mean-spirited bandleader decided to bust his regular music-stand light as a prank. In the late '50s the drummer worked with artists such as Bud Freeman and Eddie Condon, pledging allegiance to the swing school for the balance of his career.

Jazz discographers track Alexander on various jazz dates up through 1988, the same year as his death, despite having been partially paralyzed by a stroke in 1980. One of the best documents of his playing is a 25-minute film that was one of five such jazz performances commissioned by the Goodyear tire company in a series entitled Good Years of Jazz. This 1962 show features a band led by guitarist Mike Bryan that is packed with swing masters, including tenor saxophonist Georgie Auld, trumpeter Doc Severinsen, and vibraphonist Harry Shephard.