Rich was born in Manhattan, New York, to Jewish vaudevillians Robert and Bess Rich. His talent for rhythm was first noted by his father, who saw that Buddy could keep a steady beat with spoons at the age of one. He began playing drums in vaudeville when he was 18 months old, billed as "Traps the Drum Wonder." At the peak of Rich's childhood career, he was reportedly the second-highest paid child entertainer in the world (after Jackie Coogan). At 11 he was performing as a bandleader. He received no formal drum instruction, and went so far as to claim that instruction would only degrade his musical talent. He also never admitted to practicing, claiming to play the drums only during performances and was not known to read music. He expressed great admiration for, and was influenced by, the playing of Chick Webb, Gene Krupa, Dave Tough, and Jo Jones, among others..
Rich first played jazz with a major group in 1937 with Joe Marsala and guitarist Jack Lemaire. He then played with Bunny Berigan (1938) and Artie Shaw (1939), and even instructed a 14-year-old Mel Brooks in drumming for a short period when playing for Shaw. At 21, Rich participated in his first major recording with the Vic Schoen Orchestra (the band that backed the Andrews Sisters). In 1938, he was also hired to play in Tommy Dorsey's orchestra where he met and performed with Frank Sinatra. In 1942, Rich left the Dorsey band to join the United States Marine Corps. He rejoined the Dorsey group after leaving the Marines two years later. In 1946, Rich formed his own band with financial support from Sinatra, and continued to lead different groups on and off until the early fifties.
In addition to Tommy Dorsey (1939–1942, 1945, 1954–1955), Rich also played with Benny Carter (1942), Harry James (1953-1956–1962, 1964, 1965), Les Brown, Charlie Ventura, and Jazz at the Philharmonic, as well as leading his own band and performing with all-star groups. In the early fifties Rich played with Dorsey and began to perform with trumpeter Harry James, an association which lasted until 1966. In 1966, Rich left James to develop a new big band. For most of the period from 1966 until his death, he led successful big bands in an era when the popularity of big bands had waned from their 1930s and '40s peak. In this later period, Rich continued to play clubs but he had stated in multiple interviews that the great majority of his big band's performances were at high schools, colleges and universities, with club performances done to a much lesser degree. Rich also served as the session drummer for many recordings, where his playing was often much more understated than in his own big-band performances. Especially notable were Rich's sessions for the late-career comeback recordings of Ella Fitzgerald and Louis Armstrong, on which he worked with pianist Oscar Peterson and his famous trio featuring bassist Ray Brown and guitarist Herb Ellis.