Elms was born in Newington, Kent, England, and died in Southampton. He joined the Royal Marine Band Service at Deal in 1934, took part in the Vaagso raid in 1940 and served on the Malta Convoys in 1941-42.
After Elms left the Royal Marines he joined Francis Day and Hunter the music publishers as a music arranger. His break into television came with the launch of commercial television in Britain during 1955. He made a contribution to ITV's The Adventures of Robin Hood (1955–59) with Richard Greene in the lead, writing the musical scores for at least two series. He also wrote musical scores for The Buccaneers (1956–57), starring Robert Shaw, The Adventures of Sir Lancelot (1956–57), starring William Russell, and Ivanhoe (1958–59) with Roger Moore. However, Elms is best known for providing incidental music for later ITC series such as Man in a Suitcase (1967–68), The Champions (1968-69) and for 14 of the 17 episodes of The Prisoner (1967–68) starring Patrick McGoohan.
Although the series on which Elms did most work had memorable main title scores by others such as Ron Grainer and Tony Hatch, Elms provided cues which represent hundreds of hours of music. His contribution has recently been recognised with the release of his complete incidental music for Man in a Suitcase on CD.
On occasions he also wrote complete film scores for movies such as Satellite in the Sky (1956), The Man Without a Body (1957), On the Run (1958), Bluebeard's Ten Honeymoons (1960), The Breaking Point (1961), The Omegans (1968), and the film version of Love Thy Neighbour (1973).
Elms also served as musical director of The Benny Hill Show for three episodes of the 1973–74 season, substituting for Ronnie Aldrich. He wrote the Corps quick march, On Parade, for The Royal Logistic Corps which formed in 1993. He lived for more than 40 years at the family home in the village of Oad Street near Borden in Kent.
Albert Elms had three sons Martin, David, and Jeremy. His wife Jo predeceased him in 1990.
- "Albert Elms: light music composer" (obituary). The Times. 11 November 2009. Retrieved 12 November 2009.