His father, grandfather and great-grandfather were guitarists. His uncle Phil was a band-leader and played trumpet.
His first instrument was the trumpet, which he began playing at age 8. A doctor was concerned that it would exacerbate his heart murmur. His family had three pianos, so he switched to piano.
He played piano with his uncle's band, in addition to groups led by Chico Marx, Joe Venuti, Charlie Barnet, and Gene Krupa. He was often featured with band-leader and trumpeter Louis Armstrong's All Stars.