The one-take master shot scene of Lenny doing his act in a raincoat near the end of the film came from an actual tape recorded from a show he did that was sent to Dustin Hoffman by a student who saw the show. Lenny's exact words in the scene on stage are verbatim from the tape of his show.
The character of Sherman Hart is based on Milton Berle; the real name wasn't used because Berle was still alive and there were fears he might sue for libel. But the character uses all of Berle's mannerisms and trademark shtick, including sticking the cigar under his top lip.
Cliff Gorman, who'd won the 1972 Tony Award for his portrayal of Bruce on Broadway, was considered for the film adaptation but wasn't deemed as bankable an actor as Hoffman. Years later, Gorman would play a Lenny Bruce type of character in All That Jazz (1979), also directed by Bob Fosse.