Louis C.K. does not have a brother in real life. He based his relationship with Robbie on his real-life friendship with Robert Kelly. As part of CK's lack of interest in full continuity, Seasons 2 and 3 were written as if Robbie had never existed at all (while Louie introduced three different women as his various sisters), only for Robbie to turn up in Season 4 as if he'd been in Louie's life all along.
According to C.K., his own mother had no problem with this portrayal because it was intended to have nothing to do with her; as he put it "she didn't relate to this woman and didn't feel like she supposed to." CK said that the portrayal of his mom in the Season 1 episode "God", where she is a thoughtful, caring woman who treats Louie very well, is exactly the kind of person his mom is, adding that he loves her a lot. CK said in another interview that the only thing "Dorethy" has in common with his real mom is that both of them are awkward around his kids, but in this episode that's due to his mom being an asshole, while in real life it's connected to her reserved personality.
A scene was filmed in which Louie asks Jasmine out and she agrees. But it was deleted because C.K. and editor Doug Abel felt that Louie should never succeed with women on the show.