The city in which the game takes place is never actually stated. However, the design team used Philadelphia as inspiration. As David Cage said in an interview with G4TV: 'We spent a tough two weeks exploring the city with a movie scout taking us to the worst places, the poorest areas, and the abandoned factories, discovering another side of America that you dont often see in Hollywood movies.'
So, although not actually set in Philadelphia, Heavy Rain's setting and atmosphere is heavily inspired by the city.
Edit: In the Move Edition of Heavy Rain, a narrator confirms Heavy Rain is in fact set in Philadelphia.