Since the show took place in a seedy, crime-laden city, the producers did not want to unfairly depict a US city. So they went to great lengths to avoid specifying where the show took place. There were many clues, though, that the series took place in the Midwest or Northeast.
Some clues include:
In the episode "Fuched Again", several officers take a plane trip to Las Vegas. When they run into Leo's Schnitz's estranged wife, she refers to them as "friend from back East." This would eliminate states such as Nevada and California.
In the episode "Gung Ho", a woman mentions committing crimes in Santa Fe, New Mexico, York, Pennsylvania and Newark, New Jersey. If the show was set in one of those states, she probably would not have used both the city and state name when mentioning the locations, thus ruling out New Mexico, Pennsylvania and New Jersey.
In the episode, "Hacked to Pieces", Mayor Cleveland's son, Lee, is desperate to get help for his addictions and wants to be sent to a special clinic in Rockford. Rockford, Illinois is located about 75 miles west of Chicago.
In the episode "Blood Money", a friend of Phil Esterhaus mentions that he has a cabin at Lake Lackawanna. Hill, Renko and Coffey overhear them talking and invite themselves to spend the weekend fishing there. Lake Lackawanna is part of a state park in Northeastern Pennsylvania. Thus it would make sense that the setting for the show was within driving distance of Pennsylvania.