Homicide is based on a non-fiction book "Homicide: A Year on the Killing Streets" by David Simon. In the 1980s, Simon was a reporter for the Baltimore Sun, working the crime beat. He spent the year of 1988 "embedded" in one shift of the BPD Homicide unit and chronicled the cases they investigated.
Fans of the show like to consider it one of the more accurate depictions of police investigation. However, like any work of fiction, Homicide diverges from reality in a number of ways.
Many of the elements, cases, and even quotes, used on the show are ones taken directly from actual events which did happen, or were reported to have happened, in the Baltimore homicide unit. For example, the unit did use a dry erase board to chronicle clearance rates for detectives. Due to publicity from Homicide, the practice was temporarily discontinued as it was felt that it projected an image of a cold obsession with clearance rates. But the board was later brought back as the BPD transitioned into a statistics based model of policing which had been pioneered in cities like New York. David Simon's own tv series The Wire depicts the use of a board for trackign clearance rates well into the 21st century.
Much of the terminology of the show: such as "red ball", "dunker", "stone cold whodunit" etc is taken from actual slang used by the Baltimore homicide unit in the late 1980s when David Simon covered them.
Some of the depictions of the day to day life of homicide detectives is also accurate, as reported in Simon's book. Like the sitcom Barney Miller, Homicide was unusual in depicting the everyday work of policing as being fairly routine and dull. Shoot outs and violent confrontations were pretty rare. Indeed, the sound of gunfire was not heard on Homicide in the first few seasons. Homicide is also fairly accurate in reporting the seemingly callous, often joking, attitude which homicide detectives take at crime scenes.
However, Homicide also was inaccurate in some respects. One of the most obvious, although perhaps least consequential, was the size of the department. Homicide had two shifts of about seven or eight detectives who worked under a Lieutenant (and later, when Kay Howard got promoted, a Sergeant). The real life Baltimore homicide unit of the 1980s was organized into three shifts, each covering eight hours of the day, and the shifts were considerably larger, being more than twice as large as depicted on Homicide. Also, while the real life squads were led by a Lieutenant, they were also broken down into sub-units which were supervised by Sergeants. The unit which Simon covered had three or four Sergeants.
David Simon has also pointed out in interviews that the actual homicide detectives rarely, if ever, engaged in the metaphysical soul searching which their TV counterparts did. They were mostly working men and women who had a job to do and were trying to make their city a little safer. Especially in later seasons, Homicide, the tv show also began dealing with more outre and sensationalist cases including a man who based his crime off an Edgar Allan Poe story, a set of thrill killers moving along the highway, and a serial killer who broadcast his murders on the internet. These contrasted with the real life cases presented in the book, which were generally a mix of violence relating to the drug trade, domestic disputes, drunken arguments, and the occasional child murder. Perhaps the most sensationalist crime Simon covered in his book was one where a woman had married and then killed several husbands for the insurance money.
Occasionally the differences between reality and fiction caused angst for real people. Retired Baltimore homicide detective Tom Pellegrini objected to what he saw as "his" characterization on the show. Pellegrini is widely recognized as being the inspiration for the character of Tim Bayliss. Both men were young inexperienced investigators who came to the unit from the Mayor's security detail and both became stymied over the brutal sensational murder of a little girl, murders which went unsolved even after marathon interrogation sessions with the suspect. However, after the first season, the portrayal of Bayliss diverged more from that of the real life Pellegrini. In service of the story, Bayliss did numerous things which Pellegrini felt reflected poorly on him: Bayliss became interested in kinky sex (including having sex in a coffin), he got into a physical confrontation with another detective over a woman, he revealed that he had been molested as a child, while drunk he attempted to hold up a convenience store, he dabbled in Buddhism, revealed that he was bisexual, and, ultimately, murdered a suspect who had been acquitted.