Gregory Peck later said regarding Robert Mitchum, "I had given him the role and had paid him a terrific amount of money. It was obvious he had the better role. I thought he would understand that, but he apparently thought he acted me off the screen. I didn't think highly of him for that."
At first, Robert Mitchum didn't want to do the film but finally relented after Gregory Peck and J. Lee Thompson delivered a case of bourbon to his home. His reply was, "Okay, I've drunk your bourbon. I'm drunk. I'll do it."
J. Lee Thompson had always envisioned the film in black and white prior to production. As an Alfred Hitchcock fan, he wanted to have Hitchcockian elements in the film, such as unusual lighting angles, an eerie musical score, closeups, and subtle hints rather than graphic depictions of the violence Cady has in mind for the family.
Gregory Peck, who produced the film, didn't like the original novel's title "The Executioners". When thinking of a new title, he decided that movies named after places tended to be very successful, so he looked at a map of the U.S. until he happened upon Cape Fear in North Carolina.
Polly Bergen suffered minor bruises in a scene where her character struggles with Cady. He was supposed to drag her through various doors on the set, but a crewmember mistakenly left all those doors locked, so that when Robert Mitchum forced Bergen through the doors, she was actually being used as a ram to push them open.
This film contains one of the few instances of a correct depiction of what someone sees when looking through binoculars. In most films, what is shown resembles a sideways figure 8 (i.e. side by side magnified images, one for each eyepiece). But what one really sees is a single round magnified image, the same as what you see when looking into the eyepiece of a telescope.
Robert Mitchum had a real life aversion to Savannah, Georgia, where as a teenager, he had been charged with vagrancy and put on a chain gang. This resulted in a number of the outdoor scenes being shot at Ladd's Marina in Stockton, Georgia, including the conflict on the houseboat at the end of the film.
According to Polly Bergen, Robert Mitchum cut his hand on a cabinet during their scene. "His hand was covered in blood, my back was covered in blood. We just kept going, caught up in the scene. They came over and physically stopped us."
In the source novel The Executioners, Cady was a soldier court-martialed and convicted on then Lieutenant Bowden's testimony for the brutal rape of a 14-year-old girl. The censors stepped in, banned the use of the word "rape", and stated that depicting Cady as a soldier reflected adversely on U.S. military personnel.
Director J. Lee Thompson complained at the time that UK censor John Trevelyan had ruined the film by making extensive cuts, and the number of edits suggested ranged from 60 to over 100. Trevelyan later replied that he had made only 15 cuts, totalling around 6 minutes, with edits made to threatening dialogue and assault references, Cady's attack on Peggy, and all shots of him staring longingly at Nancy. All later UK video releases restored the cinema cuts.
According to Robert Mitchum, during the filming of the final fight scene between him and Gregory Peck, Peck once accidentally punched him for real. Mitchum, knowing that Peck didn't mean to and ever the professional, refused to break character and continued filming the scene. However, upon entering his trailer, Mitchum said that he "literally collapsed" due to the impact of the punch and said that he felt it for days afterwards. Mitchum said, "I don't feel sorry for anyone dumb enough who picks a fight with him (Peck)."