Bette Davis walked off the set once in a fight with William Wyler over the film's climactic scene in which Leslie says to her husband, "With all my heart I still love the man I killed." It was a crucial line, and the way it was delivered was of utmost importance to the drama. Wyler believed that Davis should look her husband's character in the eye as she delivered the devastating blow. Davis, however, disagreed. She recalled in her 1962 memoir The Lonely Life, ""It was such a cruel thing to say to the husband, I felt I could not say it to his face. I couldn't conceive of any woman looking into her husband's eyes and admitting such a thing. I felt it would come out of her unbeknownst to herself, and therefore she would not be looking at him. Willie disagreed with me - most definitely. I walked off the set! Something I had never done in my whole career...I could not see it his way, nor he mine. I came back eventually - end result, I did it his way. It played validly, heaven knows, but to this day I think my way was the right way. I lost, but I lost to an artist."
After shooting was completed, William Wyler watched a rough cut and decided that he wanted the character of Leslie to be more sympathetic. He ordered some re-writes and planned to shoot them. Bette Davis recalled - "I was heartbroken," she said, "as I felt, after reading the rewrites, that my performance could be ruined with these additions. I asked Willie if I could see the film before doing the retakes. To my horror I was crying at myself at the end of the showing. There was dead silence in the projection room when the lights came up. I said, 'If we film these retakes, we will lose the intelligent audience. It is impossible to please everyone with any one film. If we try to accomplish this, we can lose all audiences.' Plus, to my shame, even though I played the part, I deeply sympathized with Leslie Crosbie. We only made one small addition to the original film. Wyler had agreed with me. Thank God!"
The first scene that William Wyler shot was the famous opening shot in which we see Leslie shoot Geoffrey Hammond. The opening shot, which lasted two minutes on screen, took an entire day to film, and that was before even a single word of dialogue was spoken. The studio expected him to shoot at a rate of 3-4 script pages a day, but the opening shot reflected a mere paragraph on page one.
Jack L. Warner originally asked William Wyler to test James Stephenson for the role of the lawyer. Wyler was surprised at how suited Stephenson was for the part and then was astonished when Warner balked at casting him, worrying about the stock player's lack of name recognition. Wyler insisted on keeping him, putting him in the odd position of having to fight to cast an actor that Warner had originally suggested.
According to Bette Davis, actor James Stephenson would often get into fights with William Wyler during filming and walk off the set out of frustration from time to time. She recalled - "Every time Jimmy would leave, I would run after him and make him come back, saying, 'It will be worth it, Jimmy - don't go. You will give the great performance of your career under Wyler's direction.'" Each time Stephenson would return to work and shooting would resume.
According to screenwriter Winston Miller, Warner Brothers commissioned him to write a Western version of "The Letter" with the evil woman rewritten as a schoolmarm. It was offered to director Raoul Walsh, who refused to read it. According to Miller he said, "I don't make pictures about schoolteachers!"
Director Wyler and star Davis had had an affair which ended well before this, their third project together. (He was married to Margaret "Talli" Tallichet two years before shooting.) Davis discovered herself pregnant during the first week of filming and, unsure of the father, kept it a secret and arranged for an abortion, her third, a week later. She later told friends, "I should have married Willy."
Davis fought several times with Wyler over the interpretation of her character, but later concluded, "I did it his way.... Yes, I lost a battle, but I lost it to a genius.... So many directors were such weak sisters that I would have to take over. Uncreative, unsure of themselves, frightened to fight back, they offered me none of the security that this tyrant did." She did not mind his customary calls for many multiple takes of scene after scene because she felt it matched her own perfectionism.