When filming the hospital scene, Anne Baxter drew upon an experience from her childhood, when she lost her three-year-old brother. Speaking of it years later, she said the scene was the best in her career, and still gave her chills.
Fox purchased the screen rights to the novel in March 1945 for $250,000 plus 20% of the net profits. To avoid another $50,000 specified in the contract if the principal photography was not started by 2 February 1946, producer Darryl F. Zanuck provided for location shooting in the mountains around Denver, Colorado (the Himalayas in the film), in August 1945. The cast had not yet been set, so the character of Larry was played by a double and seen only in long-shot. Zanuck hoped to get Tyrone Power to star and delayed casting until Power was released from military service in January 1946.
Anne Baxter had to leave the set for several weeks and when she returned found she felt like an outsider, everyone else having developed working relationships in her absence. She liked this and used it, since her character Sophie is also on the outs and not able to cope with the loss of her husband and child.
There were 89 different sets built for the film, which had the longest shooting schedule for any film at the studio to that date. According to some news items, the film broke all previous studio box office records.
Tyrone Power got along well with Gene Tierney. In the movie her character falls for his, but on the set it was Power who fell for Tierney. As soon as this was noticed, rumors began to fly that the two were romantically involved in real life. After the premiere, Power brought her a scarf with the word "Love" embroidered on it as a gift and she had to tell him she was seeing John F. Kennedy, one of the sons of Joseph Kennedy, still years away from his political victories in the US Senate and the Presidency. Power understood and made no more advances. Tierney's own husband, Oleg Cassini, was also working on the movie, designing her dresses, but the two had already decided to divorce and there was no tension between them at any point during the shoot.
Although Fox did not use W. Somerset Maugham's screen adaptation of his own novel, it felt it owed the author something. George Cukor suggested giving him a fine modern painting, which the director later recalled was a Matisse.
Tyrone Power got along well with Edmund Goulding. He later remarked that Goulding was his personal favorite even after Goulding made a strange request of him on the set. To capture the essence and mindset of Larry Darrell, the film's protagonist, Goulding asked Power not to have sexual relations until after the scenes with the Yogi in the east had been shot. Power happily agreed and later said, "I know by personal experience that in nothing are the wise men of India more dead right than in their contention that chastity intensely enhances the power of the spirit." Later, when Power found out Goulding asked this of all his leading men as a way of achieving a certain look, he broke into laughter.
After principal shooting, Darryl F. Zanuck took over, instructing editor J. Watson Webb Jr. on what to cut and how to cut it. The Fox mogul took control of post-production like few producers today and directors working with him understood, implicitly, that once the principal photography was done, the movie was out of their hands. It wasn't a bad deal, as Zanuck had a good feel for pacing, and the final result, coming in at almost 2-1/2 hours, moves along at an easy pace. It wasn't the Oscar-winning hit Zanuck had wanted, but it was a box-office success all the same, with almost everyone who worked on it, even Clifton Webb, expressing delight with their experience.
Tyrone Power hadn't been cast when the Colorado (doubling for The Himalayas) scenes were filmed. The stars had not yet been cast; Larry Darrell was played by a stand-in and was filmed in extreme long shot.
Darryl F. Zanuck experienced conflicts with the PCA over the depiction of alcoholism in the film. In materials contained in the PAA/PCA files at the AMPAS Library, Zanuck argued in a April 1, 1946 letter to Joseph I. Breen that he could not comply with any of the requests to eliminate drinking from the picture because "alcoholism is the basic foundation of our plot."
Edmund Goulding was a director actors either loved or hated. He had a habit of asking to "be" the actor to get what he wanted. In her autobiography, Gene Tierney relates exactly how Goulding did it: "When he wanted to describe to you how a particular scene should be played, he would step in front of the camera and say, 'May I be you?' Then he would promptly act out the entire scene." Tierney found it delightful and even wrote, "I don't recall a set where there was more cheerfulness." Others, like Clifton Webb, adamantly disagreed, remarking, "He had everybody entranced but me, and I'm afraid I remained cold to this type of thing to the very end."
W. Somerset Maugham's contract stipulated that unless principal photography was begun by February 2, 1946, the studio would have to pay the author an additional $50,000. Location shooting began in Denver in August 1945, thus meeting the terms of the contract.
Although various "Hollywood Reporter" news items add Jamiel Hasson to the cast, his appearance in the released film has not been confirmed. Roman Bohnen was also mentioned in a "Hollywood Reporter" news item, but he was not in the film.