The events take place during a single summer day in Maine, in a cottage on an island. The camera stays indoors except when the characters take walks to the water. Two elderly widowed sisters are summering in the cottage, as they have done for 60 years. The sisters frequently relate events of the past to each other. Sarah Webber (Lillian Gish) is the older of the two, the owner, and is caretaker to her blind sister Libby Strong (Bette Davis). Every August, they have watched the whales passing in the nearby waters, but the sense is that this may be their last summer together.
Sarah is living life the way it's important to her, friendly, attentive, gracious, worried about continuing to care for her sister. She cleans, makes beanie animals for the fall fair, cooks, serves meals and tea and keeps the house in order. Libby is contrary, cantankerous, and psychologically ready to die.
Mr. Maranov (Vincent Price) is an old man originally of the Imperial Russian aristocracy who has been living in a rented room in the house of a neighbor woman who has recently died. Maranov arrives in mid-morning, announces his presence and makes small talk, asks permission to fish from their shore, and goes to the shore to fish, but he really is fishing around for a place to live.
Joshua Brackett (Harry Carey, Jr.) is a noisy handyman man who drops in to offer to install a picture window.
Joshua says he can get a good deal on the materials and can do the work in about two weeks. Libby brushes off the idea, saying they are too old for anything new.
A neighbor and life long friend of the sisters, Tisha Doughty (Ann Sothern) shows up to gossip, complain about newcomers and losing her drivers license, and by the way suggests to Sarah that she might want to think of selling the old cottage and leave Libby's care to her daughter. Sarah is close to being convinced, until she remembers how Libby helped her when her own husband died. They guess that Maranov is trying to ingratiate himself with Sarah to see whether she will rent him a room, as he needs to move soon.
There are quiet scenes in which Sarah talks to old photographs as she cleans or fixes up her hair and makeup.
Later in the day, Maranov has caught some fish and offers part of his catch to Sarah. Sarah accepts, puts the fish in her refrigerator, and invites Maranov to come to dinner to eat the fish. Libby protests that she will not under any circumstances eat of that fish, nor get dressed for dinner. Sarah promises to fix a pork chop for Libby, as she is sweet in her sisterly devotion to Libby and avoids getting drawn into her moods. She always calls her dear. She brushes her hair, fixes breakfast for her, gets her clothes together and tends to the garden. "Busy, busy, busy" is how Libby talks about her, and irritatingly calls her Say-rah throughout. In a moving sequence, Libby brushes her own face with a lock of her husband's hairwhile sitting alone in her bedroom.
Mr. Maranov arrives for dinner, dressed up as best he can, after having picked a few flowers, and the threesome have a candlelight dinner, during which Maranov entertains the sisters with stories from the old aristocratic days, exile in Paris, and his life since then. He has little money and has survived mainly on the kindness of others.
The relatively pleasant dinner session comes to an end when Libby unequivocally announces that no way will they even consider letting Maranov live in their house. Sarah is shown in an emotional scene in which she celebrates her 46th wedding anniversary by having an imaginary conversation with her long deceased husband. "Forty-six years, Phillip", she tells him. "Forty-six red roses; forty-six white. White for truth--red for passion. That's what you always said--passion and truth; that's all we need. I wish you were here, Phillip."
Tisha Doughty arrives together with Mr. Beckwith (Frank Grimes) in a vintage car. Beckwith is a real estate agent who believes that Sarah may want to put her house up for sale. Sarah, much annoyed at the suggestion, eventually orders Mr. Beckwith out, determined not to sell.
Joshua Brackett the handyman returns to the house to look for a wrench he might have left behind earlier. He is constantly making a racket as he enters or leaves or rummages in the garden. Libby asks for Joshua to come into the house, asks questions about the proposed picture window, and announces that she and her sister have agreed to go ahead with the project.
The picture window project is symbolic, in that Libby shows her first sign in possibly many years of not giving in to death. Sarah is delighted with the prospect of getting her picture window installed. Libby asks Sarah to walk her out to the point, the spot from which they used to watch the whales go by.