Icehouses always stored blocks of ice in sawdust or straw to insulate them. There is none seen in Walnut Grove.
When Willie is talking to his father about the ice house incident, look behind him in the next room and you can see Katherine MacGregor's shadow as she waits for her cue to come in.
If Wilie had been locked in the icehouse before, then the door would have been fixed already to open from the inside.