During the song, "Almost Like Being In Love" Gene Kelly dances around. When he passes the bulls, you can see the bushes to the right of the pig moving as if someone off screen is putting them back in place after moving them out of their way.
If Minister Forsythe made the miracle happen from outside the village and was not part of it, only two days prior to the villager's time line, and they could not leave the village, they would have no way of knowing it had actually happened. The minister could merely be missing for a couple days for all they knew. In fact, the minister's missing presence conveniently allows the story to develop angles that can't be explained, such as why he felt he had to leave his beloved flock (his advancing age is given as a reason, but that makes no sense, a devoted minister would never leave his flock), why he had to say his prayer from outside the affected area instead of remaining with them, or why they only wake up every 100 years instead of just becoming invisible to the world and continuing to live life as normal, or any number of plot points that don't make sense when tossed together.
The inhabitants of Brigadoon would have no way of knowing that a single person leaving the village would make them all disappear forever. It's highly illogical to believe that was part of the prayer that enchanted them, especially as that was not related by Mr. Lundie in his description of events. In fact, it's a very implausible part of the story, though it does add to the story line. One renegade could ruin it forever for the whole village, negating the entire miracle. And in a village of that apparent size (many dozens of people partaking in events), it's inevitable that someone would want to leave, as happened less than two days after the miracle, dooming the entire village instead of saving it. In fact, the 'leaving' rule runs contrary to the nature of the miracle, which is to save the entire village from outside influences, not inevitably doom it because of one foolish person.
For only being enchanted for a period of less than two days (the day of the movie being only their second day awake since the enchantment), by their own time line, the villagers have developed a rather implausibly complex set of customs and beliefs related to their enchantment.
The "plot hole" saying the town could not know about Minister Forsythe's contract/miracle with God is in err. During the scene where Fiona takes Tommy to see the school master, Mr Lundie, he says that the minister spoke to him of his plan to pray for a miracle, and therefore, the whole town would know after the school master told them. Mr. Lundie also said that for such a miracle to occur it would require a great sacrifice and that was the minister leaving the village to pray for the miracle so that debunks another "plot hole" posted here.
As Mr. Lundie tells the story of Brigadoon, in every reference it happened long ago. He always says 'was' instead of 'is' even though the enchantment only happened less than two days ago by his reckoning, and thus is still in his present time.