In The Matrix Reloaded, when Neo speaks with the Architect, he tells Neo: "The function of the One is now to return to the source, allowing a temporary dissemination of the code you carry, reinserting the prime program. After which you will be required to select from the matrix 23 individuals, 16 female, 7 male, to rebuild Zion. Failure to comply with this process will result in a cataclysmic system crash killing everyone connected to the matrix, which coupled with the extermination of Zion will ultimately result in the extinction of the entire human race." The Architect then gives Neo a choice. The choice to go through one door which leads to him to select the 23 individuals, the other leading to The Matrix, where Neo can save Trinity and continue the war. The latter would be considered non-compliance to which The Architect said would result in the cataclysmic system crash, killing everyone connected to the Matrix.
Obviously, Neo chooses "Door Number Two" and goes to save Trinity. In this film, Smith has gone on to infect every single person inside The Matrix and the machines are on the brink of destroying Zion. Neo then makes the choice to go to "the source" and bargains with the machines. He tells them Smith will not stop with The Matrix and will spread to the machines themselves, effectively destroying existence for man and machine alike. Neo offers to stop Smith in exchange for peace among man and machine. The Source agrees to this and sends Neo into the Matrix. Neo finally "surrenders" to Smith and allows Smith to infect him and The Source sends a jolt through Neo (more or less like sending an anti-virus program through your computer) effectively deleting every infected file. This kills Neo, destroys Agent Smith and every copy of him. Some think that this also kills every person that Smith infected, but this does not seem to be the case. Notice that as all copies of Smith 'explode', they leave a bright spot in the overhead shot of the city. Apparently, something remains. In the following shots, we see the Oracle lying in the place of the Smith copy that infected Neo. She wasn't destroyed but restored to her original program. We can assume the same thing happened to all the other people inside the Matrix, although another possibility is that only programs were restored.
Neo's return to the Machines has apparently disseminated the code, because as he is carried away (respectfully by the machines) in the real world, the Matrix program re-initiates and everything is restored again. The Architect meets up with the Oracle, and the Oracle asks him what will happen to the "others... the ones who want out" (obviously referring to those still connected to the Matrix). He promises that they will be released. The machines honor their word and leave Zion alone for the sake of peace. So The Architect was telling the truth about the promise of Zion being destroyed and everyone in the Matrix dying. However, he didn't comprehend human choice and he didn't foresee the Smith program running out of control, which resulted in Zion being left more or less intact and those inside the Matrix learning the truth, so they can choose to leave the Matrix. Remember that Morpheus said in the first movie that many people are quite comfortable with their existence inside the Matrix, something they will even defend over the truth. So it is most likely that enough people will remain inside to provide the Machines with energy, and the rest will construct cities in the real world.