Vietnam was temporarily partitioned at the 17th parallel, and under the terms of the Geneva Convention, civilians were to be given the opportunity to freely move between the two provisional states for a 300-day period. Elections throughout the country were to be held in 1956 to establish a unified government. Around one million northerners, mainly minority Catholics, fled south, fearing persecution by the communists, following an American propaganda campaign using slogans such as "The Virgin Mary is heading south", and aided by a U.S. funded $93 million relocation program, which included ferrying refugees with the Seventh Fleet. It is estimated that as many as two million more would have left had they not been stopped by the Viet Minh. The northern, mainly Catholic refugees were meant to give Diem a strong anti-communist constituency. Diem later went on to staff his administration's key posts mostly with northern and central Catholics.

In addition to the Catholics flowing south, up to 130,000 ‘Revolutionary Regroupees’, went north for "regroupment" expecting to return to the South within 2 years. The Viet Minh left roughly 5,000 to 10,000 cadres in South Vietnam as a "politico-military substructure within the object of its irredentism." The last French soldiers were to leave Vietnam in April 1956. The PRC completed their withdrawal from North Vietnam at around the same time. Around 52,000 Vietnamese civilians moved from south to north.

In the north, the Viet Minh ruled as the DRV and engaged in a drastic land reform program in which an estimated eight thousand perceived "class enemies" were executed. In 1956 the Communist Party leaders of Hanoi admitted to "excesses" in implementing this program and restored a large amount of the land to the original owners.

In the south, former Emperor Bảo Đại's State of Vietnam operated, with Ngô Đình Diệm (appointed in July 1954) as his prime minister. In June 1955, Diem announced that elections would not be held. South Vietnam had rejected the agreement from the beginning and was therefore not bound by it, he said. "How can we expect 'free elections' to be held in the Communist North?" Diem asked. President Dwight D. Eisenhower echoed senior U.S. experts when he wrote that, in 1954, "80 per cent of the population would have voted for the Communist Ho Chi Minh" over Emperor Bảo Đại.

In April–June 1955, Diem (against U.S. advice) cleared the decks of any political opposition in the south by launching military operations against the Cao Dai religious sect, the Hoa Hao sect of Ba Cut, and the Binh Xuyen organized crime group (which was allied with members of the secret police and some military elements). As broad-based opposition to his harsh tactics mounted, Diem increasingly sought to blame the communists.

In a referendum on the future of the State of Vietnam on October 23, Diem rigged the poll supervised by his brother Ngo Dinh Nhu and was credited with 98.2 percent of the vote, including 133% in Saigon. His American advisers had recommended a more modest winning margin of "60 to 70 percent." Diem, however, viewed the election as a test of authority. On October 26, 1955, Diem declared the new Republic of Vietnam (ROV), with himself as president. The ROV was created largely because of the Eisenhower administration's desire for an anti-communist state in the region.