The accident occurred on Saturday night near Vigo and the border with Portugal. The regional La Voz de Galicia newspaper said the bus was carrying people visiting their loved ones jailed in Monterroso in central Galicia.
The emergency services said two corpses had been recovered while two others, including the bus driver, had been rescued and taken to hospital.
Spain's Civil Guard later said two more bodies were found on Sunday. The woman's body was recovered but a man's corpse was yet to be retrieved.
"There could have been more people inside the bus," the emergency services had said on Twitter and posted a photo of the vehicle in turbulent waters.
Rescue operations had to be suspended overnight due to bad weather but resumed Sunday morning.
The accident took place "at a spot with a steep gradient", making access difficult, said a Civil Guard spokesman in the city of Pontevedra, some 30 kilometres (18 miles) from the site of the accident.
The Civil Guard said the bus driver had tested negative for alcohol.