The Lady Mary Joy 3 was travelling from Zamboanga City on Mindanao Island to Jolo Island in Sulu province when the fire broke out late Wednesday, prompting passengers to jump overboard, disaster officer Nixon Alonzo said.
Rescuers, including the Philippine Coast Guard and fishermen, saved 195 passengers and 35 crew as the blaze ripped through the ferry off Baluk-Baluk Island in Basilan province.
Basilan Governor Jim Salliman said 31 bodies had been retrieved, including 18 found inside an air-conditioned cabin.
Photos released by the coast guard showed one of its vessels spraying water on the burning ferry as personnel in smaller boats plucked people from the dark water.
Firefighter Jayson Ahijon, who was among the passengers, said people panicked as fire and thick smoke overwhelmed the vessel.
Some jumped overboard before putting on life jackets.
"It's beyond my imagination, the fire was spreading so fast," Ahijon, 36, said.
Ahijon said he and several coast guard officers helped other passengers to safety, including a baby they put in a bucket and lowered over the side of the boat.
The captain ran the vessel aground as the fire spread "so many more could survive since it would be easier to swim to shore", Commodore Rejard Marfe from the Philippine Coast Guard told AFP.
Salliman said at least three children, including a six-month-old baby, were among the dead.
Authorities also said 14 people were injured.