About 90 people have been pulled alive from the sea by rescue teams and private boats.
The vessel was on a short route from the island of Itaparica, in Bahia state, which is popular with tourists.
This is the second major boat accident in Brazil this week. A ferry sank on Tuesday on the Xingu river killing at least 19 people.
Survivors of the latest incident said the ferry capsized when it was hit by a powerful wave.
“It was raining and we were hit by a wave. It took a long time for us to be rescued. I was in the sea for two hours,” Edvaldo Santos de Almeida told O Globo newspaper.
Survivors have been taken to hospitals in the state capital, Salvador.
The boat that sank late on Tuesday in the state of Para was carrying about 70 passengers.
It was travelling overnight from the city of Santarem to Vitoria do Xingu in Brazil’s Amazon region.
Police are investigating what may have caused the vessel to sink and whether it may have been carrying more passengers than permitted.