The destruction was most severe in neighbourhoods built along rivers that lead into the Indian Ocean.
With roads unpassable, some schoolchildren and workers were forced to stay at home on Monday.
Dozens of families are assessing the damage caused to their homes, and trying to recover their belongings.
Heavy rains hit the city on Saturday and Sunday, with the most-affected areas being the Kinondoni and Ilala districts.
A bridge that is a gateway to the city centre – including the official residence of President Samia Suluhu Hassan – could not be crossed on Sunday because of flooding, but this is now possible as water levels have fallen.
The Tanzania Meteorological Agency has warned of heavy rains for the rest of the month, while the president has urged people to move away from flood-prone areas.
The authorities have not given any casualty figures, but one man told a radio station that his wife had been swept away, while another family said they are searching for their missing son.
They are not sure whether he was swept away or whether he survived and is being looked after by another family.
Kinondoni resident Jonathan Urassa told the BBC that “some people have lost everything”.
Pointing to an elderly woman, he added: “She just buried her mother last year, and now her house is completely gone. She is living at her neighbour’s house.”
Another resident, Elinaike Shoo, said she too had lost her home in the floods.
“I have nothing left behind. I’ve even borrowed clothes from my neighbour,” she said.
Flooding is said to be the greatest natural hazard in Tanzania, affecting tens of thousands of people each year.
More than 80 people died following floods and landslides in Tanzania’s northern Hanang district last month.