Since 1978, the Sub Laban family had fought in the Israeli courts against their eviction from their home in the Muslim Quarter of the walled Old City.
But early on Tuesday, police arrived to remove the family from their home following a court order.
"They do not have the right to expel me from my house," Nora Abu Laban, 68, told AFP.
"They are thieves and they steal everything from us, they stole the house, the lands, the youth."
Israeli and Palestinian activists jostled with police in the aftermath of the eviction.
One held a placard that read "A family was evicted today" as Jewish settlers looked on, video footage filmed by AFP showed.
In May, the Sub Laban family had been served with an eviction notice and told to vacate the building by June 11.
The "family was forcibly evicted from their home by Israeli police," Ajith Sunghay, head of the UN Office of the High Commissioner for Human Rights for Palestinians, said in a statement.
He said 12 Israeli activists protesting against the eviction, seven women and five men, were arrested.
"Concerted efforts to evict Palestinians from their homes in occupied east Jerusalem may amount to forcible transfer," Sunghay said.
"Forcible transfer is a grave breach of the Geneva Conventions and a war crime."
The European Union expressed "regret" over the decision.
It urged the "Israeli government to respect international law and let these families live where they have been living for decades".