Ms Yates, who was appointed by Mr Obama, said she was "not convinced that the executive order is lawful".
“I am responsible for ensuring that the positions we take in court remain consistent with this institution’s solemn obligation to always seek justice and stand for what is right,” Ms Yates wrote in a letter to Justice Department lawyers.
“At present, I am not convinced that the defence of the executive order is consistent with these responsibilities.”Mr Trump has named Dana Boente, US Attorney for the Eastern District of Virginia, to serve in her place.
The White House press office said in a statement that Yates "has betrayed the Department of Justice by refusing to enforce a legal order designed to protect the citizens of the United States."
The statement called Ms Yates an Obama administration appointee "who is weak on borders and very weak on illegal immigration."
Dana Boente was sworn in at 9 p.m. ET. A few hours later, Boente issued a statement rescinding Yates' order, instructing DOJ lawyers to "defend the lawful orders of our President."
Boente told The Washington Post that he will agree to enforce the immigration order.
Trump didn't call Yates to dismiss her, she was informed by hand-delivered letter, according to a different administration official.
Ms Yates was due to be replaced by Jeff Sessions, Mr Trump's pick for attorney general, once his appointment is confirmed by the US senate.