The UN refugee body said in a statement it was not party to the agreement and stressed that "return of refugees should be made free from undue pressure."
"The UNHCR is not in a position to ascertain to what extent refugees, who are civilians by definition, are returning in this movement," it said.
The deal included the release of eight Hezbollah fighters by the Nusra Front in exchange for individuals held by Lebanon. Three of the Hezbollah fighters were released overnight. The remaining five are expected to be released once the first convoy reaches its destination in Syria.