Canadian Joshua Boyle, his American wife Caitlan Coleman and their three young children landed at Toronto's airport late Friday night, where Boyle told reporters that the network killed his newborn daughter and raped his wife during their five years in captivity. Their three children were born during their time in captivity.
"And I have to try to teach religion to my children, and to teach them Islam, but to say do not think that these bad men are Muslim", said Boyle.
He says the kidnappers said they believed the American government would pay to prevent the baby from being born in captivity.
"The allegations floating around in the media have nothing to do with the reality because the said people are now in the hands of our enemy".
"Taking your pregnant wife to a very unsafe place to me and the kind of person that I am is unconscionable", Jim Coleman, father of Caitlan Coleman said.
"Even the murder of a child is unfounded", he continues, explaining that "the woman fell ill, and because of his serious state of health, had a miscarriage (lost) a girl".
CNN, quoting an unidentified senior USA official, said Boyle balked because he feared possible arrest on American soil.
In a video released by Pakistan's military that was filmed before he left that country for home, Boyle said Pakistani security forces positioned themselves between the hostages and their Haqqani network captors to keep the family safe amid the gunfire.
"He wants to be home", the father told CNN from his home in Canada.
Meanwhile Coleman's father criticized his son in-law for having taken his heavily pregnant wife to Afghanistan in 2012.
A senior official had said Boyle refused to board an American military plane on Thursday over concerns he could face arrest. Boyle told his parents that he, his wife and their children were intercepted by Pakistani forces while being transported in the back or trunk of their captors' vehicle and that some of his captors were killed. The army recovered the hostages safely from the vehicle.
Meanwhile, Mr Boyle denied reports he had had refused to board a plane that was preparing to take the family home.
A family held hostage by Taliban-linked extremist Haqqani network in Afghanistan has returned to Canada where they will undergo full medical exams.
Boyle previously had been married to the sister of Omar Khadr, a Canadian who spent 10 years at Guantanamo Bay after being captured in 2002 in a firefight at an al-Qaida compound in Afghanistan.
While the family were being held, the Haqqani network demanded the release of the son of its founder in exchange for turning them over and released several videos in which Mr Boyle pleaded with the Afghan authorities.