The mystery of James Perry is close to being solved. Thanks to people on Ancestry.com who have been willing to answer questions, we have found what happened to James.
It appears as though he left his family in Winnebago MN around 1869 and returned to Illinois. There, he married a widow who also lived in Winnebago MN for a while and whose husband died there. James and Jennie (aka Emma and Jane Elwell) had 2 daughters in Pontiac Illinois, then moved to Girard Kansas and had one more daughter. The two oldest daughters died very young, and only the baby, Daisy Perry, survived to adulthood.