It was most likely this house where James MacMillan would have been born and raised. Documents in America indicate that he inherited some property.  I speculate that it came from the maternal side, since there is no history of MacMillans being landowners in this area.  The Dalrymple-Hay family owned the property from at least the late 1600's to the early 1900's. 

It isn't currently known how much property this involved.  Dunragit was a large estate, but there were other heirs as well.  When American descendants attempted to try and recover the property in the late 1800's, they discovered the property rights conflicted with other heirs to the extent that the actual value would not have been worth the legal expense of pursuing it.