I recently went to the local library's Local History Room to do some research on the house.
What I already knew was that from 1770/1771 to 1843, the house was owned by Stephen Wilcox. When he died (at 97), the house was then bought or given to his great-great-grand-niece Adelia Wilcox. The house remained in the Mildrum family (three unmarried daughters lived there until their deaths) until 1953 when the last Mildrum sister, Hattie, died.
At the library and town hall I was able to find property transfers. It appears that Hattie sold the house on November 18, 1942 to Winslow G. Merrit. On February 15, 1957 Winslow Merrit sold the house to Leslie J. Wagner and Alice B. Delaney. On September 11, 1963, Leslie Wagner sold the house to Spence W. and Elizabeth A. Nelson. They sold the house to the person we bought it from in 1966.
I was also able to get a couple simple property maps from the town hall. They show inaccurate property lines (according to the map, our next door neighbor owns our driveway, garage and a corner of our house), which we know aren't true because there are metal stakes in the ground indicating the property lines.
I was also able to find a couple building permits from when the previous owner installed the pool and when he upgraded to 200 amps.
I love researching stuff :)