John Kilby was raised in the house of his father, William, who came from Hingham, Massachusetts with Theodore Lincoln in 1786, where the former Congregational parsonage stands. John Kilby built his own house on Smokehouse Hill in three installments. The small back two story section was erected about 1818, followed by a larger structure in 1822. Twenty years later, in 1842, the large, two-story, front main house was added to the rest of the building, complete the rambling, commodious home in which he and his wife, Lydia C. Wilder, another daughter of Ebenezer C., Sr., raised their large family of nine children. One interesting feature of the fine home, the landscaped wallpaper illustrating an English hunting scene, which was hung on the parlor wall by the old Scotsman, Steadman, has been preserved through the years. The dwelling was not occupied by John Kilby's granddaughter, Mary (Vose) and her husband, Edmunds B. Sheahan. the last Kilby to reside in the old house was Keith, who later sold it to R. Daniel and Elsa Chubbuck.