Within its walls lived three generations of one remarkable family that made significant contributions to the political, literary, and cultural life of New England and the United States. General Peleg Wadsworth built the house in 1785–1786, and the last person to live there was Anne Longfellow Pierce, Henry's younger sister.
Virtually all of the household items and artifacts are original to the Wadsworth and Longfellow families. Furnishings from the three generations illustrate changes in style, technology, and attitude over the 19th century. Peleg and Elizabeth Wadsworth raised ten children in the house before retiring to the family farm in Hiram, Maine, in 1807. The only single–family residence to survive downtown Congress Street's change from a mixed commercial and residential neighborhood on the edge of town to an urban business district, it is the oldest standing structure on the Portland peninsula.