The Three Village Historical hung its sign outside the Society Headquarters at 93 North Country Road in Setauket. For thirty four years the Society had been operating out of rented and borrowed space. On May 14, 1998, the Historical Society purchased its first headquarters building from the Society for the Preservation of Long Island Antiquities (SPLIA). The new headquarters building gives the Society a physical presence in the community and establishes a cultural area on North Country Road with the neighboring Thompson House and Gallery North.
The two story farmhouse belonged to Ebenezer Bayles and later to Stephen Swezey. It has a central chimney and gable. The house, built about 1800, is located adjacent to the Thompson House (1700) an historic property belonging to the Ward Melville Heritage Organization. In 1961, the New York Telephone Company donated the Ebenezer Bayles/Stephen Swezey house to SPLIA. SPLIA moved the house from its original site 300 yards south on North Country Road to its present location on 2.2 acres at the intersection of North Country Road and Ridgeway Avenue. The house, now 2,500 square feet, has been converted into offices with the addition of a North wing lined with concrete blocks to provide fire proof storage space.