The 140 acres of rolling countryside which house the gardens include a mile and a half driveway enveloped by an alley of live oaks. The landscaping effects at Afton Villa were achieved by taking advantage of the natural contours of the property. Like many traditional formal southern gardens, Afton Villa has terraces which descend in stages away from the house. Afton Villa Gardens' most typical traditional features are its maze and its parterre garden. Both retain their original designs, although time has allowed for some alterations. A sundial now marks the spot where a small gazebo once stood. Next to the parterre garden is the Barrow Family Cemetery. The centerpiece of the cemetery is a large marble Tuscan obelisk, erected by the United States Congress in memorial to Senator Alexander Barrow upon his death. The cemetery is the only feature of the present garden which predates 1849, dating to the time of the first plantation on the site in the late 18th century. A large hedge surrounds the cemetery, and an artificial pond and lake dot the grounds.
The history of Afton Villa is entwined with that of the Barrows, one of the richest and most prominent families in antebellum Louisiana. Bartholomew Barrow purchased the land in 1820 from his brother William, and in 1839 he sold it to his son, David. David would eventually carve out a thriving plantation empire of some 2,000 to 3,000 acres, which would make him, according to the 1860 census, the wealthiest planter in West Feliciana Parish. In 1849, he and his second wife, Susan A. Woolfolk, built around an existing small house to create an imposing Gothic Revival villa of some 40 rooms, and added the gardens. David Barrow died in 1874 and his wife continued to live at Afton Villa until 1876, when she sold the estate. The house was destroyed by fire in 1963. Afton Villa Gardens is popularly known for the azaleas which grow there. One particular strain, known as the Pride of Afton or Afton Villa Red, was developed at the gardens.