Pisgah Forest is located northeast of Brevard, North Carolina, near the Davidson River. The Elizur and Ann Patton Federal-style home was built in 1846 in Transylvania County. The rear-ell was built in 1860 to accommodate the growing family. According to the 1860 census, there were seven children.

Patton owned over 600 acres and farmed on some of that land. According to the 1850 Slave Census, the Pattons enslaved one woman. The 1860 Slave Census contained one woman and one child. It is not known where they lived on the land.