Herbert “Hub” Dudley was the son of Clayton and Katie Dudley, who started several businesses in Dudley. At the corner of Jackson and Truxton Streets, the family opened up a motel, gas station, and cafe to accommodate Black travelers.

Dudley opened up this space to provide food for the Black officers of the United Services Organization, hence the name “Retreat.” After the conclusion of World War II, Dudley continued to serve food, but now It was to the general public. Well-known acts such as Little Richard, Rosetta Tharpe, and James Brown stopped by the restaurant; sometimes, they entertained patrons. It was a full-service restaurant that offered such treats as fresh doughnuts, ice cream, and Coca-Cola.