The history of the Nepperhan Business Center predates the civil war by a number of years. During the 1820s, new tariff laws caused the imposition of duties on foreign-made carpets, restricting their importation and stimulating the domestic manufacture of this luxury commodity. Enter Alexander Smith, a young man from a New Jersey farming background, who used family money to purchase a small rug mill at West Farms in the Bronx (at the time a part of Westchester County). Equipped with 25 hand looms, the mill was plagued by two fires and, after the second fire, in 1864, Smith moved his operation to the former Waring Hat Factory in Yonkers, relying on the Saw Mill River for power.

The Alexander Smith Carpet Mills in Yonkers became a bustling, world-famous enterprise, expanding to over 45 buildings, 800 looms and employing over 4,000. During World War II, they produced tents and blankets for the military and put nearly 7,000 people to work, 24 hours a day. After the war, however, due to competition from inexpensive imports and a changing labor environment, the mills fell on increasingly harder times and finally ceased operations on June 24, 1954. The buildings were used for various purposes and, during various times, hosted tenants such as Gestetner (their first U.S. headquarters) and the famed Finkelstein Hat Company.

In 1978, two of the mill's principal buildings were purchased and renovated by Yonkers Industrial Development Corp. Operating under a philosophy, YIDC's president calls "adaptive re-use," the buildings became the Yonkers Industrial Park and at first was used mainly for manufacturing. During the early 1990s, the renovated  property was converted into and renamed Nepperhan Business Center. Since then, it has attracted tenants from every area of business life, stimulated reinvigorated economic development in its surroundings and given rise to the more retail-oriented Nepperhan Plaza directly across the street.

In 1983, Nepperhan Business Center was declared a national historic landmark by the U.S. Department of the Interior.