Circa 1820, this settlement beside a scenic waterfall on Cooligan Creek was known as "Smith Mills" after Timothy Smith, an early settler. In 1851 when a post office was established it was found that there were too many places in Upper Canada called "Smith". The local teacher suggested "Haarlem" after his native village in Holland. The name was approved but one "a" was dropped. Throughout the 19th century Harlem was a thriving community centred on its mills powered by water wheel and turbine and its cheese factory. By the mid 20th century, Harlem had become a quiet residential community.
In the vicinity of Harlem along the Harlem Road and the Thousand Acre Road can be found a number of fine 19th century homes. Of particular note are the John Green House, built in the 1860s (Bastard Ward, South end Lot 15, Con 5) and the James Hanna House, built in the mid-1850s (Bastard Ward, Lot 16, Con 4).
