In 1858, following the opium war, Taiwan was forced to open its ports for free trade under Tienjin Treaty.  In 1862, Danshui was officially opened for free trade, and the custom office was set up.  Initially, there were only few foreign taxation officers; most of them lived in the customs office building or rental houses near by.  The Danshui Customs bought a piece of land in 1866 and started building the residences in 1869.

　　The building paralleled with the foreground and forms a square layout, which provided a splendid view of the Danshui River and Mt. Guanyin.  The foundation of the building is raised 3-foot high, with ventilation holes to prevent humidity. The building is a classic colonial style building with ash white outer walls, an arch, balcony, and veranda, together with equipments like fireplaces and chimney.  Since the walls of the building were painted white, it was also nicknamed “Xiao Bai Gong”, meaning the little white house.