AHD seems to think the tea name is from the fabric. They derive the fabric name from a Flemish version of the name of a town in northern France and the tea name because it is "thin and white" like the cloth.