Esther Herrman was born on August 7, 1823, in Utrecht, the Netherlands, to Sophia (Van Ysen) and Emanuel Mendels. She had three sisters, Gamma (b. 1821), Jette (b. 1821), and Adelaide (b. 1825), and came to the United States as a child following her mother’s death in 1827. In 1843, she married Henry Herrman, a native of Baden who was born October 13, 1822. By 1847, Esther and Henry had moved from New York City to New Bedford, Massachusetts, where he started a business supplying sailing vessels. Their first children were born there: Sophia (1847) and Henrietta (1848). They moved to Boston, where Henry operated a clothing business and their son Abraham was born (1850).