In February 1883, 33 Jewish men, including several members of the Weil family, "united for the purpose of building a synagogue" in Goldsboro. The Jewish community already owned a Torah and was home to a Cemetery Association and a Ladies' Hebrew Assistance Society, indicating that Jewish life existed in Goldsboro prior to the formal creation of a congregation. The expansion of the town's communal and educational institutions and activities in the early 1880s likely prompted Jewish families to follow a similar path and formalize their organization.

Within a few months, the congregation had established regular services, a Sabbath school, a day school and a choir, and had begun preparations for constructing a temple building. It had also hired the Rev. I.M. Bloch to serve as "rabbi, reader, teacher and shochet [ritual slaughterer]." The synagogue building was dedicated with a gala ceremony on December 31, 1886.

With several congregants, including the Weils, having connections to Baltimore Jewry, the Goldsboro Jewish community decided to model its congregation—in name and form of worship—on Baltimore's Oheb Sholom Temple. In the early days, the temple's constitution specified that the Sabbath and Holy Days were to be governed by "Biblical injunction, rather than by expediency," and religious services were to be held exclusively in Hebrew. Yet the choir was mixed-sex, and an early rule requiring the wearing of hats during services was soon dropped in favor of a provision that read, "nothing shall be done about the wearing of hats." By the late 1890s, the sermon and much of the service were delivered in English, indicating a steady turn toward Reform practices.