According to Josephus, Kos had a Jewish Community as early as the Second Temple.

According to Maccabees I (15.23), Jews had been living on the island since 142 B.C. In 49 B.C., under the Roman rule, the residents of the island were ordered by Governor Fanios to respect the decree of the Government and permit the safe passage of Jewish pilgrims on their way to Jerusalem.

It is known that the Knights of Rhodes expelled the Jews for two centuries, but after they were defeated by Sultan Suleiman the Great, the Jews returned in 1522. By 1685 a small Community, which depended on the Chief Rabbi of Rhodes for religious services, had settled in Kos. In 1747 Eliezer Tarsia financed the construction of a small but impressive Synagogue.

 In 1850, 40 families lived in Kos, but were reduced to 25 in 1872. In 1901 the Jewish Community of Kos had only 10 families. Its members exported grapes and raisins, were involved in metal and clothing trade and spoke Greek, Turkish and Judeo-Spanish. The Community grew significantly during the Greek-Turkish War between 1918 and 1922, as refugees came from Asia Minor and particularly from Smyrna.

In 1933 Kos was shaken by a very strong earthquake. As a result many human lives were lost important monuments, including the Synagogue, were ruined. Immediately after the destruction of the old Synagogue a new one was built very close to the port. It exists up to this day.