Jews have always lived there, in the Ottoman census of the mid 1800s Jerusalem was majority Jewish. This is despite a pogrom in Hebron in 1834. Tiberious, which was the home of Kaballah, and which also had pogroms, was a centre of Jewish learning since the 16th century. Safed also had a strong Jewish presence despite pogroms there. Maimonides visited the community in Palestine before taking the role of doctor to the Sultan of Egypt. For a thousand years, Jews would travel to Palestine to retire and die, or have their remains buried there.
None of the Jews who came to the Holy Land prior to Herzl were what we call today Zionist. Political Zionism did not exist. Most were Haredi, defined today as Anti-Zionist. They were not trying to form a secular state or any other state, just returning to their ancestral homeland and legally purchase land to do so. Our numbers had been kept artificially low by various rules and our land sold off to Arab and Ottoman landlords( and then had tenant farmers placed on it). We chose to buy that land back, a lot of it just swamp or barren- you could read Matk Twain's or Karl Marx's visits to Palestine, noting Al-Aqsa was covered in weeds and the place was a forgotten backwater of the Ottoman Empire.