I'm not talking about Byzantines and Ottomans; I'm talking about Muhammad the warlord and his jihad successors, the Rashidun and Umayyad Caliphates. They offered conversion or death across the whole Middle East. Early Muslims weren't exactly tolerant of pagans, so for Israel to "give land back to the pagans" is impossible at this point. No pagans left to give it to.