Why did Hashem need to stop others from harming them?
Rashi (Divrei ha'Yamim I, 16:21): The norm is to afflict travelers when they come to a lodging place. They make accusations against them. Hashem did not let others harm or oppress 1 [the Avos and their families]. "Le'ashkam" is like "Oh Ashak Es Amiso" (Vayikra 5:21).
Malbim (15): The Avos rebuked people against idolatry and called in the name of Hashem.
Rashi (ibid.): Do not say that others did not harass the Avos because they did not notice them. "He rebuked kings about them" - Pharaoh oppressed Avraham, Avimelech oppressed Yitzchak, Lavan and Shechem oppressed Yakov, and Hashem told them not to! (Where did Hashem speak to Avimelech about Yitzchak, or to Shechem?! - PF)
Where do we find that Hashem rebuked kings about them?
Rashi: "Va'Ynaga Hashem Es Pharaoh", "Ki Atzor Atzar Hashem Be'ad Kol Rechem l'Veis Avimelech" (Bereishis 12:17, 20:18).