What exactly, does "va'Yisnaklu Oso la'Hamiso" mean?
Ramban (citing Bereishis Rabah 84:14): It refers to the previous phrase ("uv'Terem Yikrav Aleihem"). They planned to kill Yosef (before he even arrived), by setting dogs on him 1 - in order to avoid having to murder him directly). 2
Seforno: They assumed that he is Misnakel (planning to kill) them, 3 or to make them sin and earn their father's curse (or Divine retribution), 4 and that he had not come on a peaceful mission.
Rashi: "Oso" is like Ito, Imo, i.e. Eilav (about him). 5
Ha'amek Davar: They judged that he is Misnakel (planning to kill) them, and therefore they are justified to kill him.
Nezer ha'Kodesh (on the Midrash): We learn (Pesachim 118a) from "la'Kelev Tashlichun Oso" (Shemos 22:30) that one who speaks Lashon ha'Ra (as Yosef had done) deserves to be thrown to the dogs.
And it was only when that plan of action failed that they discussed killing him directly when he arrived.
Ha'amek Davar: If so, it should not say "'la'Hamiso"!
Seforno: So that he alone of all Yaakov's sons would remain blessed. And the Torah says that if someone comes to kill you, beat him to the draw and kill him first' (Sanhedrin 72a). Otherwise, following all the harm that they caused him and the lack of pity that they displayed towards him when he pleaded with them, it is inconceivable that they would all have been considered complete Tzadikim and that their names would have appeared on the Efod.
Divrei Eliyahu, Kol Eliyahu: "Va'Yisnaklu" is 'plotting against,' as in the verse, "Asher Nikelu Lachem Al Devar Pe'or" (Bamidbar 25:18). It is not appropriate to say "Oso"! (This pushed Seforno to explain that they assumed that Yosef plotted against them - refer to 37:18:1:2 - PF). Therefore, Rashi explains that Oso is like Ito, and we find that Ito and Imo can mean 'about him' or 'to him,' e.g. "Asher Hufkad Ito" (Vayikra 5:23), "l'Daber Ito" (Shemos 34:35), "Lo Na'aseh Imo Davar" (Esther 6:3).
How could the Shevatim, who were great Tzadikim, plan to kill Yosef?
Moshav Zekenim: He was Chayav Misah by making himself idolatry, through saying "the moon and stars are bowing down to me."
R. Chaim Paltiel: Yosef said that they transgress Arayos and Ever Min ha'Chai, for which a Ben Noach is Chayav Misah, so he is like a Rodef, whom one kills to save the victim. Malbim - They judged that he is like a Rodef, trying to eradicate them from this world and the world to come.
Shelah (Vayeshev 40): They judged that he had rebelled against Malchus Beis David (from Yehudah).
Seforno: Refer to 37:18:1:2 and the note there.
Bereishis Rabah (84:14): They saw that later, he (i.e. Yarav'am, who descends from him) would lead people to serve idolatry. 1
Keli Yakar (to 37:26): A talebearer causes murder. Lashon ha'Ra kills three (the speaker, listener and subject) - and if someone seeks to kill you, kill him first!
Ohr ha'Chayim: Yosef falsely told Yaakov that they transgressed Ever Min ha'Chai or Arayos. A Ben Noach is killed for these, even through one witness, even a relative, and even without warning. They judged him to be an Ed Zomem. 2
Later, Yosef told them - Just like you judged your brother based on the future (refer to 37:17:1:3), so I judge Binyamin (Aderes Eliyahu of the Ben Ish Chai, Vayigash) or you (to be Meraglim - Bris Torah Miketz DH v'Yuvan).
Ed Zomem is only one who was not where he claimed to see the testimony, and he is killed only if the defendant was sentenced to die through his testimony! Perhaps these conditions are only for Beis Din to kill him, but any witness who seeks to kill through false testimony is Chayav Misah bi'Yedei Shamayim. (PF)
Why does it say that they saw him from afar, "and before he came close..."?
Ohr ha'Chayim #1: This teaches that he was very far away.
Ohr ha'Chayim #2: This teaches that "from afar" means far from brotherhood, and "before he came close" is another matter.
Ha'amek Davar (to 37:19): Before he came close, they looked for a way to kill him without overtly spilling blood. 1 They did not find a way until he came close.
QUESTIONS ON RASHI
Rashi writes: "'Oso' - is like the word 'Ito', 'Imo', i.e. 'Eilav.'" What does Rashi mean?
Gur Aryeh: The preceding word, "va'Yisnaklu," is a verb in reflexive (Hispa'el) form. Therefore, the phrase "va'Yisnaklu Oso" is difficult. 1 "Oso" must be understood as 'Eilav' - meaning "they became treacherous, towards him." In order to do so, we interpret "Oso" as 'Ito' (spelled the same way), meaning 'Imo' - 'with him,' i.e. towards him.
The Hispa'el verb form indicates an action one does to oneself, not to someone else; while the pronoun Oso associates a verb with another object that receives that action!