Gemara

Question: To what extent must one rebuke?

Answer #1 (Rav): One must rebuke until the point that he (the transgressor) is ready to hit him.

Answer #2 (Shmuel): One must rebuke until he is ready to curse him.

Answer #3 (R. Yochanan): One must rebuke until he gets angry.

In a Beraisa, R. Eliezer holds like Rav, R. Yehoshua holds like Shmuel, and Ben Azai holds like R. Yochanan.

All of them learn from "va'Yichar Af Sha'ul bi'Yehonason... va'Yatel Sha'ul Es ha'Chanis Lehakoso";

The three opinions learn from, respectively, "Lehakoso," "l'Voshtecha ul'Voshes Ervas Imecha"; and "va'Yichar Af Sha'ul."

Question (against the third opinion): Even after Sha'ul was angry, Yonason rebuked him to the point of hitting and cursing!

Answer: R. Yonason loved David exceedingly, therefore he rebuked his father (for wanting to kill David for no reason) more than one is required to.

Yevamos 65b (R. Ilai): Just as it is a Mitzvah to say something that will be heeded, it is a Mitzvah not to say something that will not be heeded.

(R. Aba): This is an obligation - "don't rebuke a scoffer, lest he hate you. Rebuke a Chacham, and he will love you."

Beitzah 30a (Abaye): If we know that people will not accept rebuke we are silent. It is better that Yisraelim transgress b'Shogeg than b'Mezid!

Assumption: This only applies to a transgression mid'Rabanan, but regarding mid'Oraisa, we protest even if they will not listen.

Rejection: Mid'Oraisa, one must add to Yom Kipur (begin fasting before nightfall), yet some eat until night, and we do not protest!

Shabbos 55a: R. Zeira asked R. Simon to protest against the household of the Reish Galusa. R. Simon said that it would not help.

R. Zeira: Still, you should protest (you do not know for sure)!

Rav Acha b'Rebbi Chanina: Only once, Hash-m said that He will do something good and retracted - "V'Hisvisa Tav Al Mitzchos ha'Anashim ha'Ne'enachim veha'Ne'enakim".

Hash-m told Gavri'el to write the letter 'Tav' in ink on the foreheads of Tzadikim in Yerushalayim, in order that angels of destruction will not harm them, and to write a 'Tav' in blood on the foreheads of Resha'im, so that angels of destruction will kill them.

Midas ha'Din: Why will these be spared and these will be killed?

Hash-m: These are absolute Tzadikim, these are absolute Resha'im!

Midas ha'Din: The Tzadikim could have protested!

Hash-m: I know that it would not have helped.

Midas ha'Din: You know that, but they did not know!

This explains the retraction in the verse. At first, "v'Al Kol Ish Asher Alav ha'Tav Al Tigashu", but then, "umi'Mikdashi Tachelu va'Yachelu ba'Anashim ha'Zekenim Asher Lifnei ha'Bayis".

(Rav Yosef - Beraisa): We read "Mikdashi" (my Mikdash) like 'Mekudashai (my holy ones)', people who fulfilled the Torah from Aleph to Tav.

Tana d'vei Eliyahu Rabah 18:40: "Hoche'ah Tochi'ach Es Amisecha" - this refers to someone of your nation, i.e. he observes Torah and Mitzvos. It does not apply to a Rasha.

Rishonim

Rif (Yevamos 21b and Bava Metzi'a 17a) and Rosh (6:21): Just like it is a Mitzvah to say something that will be heeded, it is a Mitzvah not to say something that will not be heeded. "Don't rebuke a scoffer, lest he hate you."

Rosh (Beitzah 4:2): The Gemara says that if we know that people will not accept rebuke we are silent. It is better that Yisraelim transgress b'Shogeg than b'Mezid, even regarding an Isur mid'Oraisa. Ba'al ha'Itur says this refers to a mid'Oraisa law that is expounded. However, if something is explicit in the Torah we protest and punish until people stop transgressing.

Rambam (Hilchos De'os 5:7): A Chacham speaks only in a situation where he sees that his words will be heeded. If not, he is silent.

Source (Kesef Mishneh): We learn from Yevamos 65b "don't rebuke a scoffer, lest he hate you."

Rambam (6:7): If one gave rebuke and it was not accepted, he must rebuke a second and third time. He must rebuke constantly until the transgressor hits him and says that he will not listen.

Hagahos Maimoniyos (3): Tosfos holds that if one knows that the sinner will not listen, it is better to say nothing. R. Eliezer mi'Mitz says that one will not be punished for not rebuking in such a case, but there is a Mitzvah to rebuke. SemaK challenges this from Yevamos 65b, which says that it is a Mitzvah not to say if he sees that his words that will not be heeded.

Shoham v'Yashfe (brought in Likutim, Frankel Rambam De'os 6:7): The Rema rules like the Ritva, who distinguishes Shogeg from Mezid. However, Tosfos (Shabbos 55a DH) holds that even regarding Mezid if one knows that protest will not help it is better that the person transgress without warning than with warning.

Alshich (brought in ha'Rif in Ein Yakov): There was no criticism or decree against Tzadikim that had no authority and were unable to protest.

Poskim

Rema (OC 608:2): Regarding every Isur not explicit in the Torah we say that it is better that people transgress b'Shogeg than b'Mezid. If the Torah explicitly forbids something, we protest.

Mishnah Berurah (5,7): If the Torah explicitly forbids something, we assume that the transgressor is Mezid. Whenever there is danger one need not protest.

Kaf ha'Chayim (7,9): If we are in Safek whether or not someone transgresses an explicit Isur Torah (e.g. Bein ha'Shemashos), according to the opinion that Safek mid'Oraisa l'Chumra is mid'Oraisa we must protest. Since many Poskim hold like this, one should be stringent and protest. The Ritva (brought in Shitah Mekubetzes Beitzah 30a) says that nowadays that people are lenient about many things, we should protest even regarding mid'Rabanan laws. The Kesav Sofer (Teshuvah 57) says that one must gauge what is proper according to the time and place.

Kaf ha'Chayim (16,17): One need not spend money to protest something explicit in the Torah. By protesting one removes himself from Arvus (responsibility of every Yisrael for every other Yisrael).

Bi'ur Halachah (DH Chayav): One should rebuke someone with whom he is familiar. One should not rebuke a stranger lest the latter hate him or take vengeance against him.

Question: Shabbos 55a connotes that had the Tzadikim known that their rebuke would not be heeded they would not have been punished, even though people were transgressing explicit Torah laws!

Answer (Bi'ur Halachah DH Aval): If one knows that rebuke will not be heeded, he will not be punished if he does not protest. Nevertheless, the Rema holds that it is a Mitzvah to protest against something explicit. Semag exempts in such a case; Tosfos exempts in public. Even the Rema does not obligate rebuking people who Mechalel Shabbos in public or sin to anger Hash-m. They are not Amisecha. Tana d'vei Eliyahu teaches that rebuke does not apply to a Rasha. It is not clear if one must rebuke one who eats Neveilah for pleasure or is Mechalel Shabbos in private.

Bi'ur Halachah (DH Mochin): Birkei Yosef says that the Chiyuv to protest regarding explicit Torah Isurim is only if we have power to stop the transgressors. Semak says that in any case it is a Mitzvah to protest.

Chafetz Chaim (Petichah Asin 5 and Be'er Mayim Chayim): If Shimon starts telling Lashon ha'Ra to Reuven, and Reuven thinks that it might help to protest, he is obligated to rebuke him. If he allows him to finish the tale Reuven transgressed "Hoche'ach Tochi'ach". The Mitzvah of rebuke applies to even before the transgression was done (Rambam, Sefer ha'Mitzvos 205).