How can it be that there was nobody in Potifar's house on that day?
Rashi and Rashbam: That day was a festival, and everybody went to celebrate; 1 so Yosef's mistress feigned illness in order to be alone with Yosef in the house.
Seforno: "Ba'Bayis" means (not in the house but) in the room. "On that day" simply means that, on that day, Potifar's wife happened to have a strong desire to be intimate with Yosef.
Rashbam: To watch the Nile overflow its banks. According to the Gemara (Sotah 36b), they went to their house of idol-worship to celebrate some festival or other.
What does the Torah mean when it writes that Yosef entered the house "to do his work"?
Rashi #1 (citing Sotah 36b) and Targum Yonasan: It means literally that he entered the house innocently 1 in order to do his work (his accounts - Targum Yonasan). 2
Rashi #2 (citing Sotah 36b): He entered the house knowing that she was there, 3 with the intention of sinning. 4
Unaware that his mistress was there.
Ha'amek Davar: Because he entered for his work, he would be preoccupied with it, and did not fear having thoughts [about her. Also, he never fathomed that she would be so brazen to entice him during the day].
Riva: Since the Torah concealed it, how could an Amora reveal it? (The Gemara asks this about Tzelofchad.) Riva answers that since he overcame his desire, it is a praise. (Moshav Zekenim asks how this could be, since he already declared "v'Chatasi l'Elokim"?)
Oznayim la'Torah: And the reason that Chazal expounded derogatively, even though the words do not imply this, is because it ought to have occurred to Yosef that his mistress would take advantage of the fact that there was nobody in the house, and gone to do his work somewhere else.
Why does the Torah add the words, "me'Anshei ha'Bayis"?
Tosfos to Sotah 36b (citing R. Moshe ha'Darshan): It implies that somebody who was not a member of the household was there; and that somebody was the image of his father Yaakov. 1
See Sifsei Chachamim.
QUESTIONS ON RASHI
Rashi writes [in his second Perush] that Yosef entered with intent to sin. How then did he avoid sinning?
Rashi: His father's image appeared to him. Sotah 36b - He appeared in the window, to inform him that, if he sinned, his name would be erased from the list of tribes on the Efod. 1 Yosef responded by inserting his hands in the ground, and his semen emerged from between his finger-tips. 2
Hadar Zekenim: It says "Ein Ish" - he checked and found that he has no male organ. Hashem did so to prevent him from sinning.
Rashi writes: "The reflection of his father's image appeared to him." The Gemara (Sotah 36b) says that Yaakov's image appeared to him in the window, and told him that if he were to sin, his name would not appear among the other tribes on the stones of the Efod. Why was this so?
Maharal (Chidushei Agados Vol. 2, p. 73, to Sotah 36b): Adultery is the essence of being Chomer (base physicality), the very antithesis of the traits of Kedushah and Taharah. One involved in adultery could not be included on the holy Efod.
Rashi writes: "The reflection of his father's image appeared to him." Why did this occur?
Rashi writes that the image of his father miraculously appeared to Yosef to save him. How did Yosef merit this?
