R. Yonah #1: Pleasures are to sustain the body, to conduct its health and widen his Da'as. Thought and learning Chachmah require Harchavas Da'as. The Gemara (Brachos 57b) says that a nice dwelling, a nice wife, and nice vessels give Harchavas Da'as; this is also for a Chacham. This should be a person's intent in pleasure. A Kesil intends to fulfill his desire, and he is drawn after physicality. This distances him from the ways of the Nefesh and Chachmah - "va'Yishman Yeshurun va'Yiv'at", "Pen Tochal v'Savata", "v'Ram Levavecha v'Shachachta" (Devarim 32:15, 8:12, 14). A Kesil with pleasure is compared to an animal; also it eats and drinks.

R. Yonah #2: It is proper for a Kesil to humble and lower himself. Pleasure is the opposite of what is proper for him.

Radak (Yeshayah 58:14, citing Rav Sadya Gaon: Your bodily delight will be in Hashem, and unlike fools, about whom it says "Lo Navah li'Chsil Ta'anug." A wise person will not over-indulge, rather, the proper amount, for he engages in Chachmah; he delights more in it. Bodily delights in the proper amount benefit the Sechel and strengthen its three powers - memory, discernment and thought.

Malbim: A Kesil knows the laws of Chachmah, just his desire overpowers him. He desires pleasures of the world. It is not proper for him. Even though the body delights, the Nefesh become submissive to the body's desires. It is improper for the body to rule over the Nefesh. The body was created to be a slave to the Nefesh; the Nefesh should rule over the body's powers!

Rashi: Pleasure is improper for a Kesil, and all the more so a slave should not rule over Nedivim. R. Yonah #1 - honor is dearer than pleasures. Therefore, it is a greater disgrace when honor is taken from someone worthy of it and given to someone not worthy of it.

R. Yonah #2: Also this is opposite - lowliness is proper for the slave, and honor for the officers.

Malbim: This is why pleasure is improper for a Kesil - it causes the slave (body) to rule over the Sar (Nefesh; refer to 19:10:1:3).