Malbim: Hashem sent him to prophesize only in Tofes. He decided on his own to go to Chatzer Beis Hashem and prophesize there. This explains why Pashchor hit him (20:2). Where he was sent to prophesize, others could not rule over him, for he was Shali'ach Hashem. What he did on his own without a command, others could rule over him.