Why does it say "v'Atah Kadosh Yoshev"?
Rashi: You are Kadosh, and You sit [to hear the praises of Yisrael from days of old].
Radak: How do You not answer now - You are Kadosh and dwell (exist) forever! You have the same power as when You saved us from Egypt - how do You not save now?! Yoshev exists forever - "Atah Hashem l'Olam Teshev" (Eichah 5:19).
Malbim: You are Kadosh, separate from any aspect of physicality. Yeshivah shows being fixed; You never change. If so, how did You change now? You always saved our fathers. Now You do not save! It is as if there is weakness in You, and a physical aspect - this is the opposite of Kedushah! Or, if You change, this is the opposite of Yeshivah (being fixed, without changing)!
What are "Tehilos Yisrael"?
Bereishis Rabah (68:11): Yisrael is Yakov Avinu. All 20 years in Beis Lavan, he said all of Sefer Tehilim.
Rashi: The praise of Yisrael from days of old.
Radak: What Yisrael praised You when You saved them from Egypt.
Malbim: Yisrael praised You, that You are Kadosh, separate from any aspect of physicality, and Yoshev (fixed, without change).