Gemara

(D'Vei R. Yanai): Four Kavim is the quantity for prayer.

Question: To which law does he refer?

Answer (Beraisa): If one was carrying a load on his shoulder, if it is less than four Kavim, he casts it on his back and prays. If it is four Kavim or more, he must put it down before praying.

Sukah 41b - Mar bar Ameimar (to Rav Ashi): My father would pray while holding his Lulav.

Question (Beraisa): One may not pray while holding Tefilin in his hand or a Sefer Torah in his arm.

(Shmuel): The same applies to holding a knife, coins, bowl, or loaf.

Answer: It is not a Mitzvah to hold them, and they distract them. It is a Mitzvah to hold the Lulav; it does not distract him.

Rishonim

The Rif and Rosh (Sukah 20a and 3:32) bring the Gemara in Sukah.

Ran (20a DH Aba): Holding Tefilin in his hand is not a Mitzvah, therefore he is dcd with them until he puts them on. The Mitzvah of Lulav is dear to him and guarded, so it does not distract him.

The Rif and Rosh (Berachos 14b and 3:28,30) bring our Gemara

R. Yonah (DH Sachin): Rashi holds that we forbid things about which he will be distracted lest they fall. A knife can damage him, a food in a bowl can get ruined, the loaf will get dirty, and coins can scatter. One may hold things that, if they drop, there will be no loss. The Gemara asked about Lulav. Perhaps it is forbidden, for he is distracted lest it fall and become Pasul. Or, because the Mitzvah is dear to him, he will not drop it and he is not distracted. We conclude the second way, it is like things that will not be ruined if they drop. Others say that we just gave common examples, but one may not pray holding anything, even if it would not be ruined if it dropped. Only a Lulav is permitted; because the Mitzvah is dear to him, he will not be distracted.

Rambam (Hilchos Tefilah 5:5): One may not pray while holding Tefilin in his hand or a Sefer Torah in his arm, for it distracts him. He may not hold Kelim or coins in his hand (at the time). He may pray with a Lulav in his hand during Sukos, for it is the Mitzvah of the day. If one was carrying a load on his head and the time for prayer came, if it is less than four Kavim, he casts it on his back and prays. If it is four Kavim, he puts it on the ground and prays.

Poskim

Shulchan Aruch (OC 96:1): One may not pray while holding in his hand Tefilin, a Sefer of Kisvei ha'Kodesh, a full bowl, a knife, coins or a loaf, for he thinks about them, lest they fall. This distracts him and spoils his concentration.

Mishnah Berurah (1): The Pri Megadim says that the same applies at the time of Kri'as Shema and Pesukei d'Zimrah.

Mishnah Berurah (2): B'Di'eved, one need not pray again, unless due to this he did not have intent in the first Berachah of Shemoneh Esre (for he will have intent if he prays again - Sha'ar ha'Tziyon 2). A Shali'ach Tzibur may hold the Sefer Torah while saying "Yekum Purkan." Since he prays for those who learn, he is not holding it to guard it, so it is like a Lulav in its time. Likewise, one may hold a Sefer Torah during Birkas ha'Chodesh.

Mishnah Berurah (4): Similarly, one may not put an infant in front of himself at the time of Tefilah.

Kaf ha'Chayim (4): Tosfos (Chagigah 3a DH Kedei) says that we bring (young) children to the Beis ha'Keneses. However, at the time of prayer we take them out.

Taz (2): The Shulchan Aruch (97:5) requires casting a load on his back. This proves that one may not hold anything in front of him, even if it would not get ruined if it falls.

Mishnah Berurah (6): If one fears lest Tefilin or a Sefer Torah be stolen, perhaps one may hold it during Tefilah. This requires investigation.

Kaf ha'Chayim (7): Surely, if he can wrap it in a cloth and hang it from his belt, it is permitted. We forbid only in his hand. Here, there is no concern lest it fall. If this is impossible, it is better to hold it in his hand than to leave it on the bench, for his mind is more settled this way. The Yerushalmi says so regarding coins.

Kaf ha'Chayim (5): One may pray with eyeglasses only if they fit his nose well or are tied on and will not fall.

Note: In those days, most eyeglasses were not secure and were prone to fall. This is why the Mishnah Berurah (301:44) forbade wearing them on Shabbos where there is no Eruv.

Shulchan Aruch (ibid): He may pray with a Lulav in his hand in the time (of the Mitzvah). Since holding it is a Mitzvah, it will not distract him.

Kaf ha'Chayim (9): Similarly, one may hold Tzitzis in his hand.

Shulchan Aruch (2): One may hold a Machzor of Tefilos in his hand while praying. Since he holds it for the sake of Tefilah, it will not distract him. It is like a Lulav. Since taking it is a Mitzvah, he is not distracted.

Source (Terumas ha'Deshen 17, cited in Beis Yosef DH v'Chasuv): All Kisvei ha'Kodesh are forbidden, like a Sefer Torah. Machzorim and Tefilos are permitted only if he holds it to pray from it deeply and precisely.

Mishnah Berurah (7): If a Sefer fell on the floor in front of him, he may pick it up after finishing the Berachah he is in only if due to it he cannot concentrate. All the more so, one may not gesture with his finger.

Rema: If he was not holding a Sidur before he began praying, he may not pursue one during Tefilah, unless it was in a prepared, designated place. Then, one may go to get it during Tefilah to pray from it.

Source: Beis Yosef ibid., citing R. Yonah 15b Sof DH v'Asur.

Kaf ha'Chayim (11): If one is unsure about a law, he may go to where he knows that there is a Sefer in which he can find the Halachah.

Mishnah Berurah (8): Similarly, one who prays from a Sidur must be careful to mark in advance all the places he needs to pray from, lest he need to search during Tefilah and cease to concentrate.

Mishnah Berurah (9): Also during Chazaras ha'Shatz it is proper to have a Siddur in front of him and see what the Shali'ach Tzibur says.

Shulchan Aruch (97:5): If one was carrying a load on his head and the time for prayer came, if it is less than four Kavim, he casts it on his back and prays. If it is four Kavim, he puts it on the ground and prays.

Beis Yosef (DH Hayah): Our text, and that of the Rif and Rosh, says that there was a load on his shoulder. The Rambam's text said 'it was on his head.' According to this, one may pray with a load on his shoulder. The Tur did not mention casting the load on his back. It seems that he explains that this is not obligatory. The Beraisa permits praying while they are on him.

Mishnah Berurah (17): A load of four Kavim on him would distract him.

Kaf ha'Chayim (29): Also, people do not stand in front of kings bearing a big load like this.