Gemara

(Beraisa): "Lo Sochlu Chol Nevelah la'Ger (... Titnenah va'Achalah)" - it is called (and forbidden like) Nevelah only if a Ger (Toshav) would eat it.

Avodah Zarah 67a (Rav Yehudah and Rabah bar bar Chanah): We permit when vinegar fell on hot grits, but if it fell on cold grits and then they were heated, this is like something that initially was beneficial and later was Pogem (harms the taste). It is forbidden.

Version #1 (Reish Lakish): We do not permit due to Nosen Ta'am li'Fgam if the mixture tastes bad due to too much or too little salt or spices;

Rather, Nosen Ta'am li'Fgam permits if it is properly seasoned, and people do not eat it due to the Pegam.

Version #2 (Reish Lakish): Do not say that if a mixture had more or less seasoning it would taste good, therefore we do not permit due to Nosen Ta'am li'Fgam;

Rather, if (as it is seasoned) now the Isur detracts from the taste, it is permitted.

Rishonim

Rambam (Hilchos Ma'achalos Asuros 15:28): If Isur gives taste to Heter, it totally forbids it. This is if it improves the taste. If the Isur harms the taste of the Heter, it is permitted. This is if it harms the taste from the beginning to the end. If it was beneficial and them Pogem, or Pogem and then beneficial, it is forbidden.

Rashba (Toras ha'Bayis 2:1:19a, brought in Beis Yosef YD 103 DH u'Mah she'Chasav u'Fgam): We permit Nosen Ta'am li'Fgam when a small amount of Isur was mixed with much Heter, but if much Heter was mixed with a small amount of Heter, or even if equal amounts mixed, we permit Nosen Ta'am li'Fgam only if it is totally spoiled, like a Nevelah that a Ger would not eat. Regarding a mouse that fell in beer, if the Heter exceeds the Isur, the Isur is Batel. The taste of the Isur, which can be felt, is not a reason to forbid the Heter. Just the contrary, it harms the taste, like we say about other matters that are Nosen Ta'am li'Fgam. It seems that even if Isur fell into a small amount of Heter and it is recognized, one removes the Isur and the liquid is permitted, for it has only taste without substance. Even though (normally) we estimate based on the entire volume of the Isur (and require 60 times the volume), because we do not know how much left, here there is no benefit from the taste it emitted, so it does not forbid the mixture at all. This is like we say about a pot that is not a Ben Yomo (it did not absorb in the last 24 hours). Surely it did not emit everything. When the emission is beneficial, we are stringent to be concerned lest it emitted enough to give taste. When the emission is Pogem, we do not estimate based on its volume, for we care only about the taste, and it is as if there is no taste, for it is Pogem.

Ran (Avodah Zarah 31a DH ud'Amrinan): Rashi says that the Gemara does not really mean that people do not eat it due to the Pegam. It means only that the Isur detracts from the taste. A proof is from a pot that is not Ben Yomo. We call it Nosen Ta'am li'Fgam, and surely it is not Pogem so much that people do not eat the food! We say that even a Ben Yomo is Pogem a little! This shows that a small Pegam permits. Another proof is from oil, honey, Chiltis (a very sharp Pri) and similar things. We permit them (Avodah Zarah 38b) due to Nosen Ta'am li'Fgam. Surely they are not spoiled from human consumption. All of them are eaten! This shows that a small Pegam permits. Indeed, we learn from Nevelah, and anything a Ger would eat is called (and forbidden like) Nevelah, even though it is Pagum. However, that is because Nevelah is eaten by itself, therefore it is forbidden until a Ger would not eat it. Mixtures depend on giving taste. If it harms the taste even a little, it is permitted, for this is not 'Nosen Ta'am'. Just the contrary, it harms the taste! However, we must resolve how we learn from Nevelah. It seems that we find that the Torah forbids Nevelah only when it is proper for a Ger, for the one who eats benefits from the Isur, even though it is slightly Pagum. If it is not proper for a Ger, since he does not benefit from the Isur, the Torah permitted it, even though it was initially forbidden. Likewise, if an Isur that gives taste to Heter slightly harms the taste of the Heter, one who eats it does not benefit from the Isur, rather, he is pained by it. Therefore, it is permitted, even though the matter mixed in was initially forbidden. Therefore, I lean to say that if the Isur increased the volume of the Heter, and the bigger volume improves it more than the Pegam harms it, it is forbidden unless it is spoiled from human consumption. Any Pegam suffices when this spoils the taste more than the improvement of the added volume. However, the Rashba permits whenever the Heter is more than the Isur and it is Nosen Ta'am li'Fgam. Mid'Oraisa, the substance of the Isur is Batel in the majority. We are not concerned for the taste, since it is Pogem. I am uneasy with this Heter.

Ran (32a DH Na'aseh): We do not say that since later he will cook the grits, we consider it as if the vinegar was Pogem from the beginning. Rather, it is like a food that was improved due to Isur, and later spoiled by itself. It is forbidden until it is spoiled from human consumption.

Poskim

Shulchan Aruch (YD 103:2): The Pegam need not spoil the mixture so much that one is loathe to eat it. Rather, even if it slightly harms the taste, it does not forbid the mixture.

Shach (2): If an Isur does not improve or harm the taste, it forbids. The Beis Yosef brings this from Hagahos Ashri (Avodah Zarah 5:9). I bring a proof from the Gid ha'Nasheh. If it dissolved, we require 60 times its volume, even though it is like wood and gives no taste. The same applies to any Isur. We require 60 times, even if it gives no taste. The Yerushalmi says so.

Sifsei Da'as: The Pri Chodosh (2) disagrees vehemently. Gid ha'Nasheh is different, for the Torah forbade it. The Shach himself (114:21) said so. I say that even though the Torah permits an Isur without taste, if it dissolved into food, one benefits from it and the Torah forbids it.

Shulchan Aruch (ibid.): Some say that this is only if a small amount of Isur was mixed with much Heter, but if much Isur was mixed with a small amount of Heter, or even if equal amounts mixed, we permit Nosen Ta'am li'Fgam only if it is totally ruined from human consumption.

Gra (4): If the majority is Isur, it is not Batel. It is as if it is intact.

Gilyon Maharsha (4): The Pri Chodosh says that since one is lashed for meat and milk even if he eats it abnormally, perhaps even if it is spoiled from human consumption it forbids a mixture unless the majority is Heter.

Shulchan Aruch (ibid.): If there is not intact Isur, rather, a mere taste, even if there is much Isur and little Heter, it is permitted even if it harms the taste only a little. Some are stringent to say that if the Isur increased the volume of the Heter, and the bigger volume improves it more than the Pegam harms it, it is forbidden unless it is spoiled for human consumption. This refers to when it is Pogem from the beginning to the end. If it was beneficial and them Pogem, or Pogem and then beneficial, it is forbidden.

Beis Yosef (DH ba'Meh): Even though the Gemara discussed only a taste that was beneficial and later Pogem, the Tur forbids also what is Pogem and later beneficial. Also the Rambam and Rashba forbid. It seems that a Kal va'Chomer teaches that. If we forbid in the end because it was initially beneficial, even though its end is to be Pogem, all the more so we forbid what was initially Pogem, since it is not really Pogem, since in the end it will be beneficial.

Taz (5): At the beginning of this Siman, the Tur wrote that vinegar that fell into grits is li'Fgam, and it is permitted. This is when it fell in while it was hot, and he ate it while it was still hot. The Prishah says that he permits even if it cooled off afterwards. This is astounding.

Mishbetzos Zahav: The Prishah must hold like the Pri Chodosh says, that the vinegar is blunted through the heat, so even when it cools (it is not beneficial, and) it is permitted.

Shach (7): If from the beginning until now it was Pogem, one may eat it, even if later it would be beneficial.

Rema: Some say that even though the Isur is Nosen Ta'am li'Fgam and the food is permitted, the pot is forbidden. If one cooked within 24 hours, when the Isur can give a beneficial taste, the second food is forbidden unless it was 60 times the volume of the first Isur.

Shach (8) and Gra (9): This (the absorptions in the pot) is like something that was Pogem, and later beneficial.

Rema: If one stirred the food with a spoon, and stuck the spoon into a second food and it is Pogem also the latter food , the pot is not forbidden. The same applies to something without any taste, e.g. a pot into which they melt honey. Even though it has bees' legs, the pot is not forbidden. The same applies to all similar cases.

Darchei Moshe (1 DH Kasav): Isur v'Heter ha'Aruch (32) says that Nosen Ta'am li'Fgam permits only the food, but the pot is forbidden, for it (the Isur) gives taste to the pot, and if one cooked in the pot within 24 hours a food that improves due to the Isur, it is forbidden. This is even if the Isur did not fall into the pot itself, just the food into which it fell was cooked in the pot. However, if one uses a spoon to stir a mixture in which Nosen Ta'am li'Fgam, and then stirred other food in another pot, the latter pot is permitted. Since the food is permitted, the pot is Nat bar Nat (it received a taste from something (the spoon) that received a taste) of Heter (the first mixture).

Taz (6): Why does the Rema discuss a spoon forbidden due to an Isur that was Nosen Ta'am li'Fgam? Even if it absorbed proper Isur, but it is Pogem the food it is put into, the pot is Nat bar Nat and does not forbid what is cooked in it afterwards, even if the Isur in the spoon improves the latter food! This is because the pot absorbed Nat bar Nat of Heter! Isur v'Heter clearly says so. If a spoon absorbed forbidden milk, and fell in boiling wine, he permits the pot even if it is not 60 times the volume of the spoon.