WARNING: The following contains spoilers for X-Men: Legends #5, on sale now from Marvel Comics.
The X-Men: Legends series has been revealing previously hidden chapters from the history of the X-Men and their allies. This includes an unseen chapter in the lives of the X-Factor team recruited to work at the behest of the United States government -- who encountered subtle persecution from not just the public and their government, but according to Quicksilver, even their fellow heroes.
X-Men: Legends #5 by Peter David, Todd Nauck, Rachelle Rosenberg, and VC's Joe Caramagna focuses on a government hearing that takes place following a battle at the Latverian Embassy. A group of mutants invaded the embassy, hoping to bring change to their home nation by force. Killing one civilian and taking the rest of the staff hostage, the mutants -- identified as Samhain, Beltane, Lughnasa, and Imbolc -- demanded that the UN assist them in usurping the nation from Doctor Doom under the threat of killing everyone in the embassy. The US Senate initially assumed that, since it was a mutant-centered crisis, X-Factor would be the first heroes sent into the fray.
But, as Havok explains, the U.N. took precedence over the United States, considering the international nature of the incident. Rather than immediately entrusting the mission to X-Factor, the UN instead called upon the Avengers, who were reportedly battling an alien invasion. They next called on the Fantastic Four who were also busy at the time. Finally, with no other options, they turned to X-Factor. Although both hero teams had explanations for why they couldn't get involved, Quicksilver revealed a different theory as to why they didn't want the mission themselves: they didn't want the negative publicity. As Quicksilver notes, the public perception of Doctor Doom as a tyrant meant a lot of the population -- championed by J. Jonah Jameson -- supported the Latverian mutants.
If the Avengers or Fantastic Four had been seen "assisting" Doctor Doom in bringing down the assembled mutants who wanted to bring democracy to their homeland, it could be seen as an unpopular use of their power. However, as Quicksilver argues, X-Factor would be perfect to carry out the mission because "no one cares about their reputation." It's a harsh view of the other heroes and their perception of X-Factor, but it's not an unfounded one. Across the history of the Marvel Universe, plenty of conflicts have been considered "mutant problems" that are beneath the attention of heroes like the Avengers -- such as the massacre of countless mutants over the years at places such as Genosha.
To the other heroes (at least by Quicksilver's observation), X-Factor was a government-sponsored and controlled branch of mutant heroes, capable of completing missions that were deemed necessary but that no one who cared about their public image would ever approach. The fact that Quicksilver -- a former Avenger -- would view his nominal allies in such a light paints the other heroes in an unflattering light. Even the fact that the other heroes don't seem to have attended the hearing to offer their support or testimony for their allies suggests Quicksilver might be onto something. As some of Earth's most popular and well-known figures, it makes sense for the heroes to be aware of how the public perceives them.
But by allowing the mutants to take the brunt of the bad press over the years rather than sully their own hands, groups like the Fantastic Four and the Avengers may have played a role in the public perception of mutants over the years.