"In a nation distracted by faction, there are, no doubt, always a few, though commonly but a very few, who preserve their judgment untainted by the general contagion. They seldom amount to more than, here and there, a solitary individual, without any influence, excluded, by his own candour, from the confidence of either party, and who, though he may be one of the wisest, is necessarily, upon that very account, one of the most insignificant men in the society. All such people are held in contempt and derision, frequently in detestation, by the furious zealots of both parties. A true party-man hates and despises candour; and, in reality, there is no vice which could so effectually disqualify him for the trade of a party-man as that single virtue. The real, revered, and impartial spectator, therefore, is, upon no occasion, at a greater distance than amidst the violence and rage of contending parties. To them, it may be said, that such a spectator scarce exists any where in the universe. [...] Of all the corrupters of moral sentiments, therefore, faction and fanaticism have always been by far the greatest."
Adam Smith Theory of Moral Sentiments 1759 Part III Chap. III end.
Further reading: Adam Smith Theory of Moral Sentiments
Michael Eldred Social Ontology of Whoness.