A bias, in general, is a tendency or inclination in behavior or judgment. In this general sense, bias is fundamentally connected with discrimination and, as regards human beings, can be understood as arising from opinions, more or less conscious, that influence the way a subject behaves or judges in relation to individuals or groups of individuals.

Biases can be due to different factors: cognitive or psychological, cultural and social, or, more generally, contextual. But these are only some examples. The concept of bias is not limited to human behavior, in fact. In physics, the notion of bias is essentially connected with that of systematic error. That is, a measurement error that has the effect of altering results always in the same direction. A poorly calibrated scale, for example, might indicate an extra three hundred grams in every measurement. The connection with the general notion of bias is clear: this scale has a tendency to indicate a weight greater than the actual weight of what is being weighed.

Algorithmic bias, too—the notion of greatest relevance in the context of artificial intelligence—concerns systematic errors. We speak of algorithmic bias when an automated decision-making or classification system produces results that differ from those expected, are unfair, and tend to disadvantage (or favor) a particular class of individuals.

This kind of bias can arise for a great many reasons: programming errors or errors in the definition of the system, errors in data encoding, partiality in the collection or selection of training data, or improper use of data during training. Considering also the extensive use made of automated decision-making and classification systems, it is evident that this kind of bias can have disastrous social, economic, and personal effects on individuals belonging to classes negatively discriminated against by one or more of these systems. Moreover, the relationship between algorithmic bias and social discrimination is twofold. On the one hand, algorithmic bias can feed on pre-existing forms of discrimination. For example, if the data used to train an algorithm reflect discriminatory practices in the collection and selection of information, the resulting algorithm might express a bias consistent with these discriminatory practices. On the other hand, algorithmic bias itself can consolidate, or even aggravate, phenomena of social, cultural, geographical, and racial discrimination. Decision-making systems that penalize disadvantaged classes will certainly hinder any social processes of progress toward a more equitable society.

Although this notion of bias is clearly negative, we must not forget that the general concept of bias refers exclusively to the partiality of behavior or judgment. And this consideration is crucial. In fact, in artificial intelligence, decision-making, predictive, and classification systems are essentially founded on methods for managing bias. Some form of partiality must exist if the goal is to distinguish between individuals and, consequently, make decisions about them. There is a formal theorem that speaks precisely to this (the Ugly Duckling theorem), but no mathematical proof is needed to understand that every decision necessarily involves choosing to consider certain properties more relevant than others, certain characteristics as positive and others as negative. Obviously, this does not imply that all biases must be accepted as inevitable evils; it does imply, however, that the fight against algorithmic bias is an extremely delicate task that depends essentially on social and political decisions. Indeed, it is true that a decision-making system that is unable to distinguish one individual from another is a fair system, but it is also true that it is a completely useless system.

Appendix

For anyone interested, here is, in a few words, the Ugly Duckling theorem.

Given a set containing n objects, suppose we want to classify its elements according to their properties. If we have no preconceptions (or biases, precisely) about which properties are more natural, or simply more relevant to us at the moment, we can do nothing but consider all possible properties. Since a property can be defined by specifying a set of objects (the set of all objects that have that property), it follows that every subset of our domain will represent a property. For mathematical reasons that need not be made explicit here, there are 2ⁿ subsets of our set, and therefore we will have 2ⁿ properties to consider. If we use all these properties to establish similarities and differences among our objects, however, then any two objects will be as similar to each other as they are to every other object in the set. In fact, for any two objects, there will be exactly 2^n-1 properties that distinguish them and 2^n-1 that they have in common. That is, without bias, a swan will be as similar to the ugly duckling as it is to any other swan.

A certain amount of bias, then, is indispensable for structuring the world and making decisions that allow us at least to survive. This bias, however, must not be confused with bias in the sense of social discrimination. It is true that, among all possible classifications, we must choose some to consider relevant, but this does not imply that every classification that makes sense and can be useful for a certain purpose must be regarded as a division between better individuals and worse individuals.

Francesco A. Genco

(translated by MIRAI)