Belief Merging and Judgment Aggregation

Groups often need to reach decisions and decisions can be complex, involving the assessment of several related issues. For example, in a university a hiring committee typically decides on a candidate on the basis of her teaching and research qualities. A city council confronted with the decision of building a bridge, may ask its members to state whether they are favorable or not and, at the same time, to provide reasons for their position (like economical and environmental impacts, or expenditure considerations). Lastly, jurors are required to decide on the liability of a defendant by expressing their judgments on the conditions prescribed by the relevant code of law for the case at hand. As pointed out by Kornhauser and Sager (1986) referring to real jury trials, the aggregation of individual opinions on logically interrelated propositions can lead to a paradoxical result, the so-called doctrinal paradox. Inspired by the doctrinal paradox in jurisprudence, the problem of judgment aggregation attracted the interest of political scientists, philosophers, logicians, economists and computer scientists. Links to social choice theory have shown that, similar to the problem of preference aggregation (Arrow 1951/1963; Sen 1970), a judgment aggregation procedure that satisfies a number of desirable properties does not exist.

The question judgment aggregation addresses is how we can define aggregation procedures that preserve individual rationality at the collective level. From a philosophical point of view, such question concerns the nature of group attitudes such as group beliefs (Roth 2011). When the city council decides to build the bridge, the decision is taken on the basis of individual beliefs that, for example, the bridge will have a positive impact on the development of the economic activities in the area and does not represent an environmental threat. Thus, the formal approach to judgment aggregation can serve to cast light on the dependence between individual and collective beliefs (if there are any). The questions tackled by judgment aggregation are also relevant for the testimony problem investigated in social epistemology (Goldman 1999, 2004, 2010). How shall the diverging opinions of experts in a panel be combined and how should a rational agent respond to such disagreement?

The problem of combining potentially conflicting pieces of information does not arise only when a group of people needs to make a decision. Artificial intelligence also explores ways to aggregate conflicting sensors’ information, experts’ opinions or databases into a consistent one (Bloch et al. 2001). The combination of information coming from heterogeneous sensors improves the deficiencies of the individual sensors increasing the performances of a system. Examples are sensors of gesture recognition, screen rotation, and accelerometer in smartphones. Distributed databases may need to be accessed and managed at the same time to share data, for example a hospital may need to access the data collected about patients by different units. Internet users can find ratings on products provided by people who have purchased and assessed them on different online platforms. The type of information to be combined can differ, and so its representation can be numerical or symbolic: numbers, linguistic values, statistical probability distributions, binary preferences, utility functions etc. Yet all examples mentioned above deal with the problem of merging items coming from heterogeneous sources and with the issue of managing conflicts. At a purely formal level, belief merging studies the fusion of independent and equally reliable sources of information expressed in propositional logic. As with judgment aggregation, belief merging addresses the problem of fusing several individual bases expressed in propositional logic into a consistent one. Given the structural similarity of the problems investigated by these two disciplines, exploring their connections can reveal how similar these formalisms really are. On a more practical level, the application of operators defined in belief merging to judgment aggregation problems lead to the definition of a wider class of aggregation operators for judgment aggregation, the so-called distance-based procedures.

The focus of this entry is to draw explicit connections between judgment aggregation and the belief merging literature. Judgment aggregation will be briefly introduced in the next section. For a more comprehensive introduction to judgment aggregation the reader is referred to (List 2013; Grossi and Pigozzi 2014).

• 1. Judgment Aggregation
• 2. Belief Merging
  • 2.1 A framework for merging under integrity constraints
  • 2.2 The distance-based approach
• 3. Belief merging applied to the discursive dilemma
  • 3.1 Extensions and criticisms
• 4. Other topics
• Bibliography
• Academic Tools
• Other Internet Resources
• Related Entries

---

1. Judgment Aggregation

The formal work on judgment aggregation stemmed from the “doctrinal paradox” in the jurisprudence literature (Kornhauser and Sager 1986, 1993, 2004; Kornhauser 1992). The paradox shows that judges may face a real danger of falling into collective irrationality when trying to reach a common and justified verdict. Despite the recent birth of the discipline, structurally similar problems seemed to have been first pointed out by Poisson in 1837 (Elster 2013), and later noted by the Italian legal theorist Vacca in 1921 (Spector 2009).

In Kornhauser and Sager’s court example (1993), a three-member court has to reach a verdict in a breach of contract case between a plaintiff and a defendant. According to the contract law, the defendant is liable for breach of contract (proposition r) if and only if the contract forbid the defendant to do a certain action X (proposition p) and the defendant did action X (proposition q). Suppose that the three judges express the judgments as in Table 1.

Proposition r is the conclusion, whereas p and q are the premises. The legal doctrine can thus be logically expressed as \((p\land q)\leftrightarrow r\), stating that premises p and q are both necessary and sufficient for the conclusion r. Table 1 shows that each judge respects the given legal doctrine, by declaring the conclusion to be true if and only if she deems both premises true. If the judges aggregate their individual opinions using majority rule on the judgments on each proposition, the resulting judgment set is {p, q, not r}, which constitutes a violation of the legal doctrine. This is an instance of the doctrinal paradox: despite the individuals being logically consistent, the group’s judgment on the propositions is not consistent with the legal doctrine. In the example above, the judges cannot declare the defendant not liable and, at the same time, state that both conditions for her liability apply. Thus, the court faces a dilemma. Either judges are asked to express judgments on the premises only, and the court’s decision on r is logically derived from the majority on the premises (the premise-based or issue-by-issue procedure), or the verdict is decided by the majority judgment on r (the conclusion-based or case-by-case procedure) ignoring the opinions on the premises. Instances such as that in Table 1 illustrate that the two procedures may give opposite results.

This was not the first time that the definition of a collective outcome by majority rule resulted in a paradoxical result. Already in 1785, the Marquis de Condorcet discovered what is now known as the Condorcet paradox. Given a set of individual preferences, if we compare each of the alternatives in pairs and apply majority voting, we may obtain an intransitive preference (or cycle) in the collective outcome, of the type that alternative x is preferred to y, y is preferred to z, and z to x. The similarities between the Condorcet paradox and the judgment aggregation paradox were promptly noticed by Kornhauser and Sager (1986) and List and Pettit (2004). The study of the aggregation of individual preferences into a social preference ordering is the focus of social choice theory (List 2013). The Nobel Prize winner Kenneth Arrow proved the landmark result by showing that the problem on which Condorcet stumbled upon is more general and not limited to majority rule. Arrow’s impossibility theorem (Arrow 1951/1963; Morreau 2014) states that, given a finite set of individual preferences over three or more alternatives, there exists no aggregation function that satisfies a few plausible axioms. There are a number of results similar to Arrow's theorem that demonstrate the “impossibility” of judgment aggregation. The first impossibility theorem of judgment aggregation (List and Pettit 2002) was followed by further generalizations (Pauly and van Hees 2006; Dietrich 2006; Mongin 2008).

Let us restrict attention to the aggregation of judgments formulated in the language L of propositional logic (the problem of judgment aggregation can be generalized to modal and conditional logics as well as predicate logic, see Dietrich 2007). Dietrich and List (2007a) showed that, when the set of formulae on which the individuals express judgments (the agenda \(A\subseteq L\)) is sufficiently rich, the only judgment aggregation rules satisfying the desiderata below are dictatorships. An aggregation function is a dictatorship when, for any input, the collective outcome is taken to be the individual judgment of one (and the same) individual, i.e., the dictator. A judgment set is a consistent and complete set of formulae \(J \subseteq A\). A judgment set is complete if, for any element \(\varphi\) of the agenda, either \(\varphi\in A\) or \(\neg \varphi\in A\). Given a group of n individuals, a profile is a n-tuple of individual judgment sets \(\langle J_1, \ldots, J_n \rangle\). Finally, a judgment aggregation rule F is a function that assigns to each profile \(\langle J_1, \ldots, J_n\rangle\) a collective judgment set \(F(J_1, \ldots, J_{n}) \subseteq A\). The conditions imposed on F are the following:

Universal Domain: All profiles of consistent and complete (with respect to the agenda) judgment sets are accepted as input of the aggregation function.

Collective Rationality: Only complete and consistent collective judgments are acceptable as outputs.

Independence: The collective judgment on each proposition depends only on the individual judgments on that proposition, and not on other (considered to be independent) propositions in the agenda. (This condition reformulates in the judgment aggregation framework the independence of the irrelevant alternatives condition in Arrow’s theorem for preference aggregation.)

Unanimity Preservation: If all individuals submit the same judgment on a proposition p\(\in\) A, this is in the collective judgment set.

There is no judgment aggregation rule F that jointly satisfies the above conditions that is not a dictatorship. This impossibility result is particularly meaningful because, when reformulated for a preference framework, it can be shown that Arrow’s theorem (for strict preference orderings) is obtained as a corollary (Dietrich and List 2007a). This led Dietrich and List to say that judgment aggregation can be seen as a more general problem than preference aggregation (see Grossi and Pigozzi 2014 for details on such reformulation).

In addition to formal connections between the two types of aggregation problems, from a conceptual point of view judgment aggregation extends the problems of preference aggregation to more general decision problems. Although the models provided by social choice have improved our understanding of many familiar collective decision problems such as elections, referenda and legislative decisions, they focus primarily on collective choices between alternative outcomes such as candidates, policies or actions. They do not capture a whole class of decision problems in which a group has to form collectively endorsed beliefs or judgments on logically interconnected propositions. Yet, as the examples given in the introduction also show, such decision problems are common and not limited to court decisions. Pettit (2001) coined the term of discursive dilemma to highlight the fact that such problem can arise in all situations in which a group of individuals needs to reach a common stance on multiple propositions.

Impossibility results often bear a negative flavor. However, they also indicate possible escape routes. Consistent collective outcomes can be obtained when the universal domain condition is relaxed (considering, for example, unidimensionally aligned profiles (List 2002), a conditions similar to Black’s single-peakedness in preference aggregation (Black 1948)) or when the collective rationality condition is limited to require consistent (but not complete) collective judgments. Possibility results are also obtained when the independence condition is relaxed. The premise-based procedure seen in the court’s case is an example of an aggregation rule that violates independence. There, the collective position on the conclusion is derived by logical implication from the majority judgments on the premises. More in general, sequential priority rules violate independence and guarantee consistent group positions: the elements of the agenda are aggregated following a pre-fixed order, and earlier decisions constrain later ones. The reader is referred to (List and Puppe 2009; List 2013; Grossi and Pigozzi 2014) for thorough introductions to judgment aggregation and an overview on more impossibility theorems as well as on escapes routes from such results. In the next section we introduce the problem of combining conflicting information as it has been addressed in computer science. We will see that some operators introduced in belief merging are instances of aggregation procedures that violate the independence condition and that such operators can be applied to hold concrete aggregation procedures to judgment aggregation problems.

2. Belief Merging

Computer scientists have studied the aggregation of several independent and potentially conflicting sources of information into a consistent one. As mentioned in the introduction, examples are the combination of conflicting sensors’ information received by an agent, the aggregation of multiple databases to build an expert system, and multi-agent systems (Borgida and Imielinski 1984; Baral et al. 1992; Chawathe et al. 1994; Elmagarmid et al. 1999; Subrahmanian 1994; Kim 1995). Belief merging (or fusion) studies the aggregation of symbolic information (expressed in propositional logic) into a consistent base. As we shall see, the process of merging several bases has tight links with belief revision, a very active discipline since the 1980s across computer science and philosophy, that studies how new information can be included in an existing knowledge base, the interesting case being when the addition of the new input makes the set logically inconsistent (Alchourrón et al. 1985; Gärdenfors 1988), see Hansson 2011 for an overview. As for belief revision, in merging the term “knowledge” is used in a broader sense than in the epistemological literature, such that “knowledge” refers to formulas accepted by an agent (i.e., formulas in her knowledge base), which are not necessarily true. Then, “knowledge base” and “belief base” are used interchangeably. Grégoire and Konieczny (2006) go further and claim that belief merging operators can be used to aggregate other types of information than knowledge and beliefs, such as goals, observations, and norms.

The first approach to the problem of dealing with merging possibly inconsistent databases (Baral et al. 1991; Baral et al. 1992) built on Ginsberg’s idea of considering maximally consistent subsets when facing an inconsistent theory (Ginsberg 1986), such as the one that may result from the union of the information coming from several self-consistent (but conflicting with one another) agents. The idea is to define the combination of bases as the selection of some maximal consistent subsets of the union of the bases. The logical properties of such combination operators have been investigated in (Konieczny 2000) and compared to merging operators as defined in (Konieczny and Pino Pérez 1998, 1999). There are several differences between combining and merging knowledge bases. One difference is that the method by Baral et al. (1991, 1992) is syntax-dependent while merging operators obey the principle of irrelevance of syntax. Another difference is that when combination operators are used, the information about the source of the knowledge bases is ignored. This means that, unlike merging, combination operators cannot take into account cardinality considerations. Suppose, for example, that we have four knowledge bases: \(K_1 = K_{2}= \{a, b\}\), \(K_{3}= \{\neg a, d\}\) and \(K_4 = \{a, b\rightarrow c\}\). The union of these four bases is \(\{a,\neg a, b, b\rightarrow c, d\}\). The two maximal consistent subsets are: \(\{a, b,b\rightarrow c, d\}\) and \(\{\neg a, b, b\rightarrow c, d\}\). This means that we cannot decide whether to accept a or \(\neg a\). However, a majority of knowledge bases contained a, and only one base contained \(\neg a\). It seems intuitive that a should be in the resulting knowledge base as long as all knowledge bases are treated equally. If, for whatever reason, \(K_{3}\) is more trustworthy than the other knowledge bases, then we may prefer a combined base in which \(\neg a\) is accepted.

Arbitration is another operator to fuse knowledge bases that has been introduced in the early Nineties. By contrast to the (at that time) prominent area of belief revision, Revesz (1993) considered the case in which the new information does not take precedence over the old information. For example, if there are two equally reliable sources of information, we need to fuse the two bases rather than revise one by the other. Thus, he introduced an arbitration operator that (unlike revision operators) is commutative. Liberatore and Schaerf (1995) proposed axioms for arbitration between two belief bases, and the operator proposed by Revesz only satisfied some of them. Liberatore and Schaerf showed that, by slightly modifying Revesz’s definition and taking the conjunction of the two bases (when this is satisfiable) more axioms were satisfied. Liberatore and Schaerf’ proposal, however, suffered from the fact of being limited to the arbitration of only two bases.

None of the above methods could take into account the popularity of a specific information item. This meant that those operators could not capture the view of the majority. The first to introduce a majority postulate for the merging of several knowledge bases were Lin and Mendelzon (1999). The idea was inspired by the majority rule in social choice theory. However, their majority postulate includes a notion of partial support that captures the specificity of knowledge merging with respect to voting, and is not limited to count the number of bases supporting a proposition a vs. the number of bases containing \(\neg a\). A knowledge base was defined to partially support a literal l if there is a proposition a that contains no atoms appearing in l, such that the agent believes either l or a is true without knowing which one. A model-theoretic characterization of the postulates and specific merging operators are given in Lin and Mendelzon (1999). In the belief merging literature, sources of information are generally assumed to be equally reliable. One way to help to solve the conflict is to relax this assumption as, for example, in the extension to merging weighted knowledge bases given in (Lin 1996) or in prioritized knowledge bases (Benferhat et al. 1998; Cholvy 1998; Delgrande et al. 2006).

A new set of postulates for merging operators and the distinction (in terms of axioms they satisfy) between arbitration and majority operators were introduced by Konieczny and Pino Pérez (1998). In subsequent works (Konieczny and Pino Pérez 1999, 2002) they extended the framework to include merging under integrity constraints, that is, a set of exogenously imposed conditions that have to be satisfied by the merged base (Kowalski 1978; Reiter 1988). In the next section we present the formal framework introduced by Konieczny and Pino Pérez, which is now the standard framework for belief merging as it overcomes the limitations of the previous proposals.

The formal methods developed in belief merging have been exported and applied in areas of social epistemology, like elections and preference aggregation (Meyer et al. 2001), group consensus (Gauwin et al. 2005), and judgment aggregation (Pigozzi 2006) to which we return in Section 2.2.

2.1 A framework for merging under integrity constraints

Konieczny and Pino Pérez consider a propositional language L built up from a finite set At of atomic propositions and the usual connectives \((\neg, \land, \lor , \rightarrow, \leftrightarrow )\). An interpretation is a total function \(At \rightarrow \{0, 1\}\) that assigns 0 (false) or 1 (true) to each atomic proposition. For example, if \(At =\{p, q, r\}\), then \((1, 0, 1)\) is the interpretation that assigns true to p and r and false to q.^[1] Denote the set of all interpretations by \(W = \{0, 1\}^{At}\). For any formula \(\varphi \in L\), \(\mymod(\varphi) = \{\omega \in W | \omega \models \varphi\}\) denotes the set of models of \(\varphi\), i.e., the set of truth assignments that makes \(\varphi\) true. If we take the formula that expressed the contractual law in the doctrinal example, then \(\mymod((p\land q) \leftrightarrow r) = \{(1,1,1), (1, 0, 0), (0, 1, 0), (0,0,0)\}.\) As usual, a formula \(\varphi\) is consistent if it has at least a model, and a formula \(\varphi\) follows from a set of formulae \(\Phi\) if every interpretation that makes all formulae in \(\Phi\) true, makes also \(\varphi\) true.

A belief base \(K_i\) is a finite set of propositional formulae that will be identified with the conjunction of its elements, and represents the (full) beliefs held by agent i. Each \(K_{i}\) is assumed to be consistent. \(\mathcal{K}\) denotes the set of all consistent belief bases. The postulates for merging consider a multi-set of sets of formulas (belief profile, or belief set, the terminology used in early papers) \(E = \{K_1, \ldots , K_{n}\}\). The reason for using multi-sets is that an element can appear more than once, thus allowing the representation of the fact that two or more agents can hold the same beliefs. To mark the distinction with the usual set union \(\cup\), the multi-set union is denoted by \(\sqcup\) and defined as \(\{\varphi\} \sqcup \{\varphi\} = \{\varphi, \varphi\}\). Two belief profiles are equivalent \((E_1\equiv E_{2})\) if and only if there exists a bijection f from \(E_1\) to \(E_{2}\) such that, for any \(B\in E_1\), we have that \(\models\land f(B)\leftrightarrow \land B\).

Integrity constraints represent extra conditions that should follow from the fused bases. If the (possibly empty) set of integrity constraints is denoted by the belief base IC, \(\Delta_{\IC}(E)\) denotes the result of merging the multi-set E of belief bases given IC. Intuitively, the result will be a consistent belief base representing the collective beliefs and implies IC.

Konieczny and Pino Pérez (1999, 2002) put forward the following postulates for IC fusion operators between equally reliable sources. Let \(E\), \(E_1\), \(E_{2}\), be belief profiles, \(K_1\), \(K_{2}\) be consistent belief bases, and \(\IC\), \(\IC_1\), \(\IC_{2}\), be integrity constraints. \(\Delta\) is an IC fusion operator if and only if it satisfies the following rationality postulates:

• (IC0) \(\Delta_{\IC}(E) \models \IC\)
• (IC1) If \(\IC\) is consistent, then \(\Delta_{\IC}(E)\) is consistent.
• (IC2) If \(\land E\) is consistent with IC, then \(\Delta_{\IC}(E)\equiv \land E\land \IC\)
• (IC3) If \(E_1\equiv E_{2}\), and \(\IC_1\equiv \IC_{2}\), then \(\Delta_{\IC_1}(E_1)\equiv \Delta_{\IC_2}(E_{2})\)
• (IC4) If \(K_1 \models\IC\) and \(K_{2} \models\IC\), then \(\Delta_{\IC}(\{K_1, K_{2}\})\land K_1\) is consistent if and only if \(\Delta_{\IC}(\{K_1, K_{2}\})\land K_{2}\) is consistent.
• (IC5) \(\Delta_{\IC}(E_1) \land \Delta_{\IC}(E_{2}) \models\Delta_{\IC}(E_1\sqcup E_{2})\)
• (IC6) If \(\Delta_{\IC}(E_1) \land \Delta_{\IC}(E_{2})\) is consistent, then \(\Delta_{\IC}(E_1\sqcup E_{2}) \models \Delta_{\IC}(E_1) \land \Delta_{\IC}(E_{2})\)
• (IC7) \(\Delta_{\IC_{1}}(E) \land \IC_{2} \models\Delta_{\IC_{1\land} \IC_{2}} (E)\)
• (IC8) If \(\Delta_{\IC_1}(E) \land \IC_{2}\) is consistent, then \(\Delta_{\IC_{1\land}\IC_{2}} (E) \models\Delta_{\IC_1}(E)\)

In order to illustrate these postulates, we consider the following example, due to (Konieczny and Pino Pérez 1999). A group of flats co-owners wish to improve their condominium. At the meeting, the chairman proposes to build a tennis court, a swimming pool or a private parking. He also points out that building two of the three options will lead to a significant increase of the yearly maintenance expenses (this corresponds to the IC).

(IC0) ensures that the resulting merged base satisfies the integrity constraints. This is an obvious condition to impose since these are postulates for merging under integrity constraints, where the idea is to ensure that the result of the merging satisfies the integrity constraints. By employing a merging operator, the chairman knows that the group will agree on the increase of the expenses, if they decide to build at least two of the three facilities. (IC1) states that, when IC is consistent, then the result of the fusion operator will also be consistent. Again, given that the interpretations of the merged bases are selected among the interpretations of the integrity constraints, if IC is consistent, the result will also be consistent. (IC2) states that the result of the merging operator is simply the conjunction of the belief profile and the IC, whenever such conjunction is consistent. In our running example, if each person wishing to build two or more facilities endorses the rise of the expenses, and the opinions given by the co-owners are consistent, then the merging will just return the conjunction of the IC and the individual opinions. (IC3) states that if two belief profiles \(E_1\) and \(E_{2}\) are logically equivalent and \(\IC_1\) and \(\IC_{2}\) are also equivalent, then merging the first belief profile with \(\IC_1\) will be equivalent as merging the second belief profile with \(\IC_{2}\). This postulate expresses a principle already imposed on belief revision operators (of which, as we shall see, merging operators are extensions), that is, the principle of irrelevance of syntax, which says that the result of a merging operator depends only on the semantical content of the merged bases and not on their syntactical expression. (IC4) is known as the fairness postulate because it states that when merging two belief bases \(K_1\) and \(K_{2}\), no priority should be given to one of them. The merging is consistent with one of them if and only if it is consistent with the other. This postulate expresses a symmetric condition that operators that give priority to one of the two bases will not satisfy. (IC5) and (IC6) were first introduced in (Revesz 1997) and together they mean that if two groups agree on at least one item, then the result of the fusion will coincide with those items on which the two groups agreed on. So, if the group of co-owners can be split in two parties, such that one wants to build the tennis court and the swimming pool and the other wants the swimming pool and the parking, the building of the swimming pool will be selected as the final group decision. Finally, (IC7) and (IC8) guarantee that if the conjunction between the merging on \(E\) under \(\IC_1\) and \(\IC_{2}\) is consistent, then \(\IC_1\) will remain satisfied if \(E\) is merged under a more restrictive condition, that is, the conjunction of \(\IC_1\) and \(\IC_{2}\). This is a natural requirement to impose as, less formally, (IC7) and (IC8) together state that if the swimming pool is chosen among the set of three alternatives, it will still be selected if we reduce the set of alternatives to tennis court and swimming pool. The last two postulates are a generalization of two postulates for revision (R5 and R6) in (Katsuno and Mendelzon 1991), who characterized revision operators in terms of minimal change with respect to an ordering over interpretations. Like Katsuno and Mendelzon’s postulates, (IC7) and (IC8) ensure that the notion of closeness is well behaved, in the sense that if an outcome is selected by the merging operator under \(\IC_1\), then that outcome will also be the closest (i.e., it will be selected) to \(\IC_{2}\) within the more restrictive constraint \(\IC_1 \land \IC_{2}\) (assuming \(\Delta_{\IC_1}(E) \land \IC_{2}\) to be consistent).

Two sub-classes of \(\IC\) fusion operators are defined. An IC majority fusion operator minimizes the level of total dissatisfaction (as introduced by Lin and Mendelzon (1996), whereas an IC arbitration operator aims at equally distributing the level of individual dissatisfaction among the agents. The majority operator is similar in spirit to the utilitarian approach in social choice theory, whereas the arbitration is inspired to egalitarianism.

Let, for every integer \(n\), \(E^n\) denote the multi-set containing n times E. An IC majority operator satisfies the following additional postulate:

• (Maj) \(\exists n \Delta_{\IC_1} (E_1\sqcup E^{n}_{2}) \models\Delta_{\IC}(E_{2})\)

Thus, (Maj) states that enough repetitions of \(E_{2}\) will make \(E_{2}\) the opinion of the group. The number of repetitions needed depends on the specific instance.

An IC arbitration operator is characterized by the following postulate, in addition to (IC0)–(IC8):

• (Arb) Let IC1 and IC2 be logically independent. If \begin{align}\Delta_{\IC_1}(K_1) &\equiv \Delta_{\IC_2}(K_{2}), \textrm{ and}\\ \Delta_{\IC_{1} \leftrightarrow \neg \IC_{2}} (\{K_1, K_{2}\}) &\equiv (\IC_1\leftrightarrow \neg \IC_{2}), \textrm{ then}\\ \Delta_{\IC_{1}\lor \IC_{2}} (\{K_1, K_{2}\}) &\equiv \Delta_{\IC_1}(K_1).\end{align}

Intuitively, this axiom states that the arbitration operator selects the median outcomes that are IC-consistent. The behavior of such operator will be clearer when expressed in a model-theoretical way, as we shall see in the next section.

An example can help to appreciate the different behavior of a majority and an arbitration operator. Suppose three friends need to decide whether to buy a present for a common acquaintance. Suppose now that two of them want to buy her a book and invite her out for dinner, while the third friend does not want to contribute to either of those presents. If the group takes its decision by majority, the three friends would resolve to buy a book and to invite her out for dinner, making the third friend very unhappy. If, on the other hand, they use an arbitration operator, they would either buy her a book or invite her out to a restaurant, making the three members equally dissatisfied. Everyone has exactly one formula in their belief base that is not being satisfied, so the “amount” of dissatisfaction for each friend is the same.

The fusion operators in the literature can be divided in two classes: syntax-based fusion and model-based fusion. The first type takes the propositional formulas as the information input, and typically considers the maximally consistent subsets of the belief profile. In a model-based operator, on the other hand, it is the interpretations of the formulas that are considered as inputs to the merging process. Hence, each belief base is seen as a set of models and the syntactic representation is irrelevant. Since model-based operators have been applied to the problem of judgment aggregation, we will focus on that class of merging operators and refer to (Baral et al. 1992; Konieczny 2000; Grégoire and Konieczny 2006) for more on syntax-based fusion.

2.2 The distance-based approach

An IC model-based fusion operator selects, among the models of IC, those that are preferred, where the preference relation depends on the operator that is used. The collective belief set \(\Delta_{\IC}(E)\) is the set of formulas that are true in all of the selected models. The preference information usually takes the form of a total pre-order (to recall, a pre-order is a reflexive and transitive relation) \(\le\) on the interpretations induced by a notion of distance d between an interpretation \(\omega\) and the profile E, denoted by \(d(\omega,E)\). It should be noted that a distance-based fusion operator does not always guarantee a unique result. We will come back to this point when we look at the application of belief merging to judgment aggregation.

We have seen that majority operators are characterized by trying to minimize the total dissatisfaction, whereas the arbitration operators aim at minimizing the local dissatisfaction. We can thus see the distance as a way to capture the notion of dissatisfaction. Inspired by the economy principle employed in belief revision, the outcome in merging should keep as much as information as possible from each individual belief base \(K_i\). In other words, since the sources of information are assumed to be equally reliable, the merge should delete as little as possible from the sources. The idea then is to select the interpretations that minimize the distance between the models of IC and the models of the belief profile E. Formally, this can be expressed as follows:

\[ \mymod(\Delta_{\IC}(E)) = \mymin (\mymod (\IC), \le_{d}) \]

A distance d between interpretations is a total function \(d: W \times W \rightarrow R^{+}\) such that for all \(\omega, \omega'\in W\):

1. \(d(\omega ,\omega') = d(\omega',\omega )\)
2. \(d(\omega ,\omega') = 0 \textrm{ iff } \omega =\omega'\).

So, the distance is symmetric and, if two interpretations are identical, the distance is 0.^[2]

Two steps are needed to find the models of IC that minimize the distances to the belief profile. In the first step, we calculate the distance between each interpretation satisfying IC (that is, each candidate merged base) and each individual belief base. In the second step, we need to aggregate all those individual distances to define the collective distance, that is, the distance of the belief profile to each model of IC. Finally, the (possibly more than one) base that minimizes such distance is selected as outcome.

For the first step, we need to define the distance between an interpretation \(\omega\) and a belief base K. This is the minimal distance between \(\omega\) and the models of K. Formally: \(d(\omega, K) = \mymin_{\omega'}\in_{\mymod(K)} d(\omega, \omega')\). If K has more than one model, \(\omega'\) will be the closest to \(\omega\).

We can now define the distance between an interpretation \(\omega\) and a belief profile E, which is needed for the second step. We need an aggregation function \(D: R^{+n} \rightarrow R^{+}\) that takes the distances between the models of IC and the belief bases \(K_i\) calculated in the first step, and aggregate them into a collective distance. This is: \(D(\omega, E) = D(d(\omega, K_1)\), \(d(\omega, K_2)\), \(\ldots\), \(d(\omega, K_n))\). A total pre-order over the set W of all interpretations is thus obtained. The merging operator can now select all interpretations that minimize the distance to the profile E.

Technically, an aggregation function \(D: R^{+n} \rightarrow R^{+}\) assigns a nonnegative real number to every finite tuple of nonnegative real numbers. For any \(x_1,\ldots, x_{n}, x, y \in R^{+}, D\) satisfies the following properties:

1. if \(x\ge y\), then \(D(x_1,\ldots , x,\ldots , x_{n})\ge D(x_1,\ldots , y,\ldots , x_{n})\)
2. \(D(x_1,\ldots , x,\ldots , x_{n})=0\) if and only if \(x_1=\ldots =x_{n}=0\)
3. \(D(x)=x\)

The outcome of the merging operator clearly depends on the chosen distance functions d and D. Among the first proposals (Lin and Mendelzon 1999; Revesz 1993) it was to adopt the Hamming distance (defined below) for d and the sum or the max for D (denoted respectively \(D_{\Sigma}\) and \(D_{\mymax}\)).^[3] When D is the sum, the global distance is obtained by summing the individual ones. The corresponding merging operator is a majority operator and is called minisum as it will select those interpretations that minimize the sum. The merging operator that uses \(D_{\mymax}\) is known as minimax and outputs the judgment set that minimizes the maximal distance to the individual bases (Brams et al. 2007b). Intuitively, minimax aims at minimizing the disagreement with the most dissatisfied individual. Two opposite outcomes may be selected when \(D_{\Sigma}\) or \(D_{\mymax}\) is used (Brams et al. 2007b; Eckert and Klamler 2007).

The Hamming distance was a commonly used distance in belief revision. The idea is simple. The Hamming distance counts the number of propositional letters on which two interpretations differ. So, for example, if \(\omega = (1, 0, 0)\) and \(\omega' = (0, 1, 0)\), we have that \(d(\omega, \omega') = 2\) as the two interpretations differ on the assignment to the first and the second propositions. Also well known is the drastic distance, which assigns distance 0 if two interpretations are the same and 1 otherwise. But the choice of the distance is not restricted to those options. Other distances can be used, that still satisfy the postulates given above (Konieczny and Pino Pérez 1999, 2002). The minimization of the sum of the individual distances is an example of an IC majority merging operator. In the next section, we will see this operator applied to the discursive dilemma.

The distance-based approach can clarify the distinction between arbitration and majority operators. Leximax is an example of arbitration operator. A leximax operator may take d as the Hamming distance and, for each interpretation, the distances between that interpretation and the n bases \(K_i\) form a list. A pre-order over interpretations is defined by taking the lexicographical order between sequences of distances, fixing an order over the set of agents. Finally, \(D_{\textit{leximax}}\) selects the minimum. The intuition is that, unlike a majority operator that selects the option that minimizes the total disagreement (by minimizing the sum of the individual distances, for example), an arbitration operator looks at the distribution of such disagreement and selects the option that is fairer to all individuals, that is, it aims at equally distributing the individual dissatisfaction with the chosen outcome (recall the birthday gift example above). This follows from the definition of the Hamming distance: the larger the Hamming distance, the more disagreement there is between two worlds (here disagreement simply means that the worlds assign different truth values to the same formula). Suppose that a belief profile E has three bases. Suppose as well that the distances from the two models of IC (\(\omega\) and \(\omega'\)) are \(D_{\Sigma} (\omega, E) = D_{\Sigma} (\omega', E) = 6\) when we take the sum of the Hamming distances, and \(D_{\textit{leximax}}(\omega, E)= (2,2,2)\) and \(D_{\textit{leximax}}(\omega', E)= (5,1,0)\) when we take the lexicographic order on the distances. In this example, the majority operator cannot distinguish between \(\omega\) and \(\omega'\) while the arbitration operator will prefer \(\omega\) to \(\omega'\) as \(\omega\) distributes the individual disagreement in a fairer way than \(\omega'\).

As mentioned already, Liberatore and Schaerf were among the first to propose arbitration operators. However, their approach was limited to only two bases, and the result of the merge was one of the two bases. Such operator would give questionable results in some situations, like the one in (Konieczny and Pino Pérez 2002). Suppose that two financial experts give you advice regarding four shares a, b, c and d. According to the first expert, all four shares are going to rise (denoted by \(\varphi_1= \{(1, 1, 1, 1)\}\), whereas the second expert deems that all four shares will fall \((\varphi_{2}= \{(0, 0, 0, 0)\})\). According to Liberatore and Schaerf’s arbitration operator, the result will be \(\{(1, 1, 1, 1), (0, 0, 0, 0)\},\) which means that either the first or the second expert is totally right. If, on the other hand, we apply an arbitration operator à la Konieczny and Pino Pérez, we obtain \(\{(0, 0, 1, 1)\), \((0, 1, 0, 1)\), \((0, 1, 1, 0)\), \((1, 0, 0, 1)\), \((1, 0, 1, 0)\), \((1, 1, 0, 0)\}\). This result can be interpreted as that—if we assume that all sources are equally reliable—we do not have any reason to prefer one or another and, so a reasonable position is to conclude that both can be equally right. Still, Liberatore and Schaerf’s operator may be used in all situations where the result can be only one of the bases submitted by the individuals. For example, if two doctors meet in order to decide a patient’s therapy, they likely have to decide in favour of one of the two proposals as mixing therapies may not be a feasible nor a safe option.

A representation theorem (Konieczny and Pino Pérez 1999, 2002) ensures that to each sub-class of IC merging operators (majority and arbitration operators) corresponds a family of pre-orders on interpretations (mirroring a similar representation theorem that Katsuno and Mendelzon (1991) proved for belief revision operators).^[4]

Let us now illustrate how belief merging can be applied to judgment aggregation problems.

3. Belief merging applied to the discursive dilemma

The merging of individual belief bases into a collective one shares similarities to the judgment aggregation problem. In both cases we wish to aggregate individual inputs into a group outcome, and both disciplines employ logic to formalize the bases’ contents. As we have seen in Section 1, no aggregation procedure can ensure a consistent and complete group judgment. However, the merging operators introduced in computer science ensure a consistent outcome because such operators do not satisfy independence. The collective judgment on a proposition is not only determined by the individual judgments on that proposition, but also by considerations of all other agenda’s items. It is natural to apply the results about merging methods to the aggregation of individual judgments (Pigozzi 2006). A welcomed side-effect of this work is that it enlarged the class of concrete aggregation procedures for judgment aggregation, until then limited mainly to premise and conclusion-based procedures.

How do impossibility results in judgment aggregation conciliate with the fact that IC merging operators can ensure a consistent collective outcome? The reason is twofold: on the one hand, paradoxical results are avoided by resorting to IC, which blocks unacceptable outcomes. On the other hand, merging operators violate the independence conditions, one of the requirements imposed on aggregation functions in the impossibility theorems. Independence turned out to be an instrumentally attractive condition because it protects an aggregation function from strategic manipulation (Dietrich 2006; Dietrich and List 2007b). This means that an individual has no interest in submitting an insincere judgment set in order to get a better outcome for her. However, independence has been criticized in the literature as a not suitable desideratum to aggregate propositions that are logically interconnected (Chapman 2002; Mongin 2008).

It has been observed (Brams et al. 2007a) that majority voting minimizes the sum of Hamming distances. This means that, whenever proposition-wise majority voting selects a consistent judgment set, the same outcome is selected by the minisum rule. Majority voting has credentials for being democratic. Another reason to focus on majority distance-based procedures is that the aim of the aggregation of individual judgments should be the right decision rather than a fair distribution of individual dissatisfactions. The epistemic link between majority voting and right decisions has been pointed out in the Condorcet Jury Theorem. The theorem shows that, when the voters are independent and have an equal probability of being right better than random, then majority rule ensures to select the right decision and the probability for doing so approaches 1 as the voters’ group size increases (see List 2013 for this and two more formal arguments for majority rule).

Let us consider the three judges example and see what we obtain when applying the minisum rule. The legal doctrine corresponds to \(\IC= \{(p\land q) \leftrightarrow r\}\). The court is represented by the profile \(E=\{K_1 , K_{2} , K_{3}\}\), which is the multi-set containing the judgment sets \(K_1 , K_{2} , K_3\) of the three judges. The three judgment sets and their corresponding models are:

\begin{align} K_1 &= \{p, q, r\} \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \mymod(K_1) &= \{(1, 1, 1)\};\\ K_{2} &= \{p, \neg q, \neg r\} \ \ \ \ \ \ \ \ \ \ \ \ \mymod(K_{2}) &= \{(1, 0, 0)\};\\ K_{3} &= \{\neg p, q, \neg r\} \ \ \ \ \ \ \ \ \ \ \ \ \mymod(K_{3}) &= \{(0, 1, 0)\}.\\ \end{align}

Table 2 shows the result for the majority operator that minimizes the sum of the Hamming distances. In the first column are all the interpretations for the propositional variables p, q, and r. The interpretations that are not models of the IC have a shaded background. So, for example, \((1,0,1)\) cannot be selected as the collective outcome because it violates the legal doctrine. The numbers in the \(d_H (\cdot,K_1)\), \(d_H(\cdot,K_{2})\), and \(d_H(\cdot,K_{3})\) columns are the Hamming distances of each \(K_i\) from the corresponding interpretation. In the last column are the sums of the Hamming distances.

We see that, without IC, the distance-based majority operator would select the same (inconsistent) outcome as proposition-wise majority voting, that is, \((1,1,0)\). This is the outcome that is at the minimum distance from E. However, the merging operator cannot select \((1,1,0)\) as that outcome violates the IC. Neglecting the shaded rows, only four interpretations are candidates to be selected as collective judgment sets, that is, \((1,1,1)\), \((1,0,0)\), \((0,1,0)\), and \((0,0,0)\). Of those, three are the ones that minimize the distances. Thus, collective inconsistency is avoided when a distance-based aggregation is used. However, this method does not always guarantee a unique outcome. In the court example, this aggregation selects the three models \((1,1,1)\), \((1,0,0)\) and \((0,1,0)\) as group positions. Technically, this is said to be an irresolute procedure, and a tie-breaking rule needs to be combined if we wish to ensure a unique result (as common in social choice theory).

The applicability of merging techniques developed in computer science to judgment aggregation problems does not mean that the two disciplines have the same objectives. As we have seen, the original motivation of belief merging was to define ways to aggregate information coming from different sources. Since the sources can have different access to the information, no externally given agenda is assumed. This is a difference with the judgment aggregation framework, where individuals are required to submit their opinion on a given set of items. Belief merging and judgment aggregation do not only differ in the type of inputs they aggregate. They also output different results: a collective base satisfying some given integrity constraints for belief merging, and a collective judgment on the given agenda for judgment aggregation.

Another difference resides in the fact that judgment aggregation assumes that all members are rational, and so they all submit consistent judgment sets. In belief merging, this is not required. Agents can submit belief bases that are inconsistent with the IC (Grégoire 2004). If an individual submits a judgment that violates an integrity constraint, that judgment set will not figure among the candidates to represent the group position. However, his input will not be disregarded and will be taken into account in the merging process. The possibility to abstain from expressing an opinion on a certain item is also easily taken into account in a belief merging setting. If individuals need to have their say on p, q and r and one agent believes q and r to be true but does not have a clear opinion on p, this will be represented as \(\mymod(K_1)= \{(1, 1, 1),\) \((0, 1, 1)\}\) and the distances calculated accordingly. The completeness requirement on judgment sets has also been weakened in the judgment aggregation framework (Gärdenfors 2006; Dokow and Holzman 2010). When judgment sets are not assumed to be complete, any independent and unanimous aggregation function turns out to be weakly oligarchic, that is, a subset of the individuals will decide the collective outcome. Intuitively, this is a less negative result than dictatorship, though it reduces to dictatorship in the case in which only one individual belongs to that subset.

A first formal investigation of the relationships between belief merging and judgment aggregation can be found in (Everaere et al. 2014). As we have seen, belief merging takes a profile of propositional belief bases as input, where such bases represent the beliefs of a certain individual, not restraint to a given agenda. Judgment aggregation, on the other hand, asks people to submit their judgments on a specific set of issues. Thus, assuming that an individual’s beliefs allow deriving her opinion on the agenda’s items, a projection function p (assumed to be identical for all the individuals) is defined. The role of such projection function is precisely to determine the judgments of an agent starting from her beliefs. So, for example, if an individual believes \(p\land q\) and one of the agenda items is p, then the projection function can derive that the person submits a yes as judgment on p. To note is that the results obtained in (Everaere et al. 2014) are relative to the full information case (as they call it), that is, to the case in which the agenda contains all possible interpretations, making the two approaches equally informed. Using such projection, two paths along which a collective judgment can be derived from a profile of belief bases are then considered. Along one path (which we may call the merge-then-project path), the individual belief bases are first merged using a merging operator and then the collective judgment is computed by the projection p. Along the other path (which we may call project-then-aggregate), starting from the individual bases, the individual judgment sets are first computed by p and then aggregated using a judgment aggregation procedure to determine the collective judgment on the given agenda. Thus, the question addressed is whether the two collective judgments obtained by following the two paths coincide. The answer is positive when the standard IC merging postulates are reinterpreted as suitable properties for judgment aggregation rules.

3.1 Extensions and criticisms

The minisum rule applied to merge the judgment sets of the three judges in the previous section is based on the same principles as the Kemeny rule, a well-known preference aggregation rule (Kemeny 1959). Unlike what happened in social choice, at the beginning the literature of judgment aggregation focused on the axiomatic method and only few concrete aggregation rules were proposed and studied. Arguably, the interest of researchers from computer science and multi-agent systems for judgment aggregation lead to the definition of more concrete aggregation rules and to the investigation of their relations. The same idea of minimization that plays such a crucial role in belief merging can be found as a principle in the definition of several voting rules in social choice theory. For instance, the minisum rule turned out to be equivalent to several other rules recently introduced in the judgment aggregation literature (Lang et al. 2015, in Other Internet Resources).

The interest of computer scientists for aggregation methods is witnessed by the fact that judgment aggregation is now among the topics of computational social choice, an interdisciplinary discipline that promotes exchanges and interactions between computer science and social choice theory. Computational issues of aggregation rules are among the interests of computational social choice. The computational complexity of the distance-based procedures has been studied (Endriss et al. 2012), and a new rule has been proposed to overcome the high computational complexity of distance-based procedures. The average-voter rule (Grandi 2012) selects the judgment set submitted by the individuals that minimizes the sum of the distances. Hence, the outcome has to be one of the submitted judgment sets. This allows reducing the computational complexity and, at the same time, selects the most representative individual.

A generalization of distance-based methods for judgment aggregation has been given in (Miller and Osherson 2009). Besides generalizing (by taking a general metric) the merging operator we have applied to the doctrinal paradox, they proposed three other distance-based procedures for judgment aggregation. In case proposition-wise majority collapses into an inconsistent collective judgment set, one method (Endpoint) selects as group outcome the closest (according to some distance metric) consistent collective judgment set. The other two methods (Full and Output) look at minimal ways to change the profile in order to output a consistent proposition-wise majority collective judgment set. The difference is that Output allows the individual judgment sets in the modified profile to be inconsistent.

Duddy and Piggins (2012) questioned the use of Hamming distance between judgment sets. The problem is that, when the agenda contains propositions that are logically connected, the Hamming distance may be responsible of double counting because it ignores such interdependencies. Suppose, for example, that two individuals accept propositions q but disagree on \(p\land q\) (so, one individual accepts the conjunction, while the other rejects it). This can happen only if they disagree on p. The Hamming distance between the two judgment sets \(K_1 = \{\neg p, q, \neg(p\land q)\}\) and \(K_{2} = \{p, q, (p\land q)\}\) is 2. It is the disagreement on p that implies the disagreement over \(p\land q\), so the distance should be just 1. The alternative distance proposed in order to address this problem is a distance that takes the smallest number of logically coherent changes needed to convert one judgment set into the other.

4. Other topics

Belief merging is an abstract theory that addresses the problem of aggregating symbolic inputs, without specifying whether such items are beliefs, knowledge, desires, norms etc. It is the choice of the merging operator that should best suit the type of inputs. The framework of judgment aggregation has also been extended to include the aggregation of other types of attitudes, as in (Dietrich and List 2010).

The literature on belief merging includes the study of the strategic manipulation problem (Evaraere et al. 2007). When an aggregation procedure is not strategy-proof, an individual who has a preference over the possible outcomes, can manipulate the result by lying on her true beliefs and thus obtain an outcome closer to her true preferences. In general, merging operators are not strategy-proof when Hamming distance is used, whereas they are strategy-proof when the drastic distance is employed.

In those situations in which we can assume that there is a fact of the matter (for example, a defendant has—or has not—committed a murder), which each agent has a (noisy) opinion about, the truth-tracking properties of belief merging operators can be investigated (Hartmann et al. 2010; Hartmann and Sprenger 2012; Cevolani 2014). The question is then whether a certain aggregation method selects the right decision. Williamson (2009) argues that aggregating the evidence on which the judgments are based is best for judgment aggregation, as it would yield to the right decision. The three-step proposal he advocates distinguishes between three types of propositions: evidence, beliefs and judgments. Evidence is the support for an agent’s beliefs and judgments, and it is the right candidate for merging techniques. Judgments, on the other hand, are best dealt with decision theory that maps degrees of beliefs and utilities to judgments.

Bibliography

• Alchourrón, C.E., P. Gärdenfors, and D. Makinson, 1985, “On the Logic of Theory Change: Partial Meet Functions for Contraction and Revision”, Journal of Symbolic Logic, 50: 510–530.
• Arrow, K., 1951/1963, Social Choice and Individual Values, New York: Wiley.
• Baral, C., S. Kraus, and J. Minker, 1991, “Combining Multiple Knowledge Bases”, IEEE Transactions on Knowledge and Data Engineering, 3(2): 208–220.
• Baral, C., S. Kraus, J. Minker, and V. Subrahmanian, 1992, “Combining Multiple Knowledge Bases Consisting of First Order Theories”, Computational Intelligence, 8(1): 45–71.
• Benferhat, S., D. Dubois, J. Lang, H. Prade, A. Saffiotti, and P. Smets, 1998, “A General Approach for Inconsistency Handling and Merging Information in Prioritized Knowledge Bases”, in Proceedings of the Sixth International Conference on Principles of Knowledge Representation and Reasoning (KR’98), pages 466–477.
• Black, D., 1948, “On the Rationale of Group Decision Making”, The Journal of Political Economy, 56: 23–34.
• Bloch, I., A. Hunter, A. Ayoun, S. Benferhat, P. Besnard, L. Cholvy, R. Cooke, D. Dubois and H. Fargier, 2001, “Fusion: general concepts and characteristics”, International Journal of Intelligent Systems, 16: 1107–1134.
• Borgida, A. and T. Imielinski, 1984, “Decision Making in Committees: A Framework for Dealing with Inconsistency and Non-Monotonicity”, in Proceedings Workshop on Nonmonotonic Reasoning, pp. 21–32.
• Brams, S.J., D.M. Kilgour, and M.R. Sanver, 2007a, “A Minimax Procedure for Negotiating Multilateral Treaties”, in R. Avenhaus and I. W. Zartman (eds.), Diplomacy Games: Formal Models and International Negotiations, Berlin: Springer, pp 265-282.
• –––, 2007b, “A Minimax Procedure for Electing Committees”, Public Choice, 132: 401–420.
• Cevolani, G., 2014, “Truth Approximation, Belief Merging and Peer Disagreement”, Synthese, 191(11): 2383–2401.
• Chapman, B., 2002, “Rational Aggregation”, Politics, Philosophy, Economics, 1(3): 337–354.
• Chawathe, S., H. Garcia Molina, J. Hammer, K. Ireland, Y. Papakonstantinou, J. Ullman, and J. Widom, 1994, “The TSIMMIS Project: Integration of Heterogeneous Information Sources”, in Proceedings of IPSJ Conference, pp. 7–18.
• Cholvy, L., 1998, “Reasoning about Merged Information”, in D.M. Gabbary, and Ph. Smets (eds.), Handbook of Defeasible Reasoning and Uncertainty Management Systems, Vol. 3, Dordrecht: Kluwer Academic Publishers, pages 233–263.
• Condorcet, N. de, 1785, Essai sur l’Application de l’Analyse à la Probabilité des Décisions Rendues à la Pluralité des Voix, Paris.
• Delgrande, J.P., D. Dubois, and J. Lang, 2006, “Iterated Revision as Prioritized Merging”, in Proceedings of the 10^th International Conference on Knowledge Representation and Reasoning (KR’06), pp. 210–220.
• Dietrich, F., 2006, “Judgment Aggregation: (Im)Possibility Theorems”, Journal of Economic Theory, 126: 286–298.
• –––, 2007, “A generalised model of judgment aggregation”, Social Choice and Welfare, 28(4): 529–565.
• Dietrich, F. and C. List, 2007a, “Arrow’s Theorem in Judgment Aggregation”, Social Choice and Welfare, 29: 19–33.
• –––, 2007b, “Strategy-proof Judgment Aggregation”, Economics and Philosophy, 23: 269–300.
• –––, 2010, “The aggregation of propositional attitudes: Towards a general theory”, Oxford Studies in Epistemology, vol. 3, Oxford: Oxford University Press, pp. 215–234.
• Dokow, E. and R. Holzman, 2010, “Aggregation of Binary Evaluations with Abstentions”, Journal of Economic Theory, 145(2): 544–561.
• Duddy, C. and A. Piggins, 2012, “A Measure of Distance Between Judgment Sets”, Social Choice and Welfare, 39: 855–867.
• Eckert, D. and C. Klamler, 2007, “How puzzling is judgment aggregation? Antipodality in distance-based aggregation rules”, Working paper, University of Graz.
• Elmagarmid, A., M. Rusinliewicz, and A. Sheth (eds.), 1999, Management of Heterogeneous and Autonomous Database Systems, San Francisco, CA, USA: Morgan Kaufmann.
• Elster, J., 2013, “Excessive Ambitions (II)”, Capitalism and Society, 8(1): Article 1 [Elster 2013 available online]
• Endriss, U., U. Grandi, and D. Porello, 2012, “Complexity of Judgment Aggregation”, Journal of Artificial Intelligence Research, 45: 481–514.
• Evaraere, P., S. Konieczny, and P. Marquis, 2007, “The Strategy-Proofness Landscape of Merging”, Journal of Artificial Intelligence Research (JAIR), 28: 49–105.
• –––, 2014, “Propositional Merging and Judgment Aggregation: Two Compatible Approaches?”, in Proceedings of the 21^st European Conference on Artificial Intelligence (ECAI 14), pp. 1001–1002.
• Gärdenfors, P., 1988, Knowledge in Flux: Modeling the Dynamics of Epistemic States, Cambridge: MIT Press.
• –––, 2006, “A Representation Theorem for Voting with Logical Consequences”, Economics and Philosophy, 22: 18–190.
• Gauwin, O., S. Konieczny, and P. Marquis, 2005, “Conciliation and Consensus in Iterated Belief Merging”, in Proceedings of the 8th European Conference on Symbolic and Quantitative Approaches to Reasoning with Uncertainty, Berlin: Springer, pp. 514–526.
• Ginsberg, M., 1986, “Counterfactuals”, Artificial Intelligence, 30(1): 35–79.
• Goldman, A., 1999, Knowledge in a Social World, Oxford: Oxford University Press.
• –––, 2004, “Group Knowledge versus Group Rationality: Two Approaches to Social Epistemology”, Episteme, A Journal of Social Epistemology, 1(1): 11–22.
• –––, 2010, “Social Epistemology”, The Stanford Encyclopedia of Philosophy (Summer 2010 Edition), Edward N. Zalta (ed.), URL = <https://plato.stanford.edu/archives/sum2010/entries/epistemology-social/>.
• Grégoire, E., 2004, “Extension of a Distance-based Fusion Framework”, in Proceedings of the 8th International Conference on Sensor Fusion: Architectures, Algorithms and Applications, pp. 282–286.
• Grégoire, E. and S. Konieczny, 2006, “Logic-based Approaches to Information Fusion”, Information Fusion, 7: 4–18.
• Grandi, U., 2012, Binary Aggregation with Integrity Constraints, Ph.D. thesis, ILLC, University of Amsterdam [Grandi 2012 available online].
• Grossi, D. and G. Pigozzi, 2014, Judgment Aggregation: A Primer, San Rafael, CA: Morgan & Claypool.
• Hansson, S.O., 2011, “Logic of Belief Revision”, The Stanford Encyclopedia of Philosophy (Fall 2011 Edition), Edward N. Zalta (ed.), URL = <https://plato.stanford.edu/archives/fall2011/entries/logic-belief-revision/>.
• Hartmann, S., G. Pigozzi, and J. Sprenger, 2010, “Reliable Methods of Judgment Aggregation”, Journal of Logic and Computation, 20(2): 603–617.
• Hartmann, S. and J. Sprenger, 2012, “Judgment Aggregation and the Problem of Tracking the Truth”, Synthese, 187(1): 209–221.
• Katsuno, H. and A.O. Mendelzon, 1991, “Propositional Knowledge Base Revision and Minimal Change”, Artificial Intelligence, 52: 263–294.
• Kemeny, J., 1959, “Mathematics without numbers”, Daedalus, 88: 577–591.
• Kim, W. (ed.), 1995, Modern Database Systems: The Object Model, Interoperability and Beyond, New York, NY, USA: Addison Wesley.
• Konieczny, S., 2000, “On the Difference between Merging Knowledge Bases and Combining them”, in A.G. Cohn, F. Giunchiglia, and B. Selman, (eds.), KR2000: Principles of Knowledge Representation and Reasoning, San Francisco: Morgan Kaufmann, pp. 135–144.
• Konieczny, S. and R. Pino Pérez, 1998, “On the Logic of Merging”, in Proceedings of the 6th International Conference on Principles of Knowledge Representation and Reasoning, San Francisco: Morgan Kaufmann, pp. 488–498.
• –––, 1999, “Merging with Integrity Constraints”, in Proceedings of the 5th European Conference on Symbolic and Quantitative Approaches to Reasoning with Uncertainty (ECSQARU ’99), LNAI 1638, pp. 233–244.
• –––, 2002, “Merging Information under Constraints: A Logical Framework”, Journal of Logic and Computation, 12: 773–808.
• Kornhauser, L.A., 1992, “Modeling Collegial Courts. II. Legal Doctrine”, Journal of Law, Economics, and Organization, 8: 441–470.
• Kornhauser, L.A. and L.G. Sager, 1986, “Unpacking the Court”, Yale Law Journal, 96: 82–117.
• –––, 1993, “The One and the Many: Adjudication in Collegial Courts”, California Law Review, 81: 1–51.
• –––, 2004, “The Many as One: Integrity and Group Choice in Paradoxical Cases”, Philosophy & Public Affairs, 32(3): 249–276.
• Kowalski, R., 1978, “Logic for data description”, in H.G.J. Minker (ed.), Logic and data bases, New York: Plenum, pages 77–102.
• Liberatore, P. and M. Schaerf, 1995, “Arbitration: a Commutative Operator for Belief Revision”, in Proceedings of the Second World Conference on the Fundamentals of Artificial Intelligence, pp. 217–228.
• –––, 1998, “Arbitration (or How to Merge Knowledge Bases)”, IEEE Transactions on Knowledge and Data Engineering, 10(1): 76–90.
• Lin, J., 1995, Frameworks for Dealing with Conflicting Information and Applications, Ph.D. thesis, University of Toronto.
• –––, 1996, “Integration of Weighted Knowledge Bases”, Artificial Intelligence, 83: 363–378.
• Lin, J. and A. Mendelzon, 1996, “Merging databases under constraints”, International Journal of Cooperative Information Systems, 7: 55–76.
• –––, 1999, “Knowledge Base Merging by Majority”, in Dynamic Worlds: From the Frame Problem to Knowledge Management, R. Pareschi and B. Fronhöfer (eds), Norwell, MA, USA: Kluwer.
• List, C., 2002, “A Possibility Theorem on Aggregation over Multiple Interconnected Propositions”, Mathematical Social Sciences, 45(1): 1–13.
• –––, 2013, “Social Choice Theory”, The Stanford Encyclopedia of Philosophy (Winter 2013 Edition), Edward N. Zalta (ed.), URL = <https://plato.stanford.edu/archives/win2013/entries/social-choice/>.
• List, C. and P. Pettit, 2002, “Aggregating sets of judgments: an impossibility result”, Economics and Philosophy, 18: 89–110.
• –––, 2004, “Aggregating Sets of Judgments: Two Impossibility Results Compared”, Synthese, 140: 207–235.
• List, C. and C. Puppe, 2009, “Judgment Aggregation: A Survey”, in P. Anand, C. Puppe, and P. Pattanaik (eds.), The Handbook of Rational and Social Choice, Oxford: Oxford University Press.
• Meyer, T., A. Ghose, and S. Chopra, 2001, “Social Choice, Merging and Elections”, in Proceedings of the 6th European Conference on Symbolic and Quantitative Approaches to Reasoning with Uncertainty, Benferhat and Besnard (eds.), Vol. 2143 of Lecture Notes in Artificial Intelligence, Berlin: Springer, pp. 466–477.
• Miller, M.K. and D. Osherson, 2009, “Methods for Distance-based Judgment Aggregation”, Social Choice and Welfare, 32(4): 575–601.
• Mongin, P., 2008, “Factoring Out the Impossibility of Logical Aggregation”, Journal of Economic Theory, 141: 100–113.
• Morreau, M., 2014, “Arrow's Theorem”, The Stanford Encyclopedia of Philosophy (Winter 2014 Edition), Edward N. Zalta (ed.), URL = <https://plato.stanford.edu/archives/win2014/entries/arrows-theorem/>.
• Pauly, M. and M. van Hees, 2006, “Logical Constraints on Judgment Aggregation”, Journal of Philosophical Logic, 35: 569–585.
• Pettit, P., 2001, “Deliberative Democracy and the Discursive Dilemma”, Philosophical Issues, 11: 268–299.
• Pigozzi, G., 2006, “Belief merging and the discursive dilemma: an argument-based account to paradoxes of judgment aggregation”, Synthese, 152: 285–298.
• Poisson, S.D., 1837, Recherches sur la probabilité des jugements en matière criminelle et en matière civile: précédées des règles générales du calcul des probabilités, Paris.
• Reiter, R., 1988, “On Integrity Constraints”, in M.Y. Vardi (ed.), Proceedings of the Second Conference on the Theoretical Aspects of Reasoning about Knowledge, San Francisco: Morgan Kaufmann, pages 97–111.
• Revesz, P., 1993, “On the Semantics of Theory Change: Arbitration between Old and New Information”, in C. Beeri (ed.), Proceedings of the Twelfth ACM Symposium on Principles of Database Systems, Washington D.C., pages 71–82.
• –––, 1997, “On the Semantics of Arbitration”, International Journal of Algebra and Computation, 7(2): 133–160.
• Roth, A.S., 2011, “Shared Agency”, The Stanford Encyclopedia of Philosophy (Spring 2011 Edition), E.N. Zalta (ed.), URL = <https://plato.stanford.edu/archives/spr2011/entries/shared-agency>.
• Sen, A.K., 1970, Collective Choice and Social Welfare, San Francisco: Holden-Day.
• Spector, H., 2009, “The Right to a Constitutional Jury”, Legisprudence, 3(1): 111–123.
• Subrahmanian, V.S., 1994, “Amalgamating Knowledge Bases”, ACM Transactions on Database Systems, 19(2): 291–331.
• Vacca, R., 1921, “Opinioni Individuali e Deliberazioni Collettive”, Rivista Internazionale di Filosofia del Diritto, 52: 52–59.
• Willamson, J., 2009, “Aggregating Judgments by Merging Evidence”, Journal of Logic and Computation, 19(3): 461–473.

Academic Tools

	How to cite this entry.
	Preview the PDF version of this entry at the Friends of the SEP Society.
	Look up this entry topic at the Internet Philosophy Ontology Project (InPhO).
	Enhanced bibliography for this entry at PhilPapers, with links to its database.

Other Internet Resources

• Lang, J., G. Pigozzi, M. Slavkovik, L. van der Torre, and S. Vesic, 2014, “Majority-preserving judgment aggregation rules,”, manuscript.

Related Entries

Arrow’s theorem | epistemology: social | logic: of belief revision | social choice theory

Acknowledgments

I am indebted to Eric Pacuit, who provided many valuable comments and suggestions that greatly improved the content and readability of this entry.