Rank things by comparing pairs
People score ten things badly and compare two things well. This asks only the question you can answer reliably, then derives the priorities — and checks whether your answers contradict each other.
Rank a list by comparing two things at a time
People are unreliable at scoring ten things out of ten and reliable at saying which of two matters more. This turns the second into the first.
Compare each pair
Read each row against each column: how much more does the row item matter?
Priorities
Judgements worth revisiting
How priorities are derived
The comparisons form a reciprocal matrix: if A is 3× B, then B is 1/3 of A. Only the upper triangle is entered; the rest follows.
Priorities are the principal eigenvector of that matrix, found by power iteration. The common shortcut is to average the normalised columns, which agrees closely for consistent judgements and drifts exactly when they are inconsistent — the case you most need to be right about.
The consistency ratio compares your inconsistency against the inconsistency of randomly generated judgements of the same size. Saaty's convention is that 0.10 or below is acceptable. Below three items there is nothing to contradict, so no ratio is reported.
Why pairwise beats scoring
Asked to rate ten things out of ten, people cluster everything between six and eight, anchor on whatever they scored first, and produce a ranking that changes if you shuffle the list. Asked which of two things matters more, the same people are consistent and quick.
Pairwise comparison only ever asks the question people can answer well, and derives the numbers from the answers rather than asking for numbers directly.
Inconsistency is information, not failure
If you say A beats B, B beats C, and C beats A, the matrix cannot be satisfied by any set of priorities. The consistency ratio detects this.
A high ratio usually means one of three things: the items are not really comparable on a single dimension, someone was answering carelessly, or two items are so close that the direction flipped between comparisons. All three are worth knowing, and none of them is visible from a ranking alone.
Which is why this shows which judgements are most out of line rather than only that something is. On more than five items, "your matrix is inconsistent" without a location is close to useless.
Keep the list short
Comparisons grow as n(n−1)/2. Five items is ten comparisons; ten items is forty-five, at which point people start clicking to get through it and the consistency ratio starts measuring fatigue.
If you have twenty things to prioritise, group them and compare the groups.