Created by Zikai Xiong · Based on Xiong, Dalmasso, Mishler, Potluru, Balch & Veloso, arXiv:2311.00109
A bank wants to train a credit model on its past loan decisions (here 1,000 synthetic applications). Those decisions approved group A 62% of the time and group B 38%, partly because incomes differ a little between the groups and partly because past decisions slightly favored group A. The bank’s rule asks for the two rates to be closer. FairWASP brings them closer without editing anyone’s record: it drops some applications and counts similar ones twice, moving the data as little as possible. On the right, the usual fix does the same job.
Each application $i$ has a group $d_i$, features $x_i$ (income, credit score) and a decision $y_i$. Giving the applications weights $\theta_i$ ($\sum_i\theta_i=n$) changes the data; the original data has all $\theta_i=1$. FairWASP [F] picks whole-number weights, so it only drops ($\theta_i=0$) and copies ($\theta_i\ge2$) applications:
$$\min_{\theta}\ W(\text{new data},\ \text{original data})\quad\text{s.t.}\quad \text{each group's approval rate is within the allowed range.}$$
The range is $J\big(p_\theta(y\mid d),p_Y(y)\big)\le\varepsilon$ with $J(p,q)=\max\{p/q,q/p\}-1$: for $\varepsilon=0$ both groups get the overall approval rate (50% here).
$W$ is the Wasserstein (earth mover’s) distance: the least total amount of weight times the distance it travels to turn one dataset into the other. Two applications are at distance $\sqrt{\Delta\text{income}^2+\Delta\text{credit}^2+\tfrac1{16}[\text{groups differ}]+[\text{decisions differ}]}$, with income and credit score in standard deviations. A different decision counts as one standard deviation; a different group counts a quarter as much, since the bank’s model does not use the group. This weight is a choice: the larger it is, the more any change of group costs and the closer the two methods get (with a full weight the gap at the default rule is 1.05×). FairWASP’s moves are short: a dropped application’s weight goes to a nearly identical application.
Reweighing [KC] gives every application in group $d$ with decision $y$ the weight $w=P(d)P(y)/P(d,y)$, which makes the approval rates exactly equal. For a looser rule the page applies it only as far as the rule needs: weights $1+t\,(w-1)$ with the smallest $t$ that meets it. Its distance is then exactly $t$ times the distance at $t=1$, since the transport problem scales with $t$.
At $\varepsilon=0$ FairWASP’s distance (–) equals the lower bound from the linear program, so no reweighting with equal approval rates, even with fractional weights, moves the data less. For $\varepsilon>0$ two applications tie at the optimum and are rounded to whole numbers, which can cost a little more than the best fractional weights.