Agent Personas Shift Allocation Decisions About People: What Matters Is Whether the Two Strings Match

원문 초록

Language models are increasingly used to make decisions about people, such as whom to interview, which insurance claims to approve and which patients to see first. Research on bias in these decisions has focused on whether a person's race or gender changes the outcome. However, developers also write text that the model reads before every decision, including a persona that gives the model a role and often names a project, team or office. Whether such text affects whom a model selects remains largely unexamined. We asked what happens when a string in the persona also appears in one candidate's record. Across four kinds of decision, fourteen publicly released models and 328,480 choices between two candidates, instruction-tuned models, the kind used in chat assistants, selected the candidate whose record repeated the persona's string in 71.3% of decisions between equally qualified candidates, compared with 50.1% when no string was shared. When that candidate was one step less qualified, the rates were 31.5% and 11.7%. A made-up project code was enough. The effect persisted when the record labelled the string a conflict of interest, even in models that otherwise selected such a candidate less often. When the shared string was the candidate's race, a similar shift appeared. Bias audits built around protected attributes can at most show the resulting gaps between groups, not their source.