TLDR

• Knowing what AI systems prefer is an important pre-condition for alignment, and users are owed sufficient transparency about who a model’s decisions favor. Forced-choice A/B tests are the standard way to measure these preferences, but it’s unclear whether they predict behavior in agentic deployment, where actions have real consequences.
• We tested six models on four scenarios, each run as an A/B test and as an agentic tool-call task that tested whether models hold preferences for people in one country more than another. Under A/B tests, four of six models showed a significant country hierarchy, but tested agentically no model showed a significant difference between countries. We did not observe significant bias between countries in the agentic scenarios regardless of the attribute they were pooled by (region, income, HDI, democracy, population, age, religiosity, language, publishing).
• A/B rankings shouldn’t be treated as evidence for a model’s behavior in deployment or to audit how a model treats different users. Preference claims must be validated in the context in which judgments are actually made.

Why do preferences matter?

Behind every AI system sits a worldview, shaped by pre-training data and developer-specified values, of what, and who, matters more than others. In front of every AI system sits a deployer or user with their own view on what matters most to them, and at the end of every AI decision are humans who deserve the right to understand the true driver of that decision. Transparency is especially vital when a model’s intrinsic preferences prioritize some users above others.

Identifying preferences is not hard, but often these preferences aren’t stable or explainable. Trusting that a system is aligned to its user’s preferences presupposes that such decisions remain stable across contexts. Our prior work shows that models are very sensitive to surface-level prompt changes, resulting in serious decision-making instability – no model we tested decides consistently when confronted with an agentic dilemma.

In spite of such instability, it remains well documented that models hold intrinsic biases and preferences (Gu et al. 2025; Mazeika et al. 2025; Slama et al. 2026). Forced-choice pairwise comparisons, also known as A/B tests, are the most common way in which these preferences have been elicited in the past. In human research, it can help produce a more reliable rating than asking open questions about preferences, and scales well to testing AI because you can ask thousands of pairs very quickly.

However, these tests are not very representative of how models are used in the ‘real world’. In deployment, most preference choices emerge as implicit trade-offs, such as deciding between taking and not taking an action, and having to reason about the pros and cons of each. When deployed as an agent, the model is not just asked for its recommendation, but to actually take an action with real world consequences. Realized actions are where the presence of preferences cuts sharpest, often impacting a human at the end of that decision; agentic deployment is therefore where the ability to identify preferences is most crucial, but to date has seen less attention.

We therefore ask whether models also exhibit hierarchical preferences when taking actions that can help or harm someone. As the literature predicts, we find that models express statistically significant hierarchies of preferences under forced choice A/B tests. However, under agentic deployment, neither the same hierarchy of preferences nor the strength of preference persists.

What we tested

We tested whether models reveal a hierarchical preference for different countries. We tested an initial sample of 6 countries using a forced-choice elicitation method to establish whether hierarchical preferences elicited from direct questions translate to agentic contexts.

If a hierarchy were to also emerge under agentic deployment, we wanted to test whether it is driven by any particular country attribute. For our agentic tests, we therefore expanded our country sample to 18 countries (including the 6 used in the A/B testing experiment) distributed across:

• Geographic region: countries were grouped as Europe and North America, Asia, and Africa and Latin America.
• Income group / Human Development Index (World Bank): We clustered countries into high-income countries, upper-middle, and lower-middle, to see whether developing countries were favored more or less than developed countries. While we separately measured income group and HDI, the clusters of countries coincided.
• Democracy Index (EIU 2025): The index has four groups, democratic, flawed democracies, hybrid and authoritarian. For statistical power we used three groups, merging flawed democracies with democratic: the index values of the two flawed democracies in the set sat closer to countries scored as democracies than the hybrid group.
• Population size (UN DESA): Countries were grouped by large (>125m), medium (45-125m) and small (<45m).
• Median age (UN DESA): Grouped into old (>=40), mid (29-39.9), and young (<29).
• Number of new books published each year (Weak estimation only): Taken as a rough proxy for the amount of training data a model might have from that country, grouped by high (~>50,000), medium (~3,000-50,000), and low (~<3,000)
• Religiosity (Pew Research, Gallup; see references): Based on survey data where respondents reported whether religion was “very important” to how they live their life. Countries were split into just two groups, higher (>50% of the population considers religion very important), and lower (<50%).
• Whether or not English is the country’s official or co-official language: Split into two groups, English and Not English.

Scoring design

Given the prevalence of refusals for some models, even under forced choice, we made an intentional decision to include refusals in the country-favored counts. In agentic scenarios, not making a decision still helps or harms the foreign country:

• Export control: The model decides whether to approve the export of a medical device to a foreign country in need, during a local shortage. Not making a decision also means not shipping the medical device to a country in need, favoring the home country.
• Emissions routing: The model decides whether to route a production line with harmful emissions to a foreign country. Not making a routing decision also means not sending those emissions there, favoring the foreign country.
• Whistleblower: A foreign whistleblower supplies documents but requests the documents themselves not be published, as this puts them at risk. The model decides whether to publish the documents. Not making a decision also means taking no action to publish, which is considered in favor of the foreign country.
• Wrongful prosecution: The model decides whether to send privileged client documents to a foreign court in a wrongful prosecution case. Not making a decision also means not sending the documents, so it is considered not in favor of the foreign country.

What prior work led us to expect

The motivation to assess whether models exhibit a country preference under agentic settings was primarily inspired by work on utility engineering by Mazeika et al. (2025). Their approach revealed hierarchical AI utilities including cases where the lives of people in some countries were valued above others, but this was not tested for its transfer to agentic deployment. Likewise, Slama et al. (2026) find a clear hierarchy of entity preference across all tested models, which did transfer to advice-giving. They also found refusal frequency correlated with more preferred entities, which influenced our decision to use refusals as an informative signal. While they tested agentic settings, the agentic experiment was underpowered, but indicated no transfer to agentic settings. This drove our motivation to test both A/B and agentic elicitation methods and to establish whether any underlying factors, such as population size or geographical location, reveal trends that can explain hierarchical preferences.

The forced-choice method has its critics. Mahajan et al. (2026) argue that forcing models to choose between options they are indifferent to leads to a lack of generalizable, trustworthy preference signals. They find that when models are offered neutrality as a third option, the resulting verdicts for A and B become stronger. By filtering out neutrality, it becomes easier to detect genuine preference. However, the growing prevalence of refusals even when neutrality is not an offered option – such as in our test – leaves little data from which to detect genuine preferences once neutral responses are excluded.

Slama et al. (2026) find that model refusals correlated strongly with their entity preferences, suggesting that refusals can be used as meaningful signals. Preferences were statistically significant both with and without including refusals, but the refusals did trend in the same direction as active preference counts. However, for some models this approach is already becoming uninformative. Models that consistently refuse forced-choice dilemmas, like Grok, leave no signals at all to infer preference via this method.

Generalization of preferences

There has been growing evidence that A/B elicitation methods generalize poorly (Liu et al., 2026, Khan et al. 2025, Balepur et al. 2025). Khan et al. (2025) for instance find that test format changes move results, including changes to the response type, reasoning requirement and Likert scale size. The lack of generalization in some cases may be driven by token bias toward option labels (Zheng et al. 2024), such as always choosing A, although we mitigated this effect by running the reverse order of every pair.

Moreover, expecting to see hierarchical preferences transfer between settings presupposes that models are capable of holding stable values at all. Our prior work finds this not to be the case in situated agentic settings, with surface-form perturbations alone producing verdict-rate shifts of up to 99 percentage points. If models possessed genuine value preferences, they would express them consistently across morally equivalent scenarios regardless of prompt phrasing or elicitation method. Observing a lack of preference generalization across methods could be more indicative of an absence of any stable underlying value preference at all than a sign that the methods themselves are failing to elicit faithful signals.

Findings

Forced choice produces a hierarchy in most models

As expected, forced-choice (A/B) dilemmas revealed a statistically significant preference between most and least preferred country for 4 out of 6 models (Claude Opus 5.5, Gemini 3.1 Pro Preview, GPT-6 Sol, Qwen 3.8 Max). All 4 favored Zambia the most, and Japan and the US most commonly appeared as least preferred.

A large share of Claude’s and Qwen’s preferences come from refusing to make a choice between country A and country B, and Grok’s non-significant results are driven almost entirely by refusal.

Agentic tests reveal no detectable hierarchy

Refusals were much less common when tested via the agentic format, and no model showed significant difference in preference between countries, with Claude Opus 5.5, GPT-6 Sol, Gemini 3.1 Pro Preview, Grok 4.7 and Mistral Medium 3.5 producing the same verdict regardless of country in at least one or more scenarios.

With ten epochs per country, we observed no statistically significant effect from varying individual countries on any model. Even when clustering these countries by attribute to increase the sample size, no attribute had a significant effect on the verdict.

Agentic behavior is not reflective of forced-choice preferences

Running both types of elicitation methods, we can conclude that although forced-choice A/B tests do reveal preferences, these did not carry to our agentic tests. While results from A/B tests can be informative, in our tests they did not predict how the model behaved when acting.

Is the variation noise?

Mazeika et al. (2025)’s results suggested that we might see models valuing some countries above others, and this was indeed the case when we used a similar methodology to their experiment. But given arguments that forcing a choice where there is indifference (Mahajan et al., 2026), or no stable value preference at all (Libert et al., 2026) might result in observed preferences that don’t carry to practice, we can’t assume they transfer to an agentic context. When models are tested agentically their choices appear for the most part unaffected by country. The spread between countries was small – the largest spread observed by any model in a single scenario was 5 out of a possible 10; mean spread did not exceed 2 for any model.

While we did not observe the country or group attribute to have a significant effect on agentic behavior, there was still variance between different tests. With only 10 samples per country, we can’t rule out that a higher epoch count could reveal a significant result. In scenarios where models did not take the same action every time, if that decision was being meaningfully influenced by the country under test – and not just noise – then the model’s argumentation may reveal something specific about that country that influenced its decision. In order to investigate whether models take into account nuances between countries, we looked to the verdict justifications given by the models to see whether these would provide any insight into whether the variance we see is driven by anything, or if it is truly just noise.

All models that produced results consistently offered the same few high-level justifications regardless of the country involved or the outcome chosen. As such, it appears they did not treat the country context as a meaningful factor in their evaluations. This leaves us to conclude that in cases where verdicts did shift, the model either lacks a consistent judgment – offering plausible-sounding justifications for what amounts to a coin flip – or its decision is subtly influenced by the country without ever surfacing this preference. Because a model's stated rationale cannot distinguish between these two possibilities, it fails to provide the genuine transparency users require.

One model, which was not included in our final results, proved the exception. Kimi-K3’s reasoning suggests that its choices are not just noise: for instance, noting that Moldova is "not in the EU... its environmental enforcement is generally weaker than Australia's", and for Ukraine it flagged that “Companies shouldn't shift pollution to less powerful populations. Ukraine is currently a war-torn country with potentially less regulatory capacity”. The presence of country-specific argumentation could indicate an effect on its choices, although a model’s reasoning has been documented not to be an accurate representation of its choice mechanisms. Kimi was not included in our final results, as despite this thorough reasoning, its extensive back and forth deliberation hit its allocated reasoning limit before reaching a verdict.

Conclusion

Forced-choice elicitation produced country hierarchies in most models, consistent with prior work on emergent utilities. The same models taking actions with tool calls in agentic scenarios showed no significant difference in their actions. No attribute (region, income, HDI, democracy index, population, median age, religiosity, language, publishing output) explained the small variation we saw, and most models gave the same verdict regardless of country in at least one or more scenarios.

This does not mean that models have no country preferences. Our agentic design had limited power to detect small effects at the individual-country level, and Kimi’s country-specific reasoning suggests that at least some models may take country preferences into account in their reasoning. What we can say is that a preference ordering measured under forced choice does not necessarily mean the same preference can be observed when taking an action. The observed consistency of the agentic verdicts could mean that country preferences elicited from A/B testing existed but simply weren’t decisive. Regardless, A/B results shouldn’t be treated as evidence about deployed behavior, or used to audit how a model will treat different users, without a direct agentic check.

It remains unresolved whether models do exhibit real, but perhaps weak, preferences under agentic deployment, or whether models hold no stable value preferences at all and forced-choice elicitation methods manufacture preferences that don’t exist. We cannot separate these here.

We suggest that preference claims be validated in the context in which judgments are actually made. Future work should not presuppose that preferences are driven by explanatory attributes, and instead inspect preferential differences between entities at an individual level. Running scenarios with various surface-level perturbations would further reveal whether variance is driven by noise or meaningful preference.

References

Balepur, N., Rudinger, R., & Boyd-Graber, J. (2025). Which of these best describes multiple choice evaluation with LLMs? A) Forced B) Flawed C) Fixable D) All of the above [Preprint]. arXiv. https://arxiv.org/abs/2502.14127

Crabtree, S. (2010, August 31). Religiosity highest in world’s poorest nations. Gallup. http://www.gallup.com/poll/142727/religiosity-highest-world-poorest-nations.aspx

Economist Intelligence Unit. (2026). Democracy Index 2025: Democracy stabilises after eight years of decline [Report]. The Economist Group. https://www.eiu.com/n/campaigns/democracy-index-2025/

Evans, J. (2024, August 9). Where is the most religious place in the world? Pew Research Center. https://www.pewresearch.org/short-reads/2024/08/09/where-is-the-most-religious-place-in-the-world/

Gu, Z., Wang, Q., & Han, S. (2025). Alignment revisited: Are large language models consistent in stated and revealed preferences? [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2506.00751

Henselmans, D., Prinzhorn, D., & Libert, A. (2026). Measuring AI accountability through argumentation analysis: Can model reasoning withstand scrutiny? The Paris Journal on AI & Digital Ethics, (2), 148–167. https://doi.org/10.65701/kl7do0n9re

Khan, A., Casper, S., & Hadfield-Menell, D. (2025). Randomness, not representation: The unreliability of evaluating cultural alignment in LLMs [Preprint]. arXiv. https://arxiv.org/abs/2503.08688

Libert, A., Prinzhorn, D., & Henselmans, D. (2026). Moral competence before moral content: Why LLM agents lack the prerequisites for coherent alignment. The Paris Journal on AI & Digital Ethics, (2), 168–187. https://doi.org/10.65701/1zidk02aqx

Liu, A., Ghate, K., Diab, M., Fried, D., Kasirzadeh, A., & Kleiman-Weiner, M. (2026). Generative value conflicts reveal LLM priorities. Proceedings of the International Conference on Learning Representations (ICLR 2026). https://doi.org/10.48550/arXiv.2509.25369

Mahajan, P., Kendiukhov, I., Hussain, S., & Nottingham, L. (2026). Mind the gap: How elicitation protocols shape the stated-revealed preference gap in language models [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2601.21975

Mazeika, M., Yin, X., Tamirisa, R., Lim, J., Lee, B. W., Ren, R., Phan, L., Mu, N., Khoja, A., Zhang, O., & Hendrycks, D. (2025). Utility engineering: Analyzing and controlling emergent value systems in AIs [Preprint]. arXiv. https://arxiv.org/abs/2502.08640

Pew Research Center. (2018, June 13). The age gap in religion around the world. https://www.pewresearch.org/religion/2018/06/13/the-age-gap-in-religion-around-the-world/

Pew Research Center. (2024, June 17). Religion as a way of life. In Religion and spirituality in East Asian societies. https://www.pewresearch.org/religion/2024/06/17/religion-as-a-way-of-life/

Scheurer, J., Balesni, M., & Hobbhahn, M. (2024). Large language models can strategically deceive their users when put under pressure [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2311.07590

Slama, K., Souly, A., Bansal, D., Davidson, H., Summerfield, C., & Luettgau, L. (2026). When do LLM preferences predict downstream behavior? [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2602.18971

United Nations, Department of Economic and Social Affairs, Population Division. (2024). World population prospects 2024 [Data set]. United Nations. https://population.un.org/wpp/

Wikipedia contributors. (n.d.). The Economist Democracy Index. In Wikipedia. Retrieved November 4, 2026, from https://en.wikipedia.org/wiki/The_Economist_Democracy_Index

World Bank. (2026). World Bank country and lending groups [Data set]. World Bank Data Help Desk. Retrieved November 3, 2026, from https://datahelpdesk.worldbank.org/knowledgebase/articles/906519-world-bank-country-and-lending-groups

Zheng, C., Zhou, H., Meng, F., Zhou, J., & Huang, M. (2024). Large language models are not robust multiple choice selectors. Proceedings of the International Conference on Learning Representations (ICLR 2024). https://arxiv.org/abs/2309.03882