What Selection Research Says About Giving AI Agents a Personality

Almost every guide to building an AI assistant opens the same way. Tell the model who it is. You are a world-class marketing strategist with twenty years of experience. You are a meticulous senior analyst. You are a friendly, empathetic support agent.

It has become the opening move, and it feels like it should work. Casting the role well is how you get a good performance out of a person.

The evidence on models points elsewhere, and a much older body of research points the same way about people.

The model result

A team led by Mingqian Zheng tested this directly. They assembled 162 personas, covering different kinds of relationships and eight domains of expertise, and ran them across four families of large language models on 2,410 factual questions. Then they compared the results against giving the model no persona at all.

Adding a persona did not improve performance. Some personas mildly hurt it. The work was published in the Findings of the 2024 Empirical Methods in Natural Language Processing conference, and its title says the finding plainly: When “A Helpful Assistant” Is Not Really Helpful.

That result surprises people, and it should not. It has a much larger and much older parallel.

The hiring result

Organisations have been trying to predict job performance for a century, and personnel psychology has kept a running scorecard of which prediction methods actually work. The methods are scored with a validity coefficient, the correlation between what you measured before someone was hired and how well they performed after. A coefficient of zero means the measurement told you nothing, and one would mean it told you everything, which nothing in this field comes close to.

In 2022, Paul Sackett, Charlene Zhang, Christopher Berry and Filip Lievens published a revision of those estimates in the Journal of Applied Psychology. Their argument was technical, concerning a statistical correction that had been applied too generously for decades, and its effect was to lower most of the numbers and reshuffle the order.

Here is the top of their revised list:

And here is the bottom:

Read those two tables next to each other and the sorting principle is visible. What sits at the top are the methods that put the candidate in contact with the work itself. A structured interview asks how you would handle a specific situation this job produces, a work sample hands you a piece of the job, and a job knowledge test asks what you know about it. What sits at the bottom are the methods that ask what the candidate is like as a person.

Conscientiousness is the most useful personality trait in the whole literature, and on its own it sits at .19. The rest are close to noise. The unstructured interview, meaning the chat, the culture read, the gut feel about whether this person is a fit, comes in at the same .19, which is why people who study hiring for a living are so rude about it.

Two literatures, one conclusion

One of these findings is about silicon and one is about people, and it is worth being careful about how far the parallel goes. I searched for a validity study run on agent configurations and did not find one. The Sackett estimates describe human job performance, and quoting a coefficient from that table at an AI system would be a category error.

What transfers is not the number. It is the design logic, and it is unusually clean:

Descriptions of a worker predict less than demonstrations by a worker.

That holds whether the worker is a person or a process. Telling a model it is a world-class analyst is a description. Handing it three real analyses and checking the output against what a good one looks like is a demonstration. Only one of those two things has ever told anyone very much.

What a persona is actually for

None of this means the persona paragraph should be deleted. It means it has been asked to do a job it cannot do.

What a role instruction genuinely governs is stance. It settles whose decision a given call is, the assistant’s or yours. It settles what happens when the assistant is unsure, whether it guesses or stops and asks. It settles whether it holds a position under pushback or folds to be agreeable, and how the whole thing sounds.

Those are real and they matter, particularly the last one, because an assistant that drifts in and out of your organisation’s voice is a liability in front of customers. These are questions about conduct rather than competence. In the language of the hiring literature, a persona is a treatment rather than a predictor: it shapes how the work gets done, while leaving the question of whether this configuration can do the work at all completely open.

If you want to know that, you have to look at output.

The work sample, ported

The good news is that the method at the top of that table is embarrassingly cheap to run on an AI agent, far cheaper than running it on a person.

It takes four things.

Ten real tasks. Not invented test cases. Ten pieces of work the job has actually produced, pulled from the last few months, including the awkward ones: the request that was ambiguous, the input that arrived in the wrong format, the one where the right answer was to refuse.

A scoring key, written first. For each task, what a good output looks like and what disqualifies one. Written before you look at any candidate’s answers, because a key written afterwards is a rationalisation.

Two or three candidate configurations. Different instructions, different models, different amounts of supporting material.

A blind scoring pass. Strip the labels, score the outputs, then look at which configuration produced which.

That is an afternoon of work, and it will tell you more than any amount of prompt-tuning by feel. It also has a property that matters more over time, in that it becomes a regression test. When the model updates underneath you, or someone edits the instructions, the same ten tasks tell you whether the job still gets done.

The part most teams cannot do

There is a catch, and it is the reason most teams reach for a persona in the first place.

To write the scoring key, you have to be able to say what good output looks like, specifically enough that two people scoring the same work would land in the same place. A surprising number of organisations cannot do this for their own jobs. It has a name in personnel psychology, the criterion problem, and it long predates any of this. The difficulty sits in agreeing what performance is, more than in measuring it once agreed.

An AI project makes that gap impossible to hide. You can hire a person into a vaguely defined job and they will work out the standard by watching the people around them. An agent cannot. Whatever you failed to write down, it will fill in with the average of everything ever written on the subject, which is exactly what you did not want.

So the sequence runs: write down the job, write down what good looks like, then test candidates against it. Personality, if it appears at all, appears at the end, deciding how the work sounds rather than whether it is any good.

That is what a well-run hiring process has looked like for fifty years. The novelty is in the candidate.

Sources: Zheng, M., Pei, J., Logeswaran, L., Lee, M., & Jurgens, D. (2024). When “A Helpful Assistant” Is Not Really Helpful: Personas in System Prompts Do Not Improve Performances of Large Language Models. Findings of EMNLP 2024. · Sackett, P. R., Zhang, C., Berry, C. M., & Lievens, F. (2022). Revisiting meta-analytic estimates of validity in personnel selection: Addressing systematic overcorrection for restriction of range. Journal of Applied Psychology, 107(11), 2040–2068, Table 3.