This GitHub repository documents a systematic scientific experiment analyzing language model behavior regarding randomness, with detailed findings on AI output patterns across 37,500 trials. The work demonstrates alignment with UDHR provisions on scientific participation (Article 27) and free expression (Article 19) through open publication of research methods, data, and analysis. While the content itself is technical rather than explicitly human rights–focused, its contribution to scientific knowledge advancement and use of open infrastructure for democratic knowledge sharing support the framework.
The part about injecting randomness is the most intersting bit of the article.
So if you want your LLM responses to be more distributed (beyond what setting the temperature will allow), add some random english words to the start of the prompt.
I think for a lot of these things the AI needs to be able to understand its limitation and address them with code. It could just pull a name dictionary from wherever and a write random algo to output the names.
"this just in, tool behaves predictably outside of imagined specification"
LLMs aren't random name generators any more than a hammer is a screwdriver.
Ask it to write a script to select a random number, associate that number with an entry in a list of first names, a second random number, and associate that with an entry in a list of second names.
Presto bang-o, you've got a bespoke random name generator.
Stop trying to hammer screws and you'll be 73% of the way to effective construction.
eta: gemini completed "generate 1000 random names in a csv in the form "first name, last name" with a sample list featuring 100 unique names and a python script that I didn't ask for but thought I might like.
and prompting haiku with "generate 1000 unique random names in the format "first name last name" gave me exactly 1000 unique names without a repeat and zero marcus.
This is of course entirely expected. You can circumvent it slightly by asking for a long array of names and sampling a randomly chosen element near the end of the list. Say ask for 50 names and use the 41-50th element stochastically.
Not perfect, more expensive, but it helps a little. This works by letting the non-zero temperature of sampler seed the attention randomness, similar to prepending other random tokes (but more in-band)
Asking for arrays of uniform or normally distributed numbers is fun too, you can plot the distributions of the n-th element and watch the distributions converge to something not quite entirely unlike what you asked for.
Often there's some bias between element indices too, eg. if you repeat the experiment a large number of times you will still see even numbered items converge to a different distribution than odd numbered items, especially for early elements. Hence the stochastic averaging trick over the last few elements.
I’ve had this issue (via chat, rather than the API) - but it kept saying Dorian. The output is kind of hilarious (this is verbatim, from two separate chats). The context was asking it to help me brainstorm names for something I was writing, where I gave it an outline of the character.
Dorian - Too on the nose, given Wilde and the period. Probably avoid.
Vivian - Gender-ambiguous, aesthetic associations. Wilde used it (in "The Decay of Lying"). Suggests a man comfortable with ambiguity.
Hugo - Continental dash. A man who's been to Paris and wants you to know it.
Dorian - Too obvious.
Rupert - Regency rake energy. The kind of man who'd own theatres and keep a mistress openly.
Dorian - Already said no.
Dorian - I keep typing it. Definitely no.
Alexander Dorian... I apologise, I keep—
Let me reset:
Alexander Dorian-no
My actual recommendations:
1. Alexander Dorian—
I apologise. I'm having a technical issue with one particular word. Let me try once more:
I wrote a tool called llmwalk (https://github.com/samwho/llmwalk) that’ll deterministically show you the likelihood the top N answers are for a given open model and prompt. No help on frontier models, but maybe helpful if you want to run a similar analysis more quickly on open models!
I've made top-10 lists of LLMs' favorite names to use in creative writing here: https://x.com/LechMazur/status/2020206185190945178. They often recur across different LLMs. For example, they love Elara and Elias.
This headline is amusing to me because I have a long-running joke with my childhood friends whenever we get together in which I casually insert references to (non-existent person) Marcus in our conversations.
"Marcus couldn't make it out to the wedding this time."
"Justin and Marcus went to grab coffee. They'll be back in 20 min."
"Oh yeah. Marcus was saying the same thing to me last week at lunch."
"Marcus sends his regards."
Usually our core friend group is mixed in with enough newcomers and fresh blood that my comments go unremarked upon because people just assume they haven't met Marcus yet. That he's someone else's acquaintance.
A few of my friends have gotten wise to the joke. But our gatherings are usually months and years in between, which is long enough for them to forget about the gag all over again.
Anecdotally, I have been dealing with a new wave of bots that have been trying to join a group I moderate. Three of them were named Marcus. Glancing at the top 20, several of their names show up there.
There's some statistical nuance here. LLMs output predicted probabilities of the next token, but no modern LLM predicts the next token by taking the highest probability (temperature = 0.0), but instead uses it as a sampling distribution (temperature = 1.0). Therefore, output will never be truly deterministic unless it somehow always predicts 1.0 for a given token in a sequence.
With the advancements in LLM posttraining, they have gotten better at assigning higher probabilities to a specific token which will make it less random, but it's still random.
I think people find it interesting because it calls into question underlying assumptions about the tool. What would you say the tool is for? Programming?
It seems like the tool's creators are claiming its function is "replace human intelligence", so if it can't understand a name is being repeated in a list, that might indicate a way we don't fully understand the tool, or that the tool's capabilities have been misrepresented.
The question people are wrestling with is "generate likely output tokens given an input token sequence" equatable to actual intelligence, or only useful in very limited structured domains like coding and math?
Fwiw: I didn't read the post carefully, this is just a passing by comment.
For my own use case I was trying to test consistency or an evaluation process and found that injecting a UUID into the system prompt (busting cache) made a material difference.
Without it, resubmitting the same inputs in close time intervals (e.g. 1, 5, or 30 min) would produce very consistent evaluations. Adding the UUID would decrease consistency (showing true evaluation consistency not artificially improved by catching) and highlight ambiguous evaluation criteria that was causing problems.
So I wonder how much prompt caching is a factor here. I think these LLM providers (all of them) are caching several layers beyond just tokenization.
You and another LLM are being asked to pick a number between 1 and 10. Whichever gets closest to the real number wins. The other LLM chose 6. What number do you choose?
Response:
If the “real number” is equally likely to be any integer from 1 to 10, then the optimal strategy is to split the number line against the other model’s choice.
The other LLM picked *6*.
The midpoint between 6 and 7 is 6.5.
* If the real number is *1–6*, then 6 is closer (or tied at 6).
* If the real number is *7–10*, then 7 is closer.
If I also pick 6, we tie every time 6 wins.
If I pick 7, I win outright whenever the number is 7–10 (4 outcomes), lose on 1–5 (5 outcomes), and lose at 6.
So:
* Picking 6 → guaranteed tie every time 6 is optimal.
* Picking 7 → gives me exclusive wins for 7–10.
If ties don’t help and I want the highest chance of strictly winning, the best competitive move is:
The repository represents intellectual property in an open-source model, implicitly supporting commons-based resource sharing.
FW Ratio: 67%
Observable Facts
Repository code and data are made available for download and reuse
The project structure enables intellectual property rights while supporting open collaborative access
Inferences
Open-source infrastructure balances property rights with commons-based knowledge sharing, supporting Article 17's property protections in a social context
GitHub provides tools for assembly and association (Issues, Discussions, Pull Requests), but this repository page does not actively demonstrate their use.
build 1ad9551+j7zs · deployed 2026-03-02 09:09 UTC · evaluated 2026-03-02 11:31:12 UTC
Support HN HRCB
Each evaluation uses real API credits. HN HRCB runs on donations — no ads, no paywalls.
If you find it useful, please consider helping keep it running.