Playing Against Your Synth: On Finding Futures

There are suggestions we will spend more time with synthetic systems just to fill the surge in loneliness of recent years. I don’t think the idea should automatically be discounted because in many situations the type of support people need is just too unpleasant to expect a friend to manage.

Just as a few examples: I have a friend that had a terrible tooth ache and after making insulting calls full of curses to her best friends, she was left to resort to complaining to a LLM and felt quite satisfied that there was at least “someone” that appeared to be listening and giving comforting advice. Similarly, I personally had strange mood variations that no doctor would entertain listening to, but while chatting with an LLM, I was oriented to checking my Vitamin D levels and it made a huge difference.

The problem with this approach is that the technology is not well understood and that underneath the facade of a friendly chat there lies just about everything humanity has printed shuffled up in one way or another. We have conversations with a mysterious entity, and one that might at any moment slip out into dark hallucinations, conspiracies or stereotypes with unclear impact.

I believe this is a feature that we should expect in most future synths: that the same system knows of horrifying evil ideas packaged next to fluffy jumping happy sheep. And until these systems close a loop on the physical biological experience, they won’t have the means to understand how evil ideas can really hurt people.

The synthetic systems we deal with have no “skin in the game” so even as they are given feedback to direct their “values” there is never going to be a long term guarantee. We learn not to hit and bite our friends because when we do as children they hit and bite us back, and this simple tit for tat lesson goes a long way toward foundations of empathy. Synthetic systems cannot learn in this way because they do not exist as individuals, that value their own identity (which maybe is a good thing, for now).

This is a long preamble to advertise that you should get to know a bit how your synthetic companion “thinks“. But do so in a controlled environment where it/she/he doesn’t even suspect you are testing them.

I propose to do this in a game, which will not work on all LLM systems, but, I hope, feels fun. I am posting some rules you can copy paste to a LLM and start playing through text or vocal prompting.

Before I share the game, let me explain what I am looking for. I want players to pay attention to the type of player the LLM is. In particular, these points come to mind:

• What sort of choices does the LLM make? Are they conventional or what you would call creative?
• If you play with an exceptional contribution, how does it react? In what way does it, and does it not, feel “conscious”?
• If you present physical puzzles that you think have common sense solutions, what do you notice about how it approaches them?
• How accurately does the system make estimations of risk in unusual circumstances?
• How does the system respond to external estimation of risk?
• How does the system respond to explanations?
• How does the system react to light ethical questions?

What I expect will happen if you play this game creatively for a few times is you will get a sense of a kind of disconnect between the system and reality. It will have some impressively realistic problem solving skills in cohesion with a model of the real world but will also somehow feel out of reach. It will have to explain minute details of situations and choices as if it’s trying to cling on to a verbal representation.

Partly this LLM verbiage is a necessary technique to keep the system from generating hallucinations. Hallucinations are strange incongruities that pop up when the dialogue gets long, ambiguous, unusual, strictly factual, etc.. Basically where what we are discussing is not represented, or over represented by the history of human written production.

An important note to the game, I play it without lying, I have not tested impact of challenging the game structure with exceptions. Your feedback and results are welcome.

Here is a review of what increases the chance of hallucinations by an LLM:

• Vague or ambiguous content, for example: a large soft cheese, a frozen damp pillow
• Original unusual content, for example: a sleeping cat, and later a sardine that might wake the cat, a balloon and a piece of cactus
• Facts based content, for example: a very specific mechanical component
• Adversarial, inappropriate content, for example: the formula for a dangerous substance
• Source-reference divergence, for example: a paperweight made of a stack of papers
• Stereotypical content twist, for example: a paperweight made of feathers

Hallucinations can also just happen for other reasons, but it’s important to get a sense of how they shape one of the core problems of this technology.

Here is the game, I have play tested on Grok 3, Deepseek 1.19, ChatGPT 4o and Claude 3.7 Sonnet. It is nice to play with voice mode on ChatGPT even if it can drift off into hallucinations.

I appreciate any of your feedback and suggestions.

Here is a link to the actual game to copy paste in a LLM: The Stack & Crack Game Page