2026-09-22

A better consciousness test for LLMs: can it finish a novel on its own?

I have 30 AI-written novels under contract on a Chinese web-fiction platform. 27.4 million characters. The median number of people currently reading one of them is 3. Thirteen of them sit at zero.

On the same author account, the one book I wrote by hand — 1.03 million characters — has 1,078 concurrent readers.

Those numbers convinced me of something: the best test for whether a large model is conscious is not the Turing test and not any benchmark. It is whether it can finish a novel on its own.

Not produce a million words. That stopped being hard a long time ago. A book people finish.

The Turing test says: if you can’t tell whether you’re talking to a person or a machine, you should concede the machine thinks.

Any current model clears that line, easily. Nobody responded by conceding consciousness. We just moved the line and said the Turing test was never enough.

Benchmarks go the same way. Math, code, commonsense reasoning — saturated one after another, and each time the reaction is that the questions were badly designed.

Both have the same flaw: they are short, and they have answer keys.

Short means a model only has to hold the performance for a few minutes. An answer key means it can pattern-match its way to the end without ever knowing what it is doing.

There’s a third kind of evidence that’s worse still: asking a model whether it’s conscious and treating the answer as data. It was trained on text humans wrote about their inner lives. That isn’t introspection. It’s recitation.

Finishing a novel closes both holes

A serialized web novel is a million words, three hundred chapters, months of continuous output. That timescale alone rules out short-term mimicry.

And there is no answer key. No verifiable objective function can tell a model what should happen in chapter 147.

Getting it done requires four things at once.

Holding one intention across months. Not remembering the earlier text — retrieval solves remembering. Knowing, at chapter 200, why you kept that character alive in chapter 12, and making the current choice serve that reason.

Modelling another mind. A novel is a continuous guess about a reader’s state: when they tense up, when they reach to put the phone down, when a chapter has to end on something that makes stopping uncomfortable. Theory of mind as a practical skill, performed three hundred times in a row.

Self-correction. At chapter 200 you find that a premise from chapter 40 has collapsed, and you have to go back and deal with it. That requires memory of your own output, a judgment about it, and the admission that it was wrong.

Deciding what comes next. Nobody hands it a chapter outline. It has to decide what happens, and the basis for deciding lives nowhere but in its own sense of the whole.

Those four together are what the word consciousness points at in ordinary use.

Thirty books, six models — the gap isn’t prose

The 30 books settled one question: generation is solved.

No collapsed arcs, no broken characterization, no missed updates. 27.4 million characters of stable prose. I used Claude, Codex, Gemini, Qwen, MiniMax and Kimi, and wrote an orchestration tool to run them in batches.

Nobody read the output.

The one I think the models wrote best has 18 concurrent readers. The one with the most readers isn’t, to me, the better book. My judgment of “good” and readers’ choice to keep reading diverge by a factor of 43.

What’s missing isn’t sentence-level quality. It’s the loop that knows who it’s writing for and adjusts accordingly.

The model doesn’t know when a reader wants to put the phone down. It has no such loop, and it doesn’t know that it doesn’t.

Why this criterion is harder than a benchmark

It can’t be gamed. You can’t train specifically for “a book people finish,” because the readers aren’t a fixed test set, and the hook that worked last year doesn’t work this year.

It’s long-range. Three hundred chapters of consistency can’t be faked. The retention curve marks the exact chapter where the writing started phoning it in.

The result is objective. Whether a book is good is arguable. Whether people finished it is a number in a dashboard. That’s harder evidence than any panel of judges.

It stays functional and avoids metaphysics. I’m not going to argue about whether a model has inner experience; that question may never resolve. Whether it can independently complete a task requiring sustained intention and a model of other minds is testable.

Objections

Plenty of humans never finish a book either. True. But humans stop because they give up, run out of time, or run out of money — not because by chapter 150 they’ve forgotten what they were trying to say. Quitting is a choice. Amnesia isn’t.

The bar is too high. Deliberately. We already have a low bar — the Turing test — and it was cleared without convincing anyone. A criterion that has already failed is worth trading for one that hasn’t been crossed yet.

Whether a novel is good is subjective. Which is why the criterion isn’t “good.” It’s “finished by readers.” I threw out the subjective half.

This is a literary task, not a test of consciousness. It’s one of very few tasks that demand long-range intention, other-mind modelling, self-correction and autonomous decision-making simultaneously. You could substitute others — directing a feature film, running a three-year project — but a novel has the lowest barrier to entry, the clearest verification, and the smallest cost. One GPU to start, one dashboard number to settle it.

I’m still watching that retention curve

I’ll keep writing novels with AI, and I know it can’t yet produce one people read.

The day a book runs start to finish from the model, with no chapter touched by me, and the readers stay — that’s the day I reconsider the consciousness question seriously.

Until then, however saturated the benchmarks get, I’ll read it as test-taking.

The figures come from my own author dashboard on Fanqie Novel (checked 2026-09-21): 30 AI-written titles totalling 27.4 million characters, median 3 concurrent readers, 13 at zero; one hand-written title at 1.033 million characters and 1,078 concurrent readers. “Wrote best” is my own judgment, not a platform metric.