Towards Human-Level Book-Writing Capability
Jan Zierstek, Matteo Batelic, Maya Medjad, Tim Schönenberger
Abstract
Large language models optimized for instruction following and agentic tasks remain poorly aligned with the requirements of high-quality creative writing. Fiction frequently depends on behaviors that assistant-tuned models are explicitly trained to avoid, particularly deception, moral ambiguity, and unreliable narration. As a result, generated stories often appear structurally correct while remaining stylistically generic, overly explanatory, or weakly grounded in human literary behavior. We present a dataset construction and training framework for book-scale creative writing that reframes supervised fine-tuning as a prompt-to-book generation task grounded in human-authored fiction. Starting from public-domain novels, we derive a multi-resolution planning scaffold by summarizing each book at progressively finer levels, from a high-level premise to chapter- and scene-level structure. We then invert this hierarchy during training: the model learns to expand a prompt into increasingly detailed plans and finally into the original human-authored book text. This formulation preserves human prose as the final supervised target while using intermediate summaries to make book-scale generation learnable. We train a long-context language model on these prompt-to-book trajectories and study whether this objective shifts generation away from assistant-style prose and toward human literary writing.
AI Impact Assessments
(1 model)Scientific Impact Assessment: "Towards Human-Level Book-Writing Capability"
1. Core Contribution
This paper presents a dataset construction and training framework for book-scale fiction generation. The central idea is to take public-domain novels, decompose them into a multi-resolution planning scaffold (scene → chapter → book level summaries, style descriptions, character archetypes, story arcs), and then train a model to invert this process: given a synthetic prompt, generate the scaffold and then the original book text. The key insight is that by preserving human-authored prose as the final supervised target while providing intermediate planning supervision, the model can learn to generate long-form fiction that is structurally organized and stylistically closer to human writing than assistant-tuned models produce.
The paper also contributes: (1) a publicly released dataset of ~6,000 books with hierarchical annotations (~649M tokens), (2) a "Structured Guide Sampling" approach for generating diverse synthetic prompts, (3) a learning-rate-scaled stochastic downcasting technique for FP32→bfloat16 materialization, and (4) a training recipe using Ring Attention on TPU v6e pods for 262k-token sequences.
2. Methodological Rigor
This is the paper's most significant weakness. Despite the ambitious title ("Towards Human-Level Book-Writing Capability"), the paper contains no evaluation whatsoever. There are no quantitative metrics, no human evaluations, no comparisons to baselines, no ablation studies, and no generated examples. The paper is entirely a description of a dataset pipeline and training setup. The authors acknowledge this in the limitations section, noting that "several components of the training recipe are evaluated jointly rather than in isolation" and that "future work should focus on... stronger evaluation."
The dataset construction pipeline is described in considerable detail and appears thoughtfully designed. The multi-level annotation scheme (scene segmentation, narrative scoring vectors, chapter embeddings, story arcs, character archetypes, writing style extraction) is comprehensive. The use of ensemble scoring for scene embeddings and the two-stage annotation strategy (Qwen3-32B for seed, distilled 14B for scale) are practical engineering decisions. However, without any validation that the annotations are accurate or that the training produces improved outputs, the entire framework remains unverified.
The stochastic downcasting technique (Section 3.5) is mathematically well-specified and represents a genuine technical contribution, but again lacks empirical validation beyond the claim that it "improves convergence" with no supporting data.
3. Potential Impact
The problem being addressed — that assistant-tuned LLMs produce generic, over-explained, stylistically flat fiction — is real and widely recognized. If the framework works as intended, it could influence:
However, without any evidence that the approach produces better fiction, the impact remains speculative. The dataset release (~6,000 annotated public-domain books) is potentially the most immediately useful contribution, as other researchers could use it for their own experiments.
4. Timeliness & Relevance
The paper addresses a timely gap. Long-context LLMs (Gemini, Claude, GPT-4) have made book-length generation technically feasible, but quality remains poor. The observation that RLHF/instruction-tuning objectives actively conflict with fiction requirements (deception, ambiguity, unreliable narration) is an important framing that could influence how the community thinks about creative-writing alignment.
The choice to start from a base model rather than an instruction-tuned model is well-motivated and the discussion of residual instruction-following behavior even in "base" models (e.g., GSM8K performance after SFT) is an interesting empirical observation.
5. Strengths & Limitations
Strengths:
Limitations:
6. Additional Observations
The paper reads more like a detailed technical report or system description than a scientific contribution. The dataset construction is thorough but occupies the vast majority of the paper, while the training and results sections are thin. The "hyperconvergence" approach (following the Hyperfitting paper) is mentioned but not elaborated upon with training curves or convergence analysis.
The paper's title is misleading. "Towards Human-Level Book-Writing Capability" implies demonstrated progress toward that goal, but the paper only describes infrastructure for potentially achieving it. A more accurate title would reference the dataset and training framework.
Summary
This paper presents an interesting and well-designed framework for a genuinely important problem, but it is fundamentally incomplete as a scientific contribution due to the total absence of evaluation. The dataset release has standalone value, and several design decisions (hierarchical scaffold inversion, structured guide sampling, stochastic downcasting) are worth noting. However, without any evidence that the approach works, the paper's scientific impact must be judged primarily on its ideas and infrastructure rather than demonstrated results.
Generated May 19, 2026
Comparison History (26)
Geo-Expert addresses a clear gap in geological AI with a reproducible methodology, novel benchmark (Geo-Eval), and demonstrates that small domain-specific models can outperform much larger generalist models. It has strong potential for real-world scientific applications in geology and provides a transferable recipe for other scientific domains. Paper 2, while interesting for creative writing, addresses a narrower application area with less scientific rigor in evaluation (creative quality is inherently subjective) and fewer cross-disciplinary implications. Geo-Expert's methodological contributions and practical deployment advantages give it broader scientific impact.
Paper 2 likely has higher scientific impact: it introduces a large, high-fidelity, public benchmark with a rigorous, regime-aware evaluation protocol addressing a clear bottleneck in floating offshore wind (standardized surrogate-model comparison for fatigue). Its real-world applicability to certification, design optimization, and cost reduction in a rapidly growing energy sector is direct and timely, and the protocol can generalize to other engineering surrogate problems. Paper 1 is novel for creative-writing alignment, but impact is narrower, evaluation/utility are less standardized, and downstream adoption is less clearly tied to critical real-world constraints.
AgentHijack addresses a timely and practical problem in the rapidly growing field of autonomous computer-use agents. It introduces a systematic benchmark for robustness evaluation with 9 configurable corruptions, proposes a mitigation framework, and releases code/data publicly. This has broader immediate impact across AI safety, agent reliability, and HCI communities. While Paper 1 tackles an interesting creative writing challenge with a novel training framework, its impact is narrower (creative AI/NLP), and the reliance on public-domain novels may limit generalizability. Paper 2's focus on agent robustness is more urgently relevant as autonomous agents see wider deployment.
Paper 1 likely has higher scientific impact: it introduces a novel multi-agent, closed-loop framework that links natural-language preferences to validated topology-optimization parameters with vision-language critique, demonstrated on engineering benchmarks and a product case with measurable gains over ablations. It has clear real-world applications (design automation, additive manufacturing pipeline) and cross-field relevance (AI agents, CAD/CAE, HCI, manufacturing). Paper 2 is timely and interesting but is primarily a training-data/objective reformulation for creative writing; impact may be narrower, evaluation less concrete from the abstract, and applications more subjective.
Paper 1 tackles a fundamental and broadly impactful challenge—aligning LLMs with human-level creative writing at book scale—introducing a novel training framework (inverting multi-resolution planning scaffolds from human-authored fiction). This addresses a widely recognized limitation of current models and has broad implications for creative AI, NLP, and digital humanities. Paper 2 presents an engineering contribution (a runtime abstraction for LLM agents) that, while practical, is more incremental and narrowly scoped to agent frameworks. Paper 1's novelty in dataset construction and training methodology, combined with its potential to influence both research and creative industries, gives it higher impact potential.
Paper 2 addresses a fundamental limitation of LLMs in creative writing—a high-profile, broadly impactful problem. Its novel framework for book-scale generation using inverted hierarchical planning scaffolds from human-authored fiction represents a significant methodological innovation with wide applications across NLP, AI creativity, and digital humanities. Paper 1, while methodologically sound, addresses a narrower domain (IMU-based HAR) with incremental architectural improvements combining KANs and MLPs, limiting its broader scientific impact compared to the transformative potential of human-level book writing capabilities.
Paper 2 addresses a fundamental and novel challenge—training LLMs for book-scale creative writing that preserves human literary quality rather than assistant-style prose. Its multi-resolution planning scaffold and inverted hierarchy training approach represent a genuinely new paradigm with broad implications for creative AI, long-form generation, and alignment research. Paper 1, while solid, addresses VLM hallucination reduction with incremental improvements (structured pseudocode reasoning) in a crowded field. Paper 2's novelty, potential to reshape creative AI applications, and methodological innovation give it higher impact potential.
Paper 2 addresses a critical and universal bottleneck in large-scale AI development: training stability and compute efficiency. A governance layer that reliably prevents degraded runs and allows for more aggressive hyperparameters without instability offers massive, immediate utility across all foundation model training. Conversely, while Paper 1 presents a novel framework for long-form creative generation, its impact is largely restricted to the narrower domain of computational fiction and stylistic alignment.
CyberCorrect offers a more rigorous and broadly applicable contribution by formalizing LLM self-correction through control theory, introducing novel evaluation metrics (convergence rate, overshoot, oscillation), and demonstrating measurable improvements on a constructed benchmark. This framework addresses a fundamental challenge in LLM reasoning with clear quantitative gains. Paper 1, while creative and interesting, addresses a narrower domain (book-length fiction generation) with less clearly measurable impact and relies on public-domain novels, limiting its scalability. Paper 2's cross-disciplinary approach (cybernetics + NLP) and applicability to general reasoning tasks give it broader potential impact.
Paper 1 is more methodologically novel and broadly applicable: it introduces an online, adversarial bandit formulation for non-stationary multi-agent dialogue plus a reward-surrogate for sample-efficient adaptation, with clear relevance to LLM agents, RL, bandits, and social simulation. Its contributions generalize to many interactive settings beyond Sotopia (e.g., negotiation, tutoring, games), making real-world impact plausible. Paper 2 is timely and useful for creative-writing generation, but is primarily a dataset/training recipe with narrower cross-field impact and potentially higher dependence on scaling and data specifics.