The Technical Architecture of Project Xanadu

(never give up, never surrender)

Abstract

This document contains my notes and diagrams as I have studied the underlying technical architecture of Project Xanadu. I do not act for Project Xanadu nor claim any ownership in any of its trademarks. I am simply a fan of the project and dream of taking it further using my technical skills.

1 Overview

We need a way for people to store information not as individual "files" but as a connected literature. It must be possible to create, access and manipulate this literature of richly formatted and connected information cheaply, reliably and securely from anywhere in the world. Documents must remain accessible indefinitely, safe from any kind of loss, damage, modification, censorship or removal except by the owner. It must be impossible to falsify ownership or track individual readers of any document.

This system of literature (the "Xanadu Docuverse") must allow people to create virtual copies ("transclusions") of any existing collection of information in the system regardless of ownership. In order to make this possible, the system must guarantee that the owner of any information will be paid their chosen royalties on any portions of their documents, no matter how small, whenever and wherever they are used.

2 An Architecture for Information Sharing

2.1 The Docuverse

  • more literate, less a visual entertainment
  • more balanced btw consumption and creation of content
    • not only original, but critique, proofing, direct rebuttal
  • ownership remains with creator
  • reputation follows ownership, a solution for trash content

2.2 'links' don't belong in the content or in the viewer but in the middle

2.3 therefore the docuverse is divided into 'content' and 'links'

2.4 The Invariant and the Variant Streams

  • original content ingested, owned, distributed
  • arrangements and taggings

2.5 A Coordinate System

2.6 Relating Coordinates from Place to Place

2.7 The Power of Separating Content from References

2.8 a new front-end in the world of web browsers

2.9 a means of identity, rights and access control

2.10 new paradigms for interacting graphically with information

2.11 new mechanisms for representing information

2.12 sifting and searching operations

2.13 storage (local)

2.14 storage (remote, federated, peer-to-peer sharing)

2.15 what about Unicode?

3 A Descriptive Walkthrough of the Backend

3.1 Initial Entry of New Text

3.1.1 The Need

3.1.2 What Happens

  • entered text is appended to the invariant stream, thereby being given a coordinate we can use to reference and recall it
  • at the same time, a link is created to represent the position of that new text within a document, which is a variant stream

    these are NOT the jump links you may be used to on the WWW, but rather content links that relate a range of positions in the document to pieces of content that make up the document

    (insert diagram of 2-way link from invariant to variant)

    A document is a variant stream because its content will change over time, as the author rewords, deletes and rearranges his content.

As the author edits his document, behind the scenes links are created, deleted, adjusted to map the specific invariant content he wants into the places in the document that he wants.

3.1.3 The Benefits

  • new text is always appended to the invariant stream, so no shifting text around is needed.
  • links give us a fast way to logically rearrange or remove text

3.2 Versioning

3.2.1 The Need

At some point the author wants to save his work, declaring it to be draft version 1, and continue making changes

3.2.2 What Happens

A new document, related to the first, is conceptually created. Since the invariant content is, well, invariant, the invariant stream is used by both the old and new document.

Since the author wants to make further changes without disturbing the old edocument, all of his links are copied into the new document. In this way the editor can modify the copy of those links to move, add or delete text in the new document.

3.2.3 The Benefits

  • there is only one copy of any text, regardless of placement or non-use
  • either version of the document can be viewed, giving us history
  • each document has its own layout and, since a document is versioned, some content may appear in different places in different versions.
  • programmatically we can query the links for both documents and figure out where a specific paragraph or phase has moved to, giving us a diff.

3.3 Giving the Links a Home

3.3.1 The Need

Notice that the links are associated with a particular version of the document, and that most of them are unchanged across versions. Yet we have duplicated them in each document.

3.3.2 What Happens

What if instead we applied the same technique to the links as we have the text, keeping only a single copy and selectively mapping each into a document?

So we store the links in the invariant stream as well, sharing them all across all versions of the document, and when a link is relevant to the current version it is mapped in as well.

To do this we give every link another leg – so we have the from, the to and the home leg. The home leg points to the specific version of document to which it applies, so if a link only points to version 1 but not version 2, the content addressed by that link is omitted in version 2.

We make the home an endpoint instead of just a reference of a specific document/version, because versions themselves represent a numberline or dimension e.g. version 1, 2, 3, 4, … This allows us to say that a link's home is on a range of versions, again avoiding duplication of links.

3.3.3 The Benefits

  • so we have a single copy of all text and links, no storage duplication
  • queries are more efficient to make that tell us which content is used in which versions

3.4 Giving the Links a Type

3.4.1 The Need

So far we've seen content links, which map text into places within a document, but links are far more powerful. One use is altering presentation, as any document system needs boldface, italics, underline and so forth.

3.4.2 What Happens

So let's distinguish between link types, by giving them a fourth endpoint, called type.

Again we make this an endpoint instead of a set of type constants, we can declare any number of link types and arrange them by function. So all the presentation link types are grouped into a range, and all the content link types are grouped into another range of the type dimension.

3.4.3 The Benefits

  • This allows us to query for all presentation links (for the front-end to mark up the presentation) or all content links that point to the first quarter of the document.
  • the dimension of types is open-ended; new ones can be easily added

4 Useful Properties of Such a System

  • distributed
  • replicated
  • versioned content
  • under control of each user

5 Social Challenges of Such a System

  • content on a local machine, for control, that is lost upon death or seizure
  • permanent identify, for reputation of quality, forced to diverge by authority
  • a focus on literature in a world of non-text communication
  • micropayments without revealing identity of the readers

6 The Building Blocks of Such a System

  • blocksfile
  • coordinates and the mathematical properties
    • ints, floats, tumblers
    • addition, subtraction, within
  • references to coordinates, either linear (range) or grouped (spanlist)
  • trees of many kinds

7 About Links (Actually xanalinks)

  • relations
  • properties
  • structures
  • connections
  • assemblies
  • arrangements

A xanalink tells the front-end software how to present some content. There are many types of xanalinks. A xanalink may be used in more than one edl while retaining its original identity. A xanalink has facets, or connections or endpoints, each with specific meanings.

8 Concepts to Grasp, Vocabulary to Learn

Transliterature
Hypertext
  • bi-directional links
  • version management
  • rights management
  • parallel pages, visibly connected
Transclusion
ZigZag
a new information structure
(no term)
Transcopyright
(no term)
xanalink (not embedded)
(no term)
Tumblers
(no term)
Enfilade

9 Good Quotes by Ted Nelson that Appealed To Me

  • "The good news about computers is that they do what you tell them to do. The bad news is that they do what you tell them to do."
  • "A user interface should be so simple that a beginner in an emergency can understand it within 10 seconds."
  • "If computers are the wave of the future, displays are the surfboards."
  • "the web today is like trying to read while walking thru traffic"
  • "it's reality in the same way jail is reality in that you know there's blue sky and possibilities out there but not within reach"

10 Additional Reading

10.4 Books by Ted Nelson

10.4.2 1982 Literary Machines

10.4.3 1977 Home Computer Revolution

10.4.4 1965 A File Structure for The Complex, The