As a new chapter begins for Manuscripts as a collaborative word processing tool, we thought it would be interesting to give a brief account of where Manuscripts as a product and software project comes from. What has been learned as part of building it also tells a lot about where we are heading now.
This project began during the summer of 2011 as a side effect of me writing my dissertation whilst intensely growing a beard. Thesis writing crystallised a need for a fresh take on word processing, with a scholarly story telling focus: allow the author to think in scholarly terms of citations, cross-references, author lists, math, etc, semiautomatically formatting the document for the author. It is of course not a new idea, with LaTeX (and LyX) closest inspirations as enablers of such a workflow in the academic context, albeit with a typesetting macro expansion language based approach.
This workflow concept, and it enabling an author to solve problems with a writing tool beyond just formatting a document, continued to circle in my head, and in fact has ever since remained rather recognisable from the early product mockups.
The extreme ironing begins
Ideas began eventually to turn into design and code, with Alex Griekspoor co-founding a company with me, and financing it, to develop the product. This project began to go everywhere with me. To demonstrate the early years of Manuscripts, I can offer no better visual depiction than the picture below. My capacity to whip up a laptop at any given point in time and space to keep working on this program began to be called jokingly “extreme coding” by a certain loved one, in reference to “extreme ironing”.
As you can probably read on my face in some of the pictures above, a fresh take on word processing turned out to be rather a complex undertaking, especially for a 1.5 head development team (me + my brother).
Manuscripts for Mac ships
We shipped Manuscripts eventually as a Mac app, after a good ~4 year effort.
For an intro video of the app in the form it was released at, check out this YouTube video. A ton of interest, positive feedback and a well used tool had resulted from our efforts, but it became quickly clear that we had reached the limits of what we could execute given the resources and the 1.0 technical solution – “extreme coding” would not help us out of this one:
- Collaboration: although we knew how to tackle collaborative writing, we could not plausibly introduce collaboration by 1.0. A lot of fundamental design and technical decisions and implementation effort had gone towards allowing for realtime collaboration, but implementation would have to wait a bigger team.
- Platforms: we had built a beautiful Mac app, but not everyone is a Mac user, and even Mac users need to work with non-Mac users. Again, a lot of preparation was done in early stages of the project to prepare for additional platform support, but actually tackling it would not be plausible amongst us two.
- Reliability: the core word processing environment, with wide interoperability with Word, LaTeX, Markdown formats, with a new rich text based model was in many ways a success and a great piece of engineering, but building it also involved a big set of features, again bringing us to our limits as a tiny team.
We had certainly reaching our limits as a small team in being able to take the product to its full potential… and then we were approached by Atypon.
Building a sustainable foundation
Atypon’s acquisition of Manuscripts two years ago, allowing us to open source the codebase, and to build a dedicated development team of half a dozen developers + quality assurance, designers etc, has entirely transformed our ability to build a great product in the last two years, allowing us to take time to rebuild it with sustainable (not extreme!) processes. Our focus has been on addressing the shortcomings we had learned were there from the 1.0:
- ✅Collaborative workflows. The new editing environment has been rebuilt, no stone unturned, for realtime collaboration.
- ✅Platform support for Windows and Linux beside macOS: we rebuilt Manuscripts starting from the browser client to allow for Windows and Linux based authors to also collaborate, and to create the foundations for an editing experience also for a new version of the native desktop Mac app. The Mac app part of this work is going to still take a moment for this to reach its 1.0, but you can observe our work on it already, since it’s all open source.
- ✅A highly modular design: to illustrate, there are 20 modules on npmjs.com, allowing us to reuse and replace parts of the system freely.
✅Good form testing practices to all critical parts of the system.
To give some examples of the parts of the system that are being tested automatically to a high degree:
- The file format specification and its compatibility with the import and export export system.
- The synchronisation system that is responsible for ensuring the edited project stays consistent after updates come in from multiple editors.
- Backend services such as the API server and the Pressroom document conversion service.
- Many user interface components.
To put this in more visual terms: the new Manuscripts.io technology solution has allowed us to reach some new grounds in scientific word processing, such as integrating Jupyter backed visualisations directly inside the the editor… and to add commenting and highlighting in a matter of months into the editor.
The collaborative re-incarnation of Manuscripts is built to enable research collaboration in new ways. We will focus on improving the existing experience based on feedback we are receiving from users. Beside that, in the coming months, you will see us focus on building on our new collaborative strengths:
- Manuscripts 2 for Mac… and iOS (!)
**The new editor environment was designed to be housed not just in an offline capable web app, but we have designed it to be embeddable in a new Mac and iOS native client that is being developed openly (check out the source code), like the rest of the system.
- Do more with projects: assignees, deadlines, status tracking for manuscripts and their parts.
- Embed more of your science in your writing: link code, data and visualisations in powerful new ways. Deeper Jupyter integration and client-side rendered dynamic visualisations.
- Create and crowd-source document templates: the document templating system in Manuscripts is one of its unique strengths. Wouldn’t it be great if you could create your own templates, and share them with other users?
- Versioning: make suggested edits, improved annotation functionality, version history, verifiably versioned documents.
As with everything regarding this project, we are very open to feedback. Please come chat with us on Slack if you have questions or ideas!