Not so optimised academic writing workflow...

19 Apr 2019

Recently I have to submit a piece of written work for university and found that there are few improvements that I can made to my current writing workflow (inspired by different peoples out there). Basically I was required to write a research proposal by synthesising many papers and information.

Initially, I thought my current setup is good - but after this week, I don’t think it is optimised.

Disclaimer: This is a record of my personal experience and what I considered as optimised workflow for me. This may not suit everyone and please don’t get offended if I criticise some of your favourite tools below.

My current not-so-optimised setup mostly involved the following two core tools:

• Papers3 (Reference Manager) - I ditched EndNote for its bulkiness and slowness - especially for large repository of papers. Papers3 is one of my favourite - although the software is not free and hasn’t been updated as often. The reason I switched from Mendeley to Papers3 is because an upgrade of Mendeley completely messed up my database. Papers3 seems pretty stable so far. The citation utility is quite handy to search for papers during writing - support multiple styles, including BibTex, Pandoc, MultiMarkdown, etc.

• Scrivener (Writing Software) - This is really really good for writing! After trying different software, and you bet, Microsoft Word, Google Docs, Text Editor with Markdown - this is by far the best. Even though you are not writing university assignments, this tool is suitable for book writers (fiction, non-fiction), thesis, journal articles, etc. It comes with different templates to get you started. Just like Microsoft Word, it comes with APA template and allow me to start with the APA format straightaway. However, this software shines when you realise it is designed for writing - modular writing. You can break up a long section into pieces, and arrange them to suit your need. So you can write bits at a time, then switch, chop as you like. Also, it supports markdown out of the box - that means you can also write LaTex formulas if required. Furthermore, it has this “Compile” functionality where it will compile all your text pieces and format into one single APA compliant Word docx. How impressive.

Problem

I used both of the tools above to write my proposal - thinking that it is good enough - but there are still a few things that I would like to improve.

One of the things that annoyed me is the default Scrivener APA template does not allow you to use Markdown syntax - that means no maths formula - So I have to go back and forth between Words to draw the formula, and copy back to Scrivener. Basically without Markdown, it kind of limit the flexibility.

New Workflow

So after I submitted the proposal, I stuck my head into looking at a partial setup that I left off a few months ago - using Pandoc, Markdown and other tools - and try to build a workflow that can support Markdown, and using Pandoc and few tools - so that I can still write in Scrivener but with Markdown support, then using scripts and pandoc to convert to Markdown - before creating to docx in APA format.

Tools

The new setup is not perfect, and it involves more tools that I expected - could be an issue down the track - but so far, it suits my need. These are the tools:

• Scrivomatic - I was inspired by this guy’s work but there are subtle differences between his requirements and mine. I used this as a starting point and changed a few things.
• Pandoc - You probably familiar with Pandoc where it is a very nice tool to generate between different formats easily and efficiently. If you are writing papers for academic journals, they probably have different style guides and this tool can combine with other style processing tools (LaTeX) to generate a nice looking document.
• Pandocomatic - This is just a tool that make it easier to integrate different configuration in Pandoc. Scrivomatic builds on top of this and provide customisation for Scrivener software.
• Ruby scripts + Paru - These are some custom Ruby scripts that I need to write to use the Paru framework to pre/post-process the Markdown file generated before Pandoc convert to DOCX.
• A .bib file output from Papers3 (for Pandoc)

Workflow

So the basic workflow is like this:

• I use Scrivener and Papers3 to write a paper or an assignment. I structured it so that it contains the core parts - Introduction, Methods, Results, Discussion, etc.
• In each section, I use Papers3 citation tool to insert at appropriate place in the text: such as [@Block:2014es] at the end of the sentence or another fancy way is without the square bracket - @Block:2014es, this will embed in the middle of sentence without round bracket.
• After I am happy and want to compile to DOCX, I use the “Compile…” function (There are a bit to setup in this step to suit my need - including pre/process Ruby scripts + Paru to move Markdown syntax to different places - to match APA style - too much to include here), it will generate the DOCX, and generate all the required references (the citation I used).
• That’s it.

I think this workflow will provide me the flexibility to include partial markdown and partial text, also it includes auto generating references to figures, tables as well - which is a big plus.

After thoughts

So far, I am quite happy with this setup. Although I am yet to find how to customise the “Running head…” header in DOCX.

This worklow relies on quite a number of tools and scripts. If any of this breakdown, I will have to spend time to investigate - which could be anti-productivity. But if it works most of the time, I think it will be a time saver for me.

My next step could investigate how to make the workflow between draft and review feedback much more seamless - at the moment, I will have to upload DOCX to Google Doc for my supervisors, receive feedback, then retrospectively change my original manuscript and re-generate another version. It would be nice if the collaboration can be more seamless.

Until next time :)