After starting my graduate studies last year, I have tried various research tools to search for a suite of tools that would help me to perform my research activities (mainly reading and writing) more efficiently and productively. My requirements of these tools are - able to streamline current activity individually while also work together as a whole between tools, essentially fit into a coherence personal workflow. The word personal is important here. I realised that there is no one-size-fits-all tool, although I guess the tool designer would like to accommodate as many peoples/customers as possible. It was an interesting journey so far….
Parallel processing is not a new concept in computing. With the need to process/analyse massive amount of datasets today efficiently, the idea of parallelism seems like an attractive idea. There are a few goodies in R that I came across previously that not only can help to speed up the processing, but also make you think about the structuring of your code - splitting your code into smaller independent chunks, and these chunks can be executed independently across CPUs or GPUs.
One of the functions of data analysis is to derive meaningful interpretation from the raw data that we collected. This process typically involves data transformation, manipulation, shaping and visualisation. Sometimes, we go through this repeatedly through different processing pipelines.
The more we transform or manipulate the data, the higher probability that we are going to make a mistake somewhere - e.g. an unintentional cartesian join instead of an inner join, coding the empty value incorrectly, incorrect matrix dimensions, mismatch column/row indices etc. If these mistakes were not detected early enough, it would invalidate all the analysis performed after that. It is an incentive to be able to detect these problems earlier. I am going to talk about a way to put some checkpoints into our code to safeguard these mistakes.
Doing statistical analysis in my small research project has taught me a few lessons about how to organise the data. So I have summarised some of what I have learnt here. It might not be the best practice for all, but I would like to think of them as my “good data hygiene”.
Recently I have to submit a piece of written work for university and found that there are few improvements that I can made to my current writing workflow (inspired by different peoples out there). Basically I was required to write a research proposal by synthesising many papers and information.
Initially, I thought my current setup is good - but after this week, I don’t think it is optimised.
According to Wikipedia, Internet of Things (IoT) is a heterogeneous network, formed predominantly by small devices (for e.g. sensors, home appliances, software, etc). This network allows the devices to communicate with each other to achieve some goals.
In a smaller scale and more domestic level, home automation can be seen as a realisation/implementation of IoT. If you are into technology gadgets, you probably won’t miss any of the smart electronics sold in store nowadays - such as the smart lighting solution (Philips hue, Xiaomi, IKEA Tradfri, etc), door/window/motion sensors, wireless door bell (Ring?), and all sorts of different small electronics that make you spent your whole weekend (or few months) in setting up this DIY home automation project.
There are many advantages in using the keyboard solely to perform day to day tasks on a computer or laptop. The main advantage is reducing the switching time between the keyboard and mouse to increase the productivity. Also for the fast typist out there, the keyboard is more convenient and quicker way to perform an action. However, the operating system (OS) nowadays is designed with usability in mind, which means it is in Graphical User Interface (GUI). It is hard to navigate to a specific area of the screen purely using the keyboard. Technically you can do it, by using the “arrow” keys to move the mouse pointers but it is counter-productive and very slow, not to mention the accuracy.
One of my new year resolutions 2019 is to improve the automation of my personal life, including tracking my finances. Fortunately, I have been tracking my expenses over the last 9 months since April 2018 and now I have the data to generate a summary report about my income and expenses, in order to produce a budget for 2019.
I have been using various note organiser tools, such as EverNote, MacOS Notes, Bear, Microsoft OneNotes over the years to keep track of my brainstorm ideas, study notes, project ideas, software commands, deployment instructions and some how-tos.
However, I still feel constrained by the limitation of the tools and unable to do exactly what I want to organise my notes. Here are some of my rants about these tools:
I was searching for any potential platform out there that could render and host Markdown/R Notebook, and stumbled across GitHub Pages. I knew I could create Wiki pages in each GitHub repository but did not realise that it can also hosts personal website using Markdown.
Excited I was after this small discovery, I found another tool Jekyll that works nicely with this GitHub pages ecosystem and cannot wait to try all these exciting tools.
Ultimately, I would like to write notes/blogs on my machine and can just push my changes to a Git repository and my website is updated auto-magically. This is exactly what I am looking for.
The next step could be to see how it can host some R notebooks.
