Wednesday, January 2, 2019

Dashboard combining workshop notes and terminal

The workshop environment described so far in this series of posts only targeted the problem of providing an in browser interactive terminal session for workshop attendees. This approach was initially taken because we already had a separate tool called workshopper for hosting and displaying workshop notes.

I didn't want to try and modify the existing workshopper tool as it was implemented in a programming language I wasn't familiar with, plus starting with it would have meant I would have had to extend it to know about user authentication, as well as add the capability to spawn terminals for each user which would be embedded in the workshopper page.

In the interests of trying not to complicate things and create too much work for myself, I therefore went the path of keeping the per user terminal instances completely separate from the existing tool. Now that I have deployment of terminals for multiple users working using JupyterHub as the spawner, what can be done about also hosting the workshop notes in a combined dashboard view.

Self contained workshop in a container

There are a couple of different approaches I have seen used to create these dashboard views which combine workshop notes with a terminal.

The first way, is to have a single common web site which provides the workshop notes. A user would visit this web site, going through a login page if necessary. Once they have been assigned an identifier, a backend instance for the terminal would be dynamically created. The common web site would embed the view for that terminal in the dashboard for that user, along with the workshop notes.

The second way, has the common web site only be a launchpad, with it dynamically creating a backend instance which hosts a web application which provides both the workshop notes and the terminal. That is, each user has their own instance of the application hosting the workshop notes, rather than it being provided by a single frontend application.

In both cases, depending on what the training system was for, the backend could have used a virtual machine (or set of virtual machines) for each user, or it could have used containers hosted on some container platform.

Whatever they use, these were always hosted services and the only way to do the workshop, was to go to their service.

When we run workshops, we don't have an always running site or infrastructue where someone wanting to do the workshop could go. We would host the workshop notes and terminals in the one OpenShift cluster where workshop attendees are also running the exercises for the workshop. When the workshop was over, the OpenShift cluster would be destroyed.

At the end of the workshop, we would inevitably be asked, how can I go through the workshop later, or can I share the workshop with a colleague so they can do it. Because it was an ephemeral system, this wasn't really possible.

One of the goals I had in mind in coming up with what I did, was the idea that everything you needed was self contained, with the container image being the packaging mechanism. So although we might run a multi user workshop and use JupyterHub as a means of spawning each users environment with the interactive terminal, you weren't dependent on having JupyterHub.

You have already seen this play out where I showed how you could deploy the terminal image standalone in an OpenShift cluster of your own, without needing JupyterHub. The only thing missing right now is including the workshop notes and way to view it in that same image.

The deployment model used is therefore sort of similar to the second option above, except that you only get a container for your terminal instance as a container in Kubernetes, you aren't being given a complete set of VMs or cluster of your own. When you run the exercises, you would be working in the same cluster as the terminal was deployed, and the same cluster that other workshop attendees were using.

This approach works well with OpenShift, because unlike plain out of the box Kubernetes, OpenShift enforces multi tenancy and isolation between projects and users. If necessary, you can further lock down users and what they can do through quotas and additional role based access control, although the default access controls are usually sufficient.

Because of the multi tenancy features of OpenShift, it is quite reasonable goal therefore, to package up everything for a workshop in a container image, and allow them to pull down that image later and deploy it into their own OpenShift cluster, knowing that it should work.

As well as using an image to deliver up a self contained workshop as an image which can be deployed in any cluster, it could also prove useful for packaging up the instructions, and anything else required, for when demonstrating how to deploy applications in OpenShift to customers.

Image implementing a combined dashboard view

In creating a combined dashboard view, because a standalone terminal was still required for use with the existing workshopper tool, addition of a dashboard was done using an image of its own. Rather than this being completely from scratch, the base terminal image was designed in a way that it could be extended, with plugins being able to be registered with the proxy to add additional route handlers.

This way if it was decided to stick with the workshopper tool for the workshop notes, just the terminal base image would be used. If a combined dashboard view with integrated workshop notes was more appropriate, the dashboard image which extends the terminal image would be used.

To see how the dashboard view would look, you can deploy the dashboard image using:

$ oc new-app https://raw.githubusercontent.com/openshift-labs/workshop-terminal/develop/templates/production.json \
    --param TERMINAL_IMAGE=quay.io/openshiftlabs/workshop-dashboard:latest

This uses the same approach before of relying on the OpenShift cluster to perform user authentication, and only an admin of the project can access it. Once user authentication has been completed, a user is redirected back to the dashboard.

img-alternative-text

Although the dashboard image doesn't have any pre-loaded content for a specific workshop yet, you can see how on the left you have the workshop notes. On the right you have a pair of terminals.

In the workshop notes, you can have one or more pages, with pages grouped into sub directories as categories if necessary. You can then define how the pages are chained together so users are guided from one page to the next as they perform the exercises.

The content for the workshop notes can be specified using markdown formatting. In the case of code blocks, these can be optionally annotated so that if the code block is clicked on, it will automatically be copied to a terminal on the right hand side and executed. This saves the user needing to type the commands. A code block can also be annotated so that if clicked on, the content will be copied into the browser copy buffer. This can then be pasted into the terminal or other web page.

Embedding a view for workshop notes

In the dashboard view, the Butterfly application is still used for the embedded terminal view. As Butterfly supports multiple sessions from the one instance, it was a simple matter of referring to a different named session in each of the terminal panes.

For the workshop notes, like with Butterfly, I preferred to not write a half baked implementation of my own, so I opted to use a knowledge base application called Raneto which fitted the requirements I had. These were that markdown could be used, it supported variable substitution, supported page meta data, and allowed for the theme and layout to be overriden in templates and CSS.

Raneto even has some features that I am not using just yet which could prove useful down the track. One of those is that it is possible to edit content through Raneto. Unfortunately that feature doesn't work properly when hosting Raneto at a sub URL, but I have a pull request submitted to the author which fixes that, so hopefully editing can be allowed at some point for when you are authoring content.

In order to combine the workshop notes hosted by Raneto, and the terminal panes implemented using Butterfly, I had to write some custom code, specifically the dashboard framing, with adjustable sizing of panels. For this part, I relied on the fact that the proxy had been made pluggable and overrode the default page handler to redirect to the dashboard view. I then used PUG in Node.js express to create the layout.

People will know me as a Python fan boy and this was really the first time using Javascript, Node.js and express on anything serious. I must say it felt a bit liberating, although I had to rely on a lot of googling, and cutting and pasting of Javascript code snippets until I got it doing what I wanted. It could still do with a bit more polish and tweaking as there are still a couple of issues I haven't been able to solve due to a lack of understanding of Javascript and the DOM page rendering model.

Coming up next, adding workshop notes

In addition to being able to run up the dashboard standalone, you can also deploy it using JupyterHub for a multi user workshop environment. The next step therefore is to add your own workshop notes.

This is done in the same was as when extending the terminal to add additional command line tools, or source files, including the ability to be able to add a build script to run your own steps during the build phase. That is, using a Source-to-Image (S2I) build, or a build from a Dockerfile. The result is the same, a self contained container image, but this time adding the workshop notes and the way of displaying them in a single dashboard view.

In the next post I will explain how you would layout your content for your workshop notes, how to markup code blocks so they can be clicked on to be executed, and how to link pages together to create the path users should follow when working through the exercises.