In September 2020, after a lot of head-scratching, I managed to create a data visualisation app with Python and Streamlit and publish it on the web using Heroku. And I did all this on a cheap old Chromebook, which is not even a ‘proper’ PC!
I thought I’d write about the whole process so that others who wish to try programming and creating data apps won’t be put off by technical jargon and the lack of powerful computing devices.
As I said above, the only computing device I used for the entire process is a 4GB RAM, 32GB local storage Chromebook that I had bought three years ago for under £200. I can install Android apps on it, but I didn’t use any Android apps for this project. A few years ago it would have been much harder, perhaps even unfeasible, to use a Chromebook for a programming project, since they cannot run local desktop applications like a Linux, Mac or Windows PC. But nowadays, with all the improvements in online programming environments and cloud services, you can do quite a lot with Chromebooks. See the section ‘Programming Environment’ below.
For many years there’s been growing interest around the world in sources of renewable energy, so I was interested to see that the World Bank had a dataset on renewable energy consumption as a percentage of total energy consumption over the years. They also have a data explorer page on their website where you can see a visualisation of this data, but I didn’t find it very easy to use. Therefore, I decided to use this dataset and create my own visualisation. I downloaded the CSV file and got started.
(You can see my post ‘Where to Find Data?’ for free sources of economic and financial data that can be mostly freely re-used.)
I naively expected the raw data that I downloaded to already be in a ‘tidy’ format — no blank rows or columns, all column headers specified neatly, and a column for the years. But it wasn’t like that at all, so I had to go through a few steps to clean up the data and put it in a tidy format. There are several online resources on how to use Pandas in Python to clean up data in dataframes. Any time I had a doubt or didn’t know how to proceed, I would just search the web for a solution. I cannot thank enough all the people who take the trouble to ask and answer questions on sites like Stackoverflow, Reddit, Github, YouTube, etc. Most of my programming achievements are due to them.
The Programming Environment
I used Google Colab for all the programming. This is a free online Jupyter notebook environment from Google and does not require installation of Anaconda or anything like that. It is perfect for Chromebook users, and also for those who are just trying out Python and don’t want to install anything yet. You will need to have a Google account though.
Google Colab comes with a lot of libraries such as Pandas, Plotly, etc already installed. For other required libraries, just use
!pip install … inside a coding cell in the Colab notebook to install those libraries (they will be installed only for that instance of that notebook, so you will have to re-run the code every time you open the notebook afresh).
Please note that Google Colab may not necessarily have the latest version of the Python library pre-installed, so you may have to check that the pre-installed version has the capabilities you are looking for, otherwise, install the latest version using
!pip install -U… as mentioned above.
I’ve tried out various visualisation libraries in Python: Matplotlib (of course), Seaborn, Bokeh, Altair and Plotly Express. They are all very good, but personally, I found Plotly Express to be the easiest to use. You may think differently according to your preference.
I wanted to show two types of charts:
- Line charts that show the progress of renewable energy consumption over the years for any country
- A geo chart that shows the levels of renewable energy use for all the countries in any particular year.
I also wanted to provide dropdown options so that viewers can choose which countries they want to view the line charts for, and which years they want to see the geo chart for.
- Write the code
- Push the code to a GitHub repo
- Connect the repo to Heroku and publish the web app
It seems simple, but there was a lot of pain in store for me.
Write the Code
This was the most time-intensive step. I’ve already mentioned how I had to clean up the data to use it. Creating the charts wasn’t very hard, since Plotly Express already has options to easily create line and choropleth (filled map) charts. What was more complicated was to figure out how to let the viewers input their choices for years and countries so that the charts would update accordingly.
The Dash library by Plotly is a great way to build dashboards and interactive charts like the ones I wanted to show. Now you can’t use Dash just like that in a Jupyter or Google Colab notebook, but you can use JupyterDash, which is nothing but the Dash library modified for use in Jupyter notebooks.
One good thing about using JupyterDash is that you can see the interactive chart outputs right away in your Jupyter notebook, even an online one like Google Colab. I was able to achieve exactly what I wanted to show, but I found the syntax to be a bit complicated. You can view my Python notebook in my GitHub repository here. I didn’t use this code for my web app, but I thought it would be useful as a reference.
Compared to JupyterDash, I found the syntax for Streamlit to be a lot simpler. But there was a catch. The code including Streamlit commands is meant to be written in a python file, and then run from the terminal with the command:
streamlit run <filename>.py
This generates a localhost URL which you can view to see the results.
Streamlit isn’t really meant to be used in Google Colab, so it is hard to view the output, especially since Google Colab doesn’t allow you to view the localhost URL. This required a workaround involving Ngrok (you need to register for a free Ngrok account).
The following article contains all the required steps to run Streamlit in Google Colab and then see what your application looks like:
How to Run Streamlit Apps From Colab
The simplicity that streamlit offers for building apps is so cool and useful. But sometimes you may be constrained by…
In this way, I was able to test my code and make sure that all the interactive features worked. Please note that the latest version of the Pyngrok package seems to be creating problems, so it is better to install and use the older Pyngrok version 4.1.1.
Push the Code to a GitHub Repo
GitHub is a very well-known place where people can post their code and share it with others. The free plan allows you to have unlimited public or private ‘repositories’ or ‘repos’, which you can think of as folders containing all the required files for a programming project.
I used to be puzzled by words like ‘push’ and ‘commit’ when used in the context of GitHub, but once I understood that they are like ‘publish’ and ‘save’ respectively, I became much more comfortable.
There is no need to use terminal commands or download anything to your PC. In fact, I wouldn’t have been able to, since I only have a Chromebook. Luckily, you can do a lot with the web interface of GitHub. If you are a new user, go through the introductory tutorial on GitHub where they explain all these things really well.
Online programming environments like Google Colab and Repl.it also allow you to ‘push’ your code directly to your GitHub repository (you have to enable access to your GitHub account).
Simply copying and pasting your code into the GitHub files on the browser can work too, but be mindful of errors.
You can view my repository here. This is the code that I used for my web app. Notice that the main code is in a .py file, not a Jupyter notebook.
Connect the Repo to Heroku and Publish the Web App
Why did I bother to post my code on GitHub? Because GitHub works very well with web app deployment services like Heroku. Their free plan (up to 5 apps) is sufficient for a beginner like me. You have to connect your public GitHub repository to the new app that you’re setting up on Heroku, and enable automatic updates (if you wish), so that changes made to your code on GitHub will be implemented in the published app as well.
The following article explains all the steps — look at item 4 in the table of contents.
Streamlit + Heroku = Magic?
How to easily deploy your streamlit application on Heroku without any headaches.
That’s it! Now I have my fully functioning web app, with a URL that I can give to anyone who wants to view and interact with the charts.
- It is now even easier to deploy Streamlit apps from GitHub public repositories — check out ‘Streamlit Sharing’ here. At present, you have to request an invitation, but I got it pretty quickly. The free plan allows you to publish up to three apps.
- There’s a new free online Jupyter notebook programming environment called Deepnote, and it looks really slick. Another good resource for Chromebook users!