Data visualization

Introduction

Visualization of data from EnergyDataDK can help you determine whether the data has gaps, assess the quality of the data, be utilized to monitor whether datastreams fall outside of tolerances, for demonstration purposes, ect.

EnergyDataDK has built-in functionality that helps you do this; Simply select a dataset you have viewing rights to, select a datastream, and enter a time range to get a preview. This will help you assess whether this time period has data, and whether there are any gaps.
You can find detailed instructions in our Platform Guide.

You can additionally use our APIs to extract data and use a variety of options to present the data visually.
Below we will provide instructions, examples, and perhaps even some inspiration.

If you encounter any errors in this document or have suggestions for improvement, please contact us at EnergyDataDK@DTU.DK.

Visualization applications

There are a number of 3^rd party visualization applications available which allow you to generate graphs based on either historical, or live data.

Many data visualizing applications can be used free of charge, are relatively easy to use, and you can usually get started within just a few minutes.

Grafana

Grafana is an open-source analytics and visualization web application. It connects to time series databases and other data sources, allowing users to build dashboards that display metrics, logs, and traces.

Wikipedia

Grafana is our preferred data visualization tool. It’s a powerful and easy to use Open Source solution, which means you can use Grafana for free. You can find it here.

Installing Grafana is relatively straight forward, so for the purpose of this guide we’ll assume you’ll be able to get started with Grafana by following the steps involved with the registration process.

In this guide we’ll keep things relatively simple, but as mentioned above, Grafana is offers a wide variety of features. If you’re interested in learning more about the solution, we refer you to their documentation page.

Live data monitoring

In this example we’ll connect Grafana to a datastream by subscribing to it using the EnergyDataDK MQTT broker.

Please ensure the following conditions are met:

You have access to the datastreams which you want to visualize.
You have an API token which grants “Connect to mqtt and read / write data in realtime” permission for the datasets which contain the datastreams you want to visualize.

The MQTT data source plugin

First and foremost, you’ll need to install the “MQTT data source plugin“. Follow the link to learn more, or continue below to install.

Step 1: Select Administration → Plugins and data → Plugins from the main menu on the left.

Step 2: In the Search field enter “MQTT” and click the MQTT tile.

Step 3: Click the Install button in the upper right corner.

Establish the connection

Next, we need to establish the connection to the datastream.

Step 1: Select Connections → Data sources from the main menu on the left and click the Add new data source button in the upper right corner.

See screenshot

Step 2: In the search field enter “MQTT” and click on the MQTT tile.

See screenshot

Step 3: You can now configure the data source. To do this, follow these steps:

Under the Connection section, in the URI field, enter “mqtts://mqtts.energydata.dk:8883”.
Under the Authentication section, enter your API key in the Username field.
Leave the Client ID and Password fields empty.
Press the Save and test button.

You should get a “MQTT connected” message. If you have an invalid API key, you’ll get a “error connecting to MQTT broker: not Authorized” message instead.

See screenshot

Create a dashboard

Now that we have a connection, it’s time to actually visualize the datastream.

Step 1: Select Dashboards from the main menu on the left, press the New button in the upper right corner, and select the New dashboard option.

See screenshot

Step 2: Enter “MQTT” in the search field and select the MQTT tile.

See screenshot

Step 3: Click or drag a pane from the menu on the right to the grid in the middle of the screen.

See screenshot

Step 4: Press the Configure visualization button.

See screenshot

Step 5: Select the grafana-mqtt-datasource option from the Data source field.

See screenshot

Step 6: In the Topic field, enter your full MQTT topic (both the prefix and the suffix), for example: dataset/datastream-collection/datastream. You should now see a graph generating.

See screenshot

Option step: You may notice that the x-axis displays Unix timestamps, which doesn’t make much sense to us, so we’ll need to convert this to something we can understand. You can achieve this as follows:

In the menu on the left, under X axis, in the X field, select timestamp.
Near the bottom of the screen, on the left side, switch to the Transformations tab.
Select timestamp and Time in the two input fields.

After making making your changes and modifications, press the Save button to save your panel, and press the Back to dashboard button. You should now see normal time and date on the x-axis.

See screenshot

You can add additional panels by repeating the steps above for each datastream you want to monitor.

Custom visualizations using API

You can use EnergyDataDK’s API to create your own custom visualization. These can be thing like graphs, diagrams, or dashboards.
Below we will demonstrate examples of visualizations utilizing the EnergyDataDK API.

Graphs generator

The Graph generator essentially allows you to upload a file which describes the dataset you want to visualize from EnergyDataDK, after which you choose which datastreams you want to generate graphs for, and for which period.

Graphs will be generated for each of the selected datastreams for that period.

The application is made entirely with client-side scripting (JavaScript). It constructs a fetch command based on the user’s input and uses this information to construct graphs using jsGraph.

Try it now

Instructions

Requirements:

An API key that grants “Fetch data from datasets via API” rights.
You can find instructions on how to create an API token here.
A definitions file.
Find the dataset you want to visualize as described here. Click on the icon, select Download, and Export to .CSV

Steps:

Start by pressing the button and selecting the definitions file. A list of datastreams should be displayed on the left side of the screen.
Select the datastreams you want to visualize.
Enter your API key.
Select start- and end dates to specify the period you want to visualize.
Press the button.

Upon completion of these steps you should see the graphs after the data has completely been fetched.

Dashboards

A dashboard generally is just to demonstrate a state of affairs. Therefor, you would generally subscribe to a MQTT broker to continuously receive the latest values.
Alternatively you would set an interval at which you collect the newest value.

Below are examples of dashboards using the EnergyDataDK API.