Tutorial overview
This tutorial demonstrates the use case how to develop an application for UCW Platform. The DHT Logger is an example how to implement a data logger for DHTx sensors for UCW Portal. In the tutorial you may use knowledge from basic tutorials - UCW Portal Hello World, UCW Portal Dashboard and Gadget, and UCW Portal Report (source code).
Requirements
Step 01: Download tutorial sources
To download tutorial sources and test their functionality, do as follows:
- Make sure you have downloaded all the requirements above.
- Open a terminal/console and navigate to the directory where you want to work on the tutorial.
Download the sources using the following command:
git clone https://github.com/unitycloudware/dht-logger.git dht-logger
Move into a newly created folder
cd dht-logger
Test downloaded files by building and running the sources. To do so, execute following commands
Unix / Linux based OS./build.sh ./run-portal.sh
or
Windowsbuild.bat run-portal.bat
The application should be up and running on http://localhost:9601 (TODO - add a link to setting tomcat port section). After clicking the link, you should see a login page. If you see the login page, you can proceed to the next step.
Log into the application using default credentials (User: admin, Password: admin). (TODO - add a link to credential settings)
Step 02: Dashboard
In this step, you will explore the various methods and files in organising and displaying received data. The data collected from sensors and transmitted to the UCW platform are displayed on a dashboard. Each dashboard consists of one or more gadgets. The nsys-plugin.xml file handles the creation of these dashboards and gadgets.
Creating a dashboard
Dashboard configuration<dashboard key="ucw-dhtlogger_dashboard" name="UCW DHT Logger Dashboard"> <description>This dashboard displays temperature and humidity data collected from DHTx sensors.</description> <label>DHT Logger</label> <viewId>ucw-dhtlogger</viewId> <imageUri>${portalResourcesUrl}/resources/images/ucw_logo.png</imageUri> <actionButtons>ucw.dhtlogger.dashboard.header.actions</actionButtons> </dashboard>
Note
Each dashboard has a unique key, name and viewId
Any gadget to be displayed on this dashboard MUST have the same view as this viewId
Step 03: Temperature and humidity gadgets
Adding gadgets to a dashboard using the nsys-plugin.xml file described earlier.
Dashboard gadgets configuration<dashboard-gadget key="ucw-dhtlogger_temperature-data-gadget" name="Temperature Data Gadget" class="com.unitycloudware.portal.tutorial.dhtlogger.gadget.TemperatureDataGadget"> <description>Displays temperature data from DHTx sensors.</description> <label>Temperature Measurement</label> <column>left</column> <order>0</order> <view>ucw-dhtlogger</view> </dashboard-gadget> <dashboard-gadget key="ucw-dhtlogger_humidity-data-gadget" name="Humidity Data Gadget" class="com.unitycloudware.portal.tutorial.dhtlogger.gadget.HumidityDataGadget"> <description>Displays humidity data from DHTx sensors.</description> <label>Humidity Measurement</label> <column>right</column> <order>0</order> <view>ucw-dhtlogger</view> </dashboard-gadget> <dashboard-gadget key="ucw-dhtlogger_temperature-humidity-data-gadget" name="Temperature and Humidity Data Gadget" class="com.unitycloudware.portal.tutorial.dhtlogger.gadget.TemperatureHumidityGadget"> <description>Displays temperature and humidity data from DHTx sensors.</description> <label>Temperature and Humidity Measurement</label> <column>right</column> <order>0</order> <view>ucw-dhtlogger</view> </dashboard-gadget>
Note
Each gadget has a unique key and name
The column specifies the position of the gadget on the dashboard
Each gadget has a Java file specified under the class attribute, that manages the manner in which data is collected and displayed. The TemperatureDataGadget.java, HumidityDataGadget.java and TemperatureHumidityGadget.java files manage the Temperature and Humidity gadgets respectively.
SensorData model represents data that are sent from the device to the data stream ucw-dhtlogger. It represents JSON in following structure
SensorData model as JSON{"temperature": 23.50, "humidity": 26.30, "heatIndex": 22.59}
The gadget uses a data model to display information about temperature, humidity, and heat index.
Template file: The gadget displays data in a tabular form. This table is generated by a Velocity template (.vm file). Snippets for both temperature and humidity gadgets are shown below
temperature-data.vm#parse ("macros.vm") #if ($!{data.isEmpty()}) #showMessage("info", "Status", "No data to display.", false) #else <table class="aui aui-table-interactive aui-table-sortable"> <thead> <tr> <th id="data-temperature">Temperature</th> <th id="data-timestamp">Timestamp</th> </tr> </thead> <tbody> #foreach ($d in $data) <tr> <td headers="data-temperature"><span class="aui-lozenge aui-lozenge-current">$String.format("%.2f", $!{d.temperature})°C</span></td> <td headers="data-timestamp">$date.format("yyyy-MM-dd HH:mm:ss", $!{d.timestamp})</td> </tr> #end </tbody> </table> #end
humidity-data.vm#parse ("macros.vm") #if ($!{data.isEmpty()}) #showMessage("info", "Status", "No data to display.", false) #else <table class="aui aui-table-interactive aui-table-sortable"> <thead> <tr> <th id="data-humidity">Humidity</th> <th id="data-timestamp">Timestamp</th> </tr> </thead> <tbody> #foreach ($d in $data) <tr> <td headers="data-humidity"><span class="aui-lozenge aui-lozenge-current">$String.format("%.2f", $!{d.humidity})%</span></td> <td headers="data-timestamp">$date.format("yyyy-MM-dd HH:mm:ss", $!{d.timestamp})</td> </tr> #end </tbody> </table> #end
temperature-humidity-data.vm#parse ("macros.vm") #if ($!{data.isEmpty()}) #showMessage("info", "Status", "No data to display.", false) #else <table class="aui aui-table-interactive aui-table-sortable"> <thead> <tr> <th id="data-temperature">Temperature</th> <th id="data-humidity">Humidity</th> <th id="data-heatindex">Heat Index</th> <th id="data-timestamp">Timestamp</th> </tr> </thead> <tbody> #foreach ($d in $data) <tr> <td headers="data-temperature"><span class="aui-lozenge aui-lozenge-current">$String.format("%.2f", $!{d.temperature})°C</span></td> <td headers="data-humidity"><span class="aui-lozenge aui-lozenge-current">$String.format("%.2f", $!{d.humidity})%</span></td> <td headers="data-heatindex"><span class="aui-lozenge aui-lozenge-current">$String.format("%.2f", $!{d.heatIndex})°C</span></td> <td headers="data-timestamp">$date.format("yyyy-MM-dd HH:mm:ss", $!{d.timestamp})</td> </tr> #end </tbody> </table> #end
The template file allows the user to add, delete or change the columns displayed on the gadget.
Test Data Utils: Each datastream received at the UCW server, consisting of a deviceId and payload, saved as device and data stream respectively, is stored under a project for efficient recording purposes. The TestDataUtils.java file creates a project, device, and data if any does not exist already. To create any of these datastream components, you first need to create a project manager, device manager, and data manager (if none already exists) respectively
private ProjectManager projectManager; private DeviceManager deviceManager; private DataManager dataManager; public ProjectManager getProjectManager() { if (projectManager == null) { projectManager = ComponentProvider.getInstance().getComponent(ProjectManager.class); } return projectManager; } public DeviceManager getDeviceManager() { if (deviceManager == null) { deviceManager = ComponentProvider.getInstance().getComponent(DeviceManager.class); } return deviceManager; } public DataManager getDataManager() { if (dataManager == null) { dataManager = ComponentProvider.getInstance().getComponent(DataManager.class); } return dataManager; }
then you can proceed to create a project, device, and data
protected void createProject() { log.debugFormat("Creating project '%s' with key '%s'...", PROJECT_NAME, PROJECT_KEY); if (getProjectManager().getProjectByKey(PROJECT_KEY) != null) { return; } Project project = new Project(); project.setKey(PROJECT_KEY); project.setName(PROJECT_NAME); project.setDescription(PROJECT_DESC); if (getProjectManager().addProject(project) != null) { getProjectManager().configureProject(PROJECT_KEY); } } protected void createDevice() { log.debugFormat("Creating device '%s' for project '%s'...", DEVICE_NAME, PROJECT_KEY); Project project = getProjectManager().getProjectByKey(PROJECT_KEY); if (project == null || getDeviceManager().getDeviceByName(PROJECT_KEY, DEVICE_NAME) != null) { return; } Device device = getDeviceManager().createDevice( DEVICE_NAME, DEVICE_DESC, DeviceType.GENERIC.name(), 120, PROJECT_KEY); getDeviceManager().addDevice(device); } protected void createDataStream() { log.debugFormat("Creating data stream '%s' for project '%s'...", DATA_STREAM_NAME, PROJECT_KEY); Project project = getProjectManager().getProjectByKey(PROJECT_KEY); if (project == null || getDataManager().getDataStream(PROJECT_KEY, DATA_STREAM_NAME) != null) { return; } DataStream dataStream = new DataStream(); dataStream.setName(DATA_STREAM_NAME); dataStream.setDescription(DATA_STREAM_DESC); dataStream.setProject(project); dataStream.setType(DataStreamType.DATA_MESSAGE); dataStream.setStorageType(StorageType.GENERIC.name()); dataStream.setEnabled(true); getDataManager().addDataStream(dataStream); }
Note
Each project has a unique key and name
A project can comprise of different data streams
- Gadget files: TemperatureDataGadget.java, HumidityDataGadget.java and TemperatureHumidityGadget.java files perform the following functions:
Sets the Velocity files as templates for the gadgets
TemperatureDataGadget.javapublic static final String TEMPLATE = "/templates/gadget/temperature-data.vm"; public TemperatureDataGadget() { setTemplate(TEMPLATE); }
HumidityDataGadget.javapublic static final String TEMPLATE = "/templates/gadget/humidity-data.vm"; public HumidityDataGadget() { setTemplate(TEMPLATE); }
TemperatureHumidityGadget.javapublic static final String TEMPLATE = "/templates/gadget/temperature-humidity-data.vm"; public TemperatureHumidityGadget() { setTemplate(TEMPLATE); }
- Data to be displayed in the gadget
- Fetches the device and data stream created by the TestDataUtils.java file described in # 4 above using device name, data stream name and project key as its search criteria.
A determined sample size of data is specified to be displayed on the gadget at a time. In this example, 10 data samples are displayed at a time.
The received sensor data is converted from a JSON string to a SensorData object
Method getData()protected List<SensorData> getData() { List<SensorData> data = new ArrayList<SensorData>(); Device device = getDeviceManager().getDeviceByName(TestDataUtils.PROJECT_KEY, TestDataUtils.DEVICE_NAME); if (device == null) { return data; } DataStream dataStream = getDataManager().getDataStream(TestDataUtils.PROJECT_KEY, TestDataUtils.DATA_STREAM_NAME); if (dataStream == null) { return data; } // Load 10 last records List<DataStreamItem> items = getDataManager().loadStream(dataStream, device, 0, 10); if (items == null || items.isEmpty()) { return data; } for (DataStreamItem item : items) { // Process only data for data stream with type of DATA_MESSAGE if (item.getType() != DataStreamType.DATA_MESSAGE) { continue; } DataMessage dataMessage = (DataMessage) item.getData(); // Transform JSON payload to SensorData object SensorData sensorData = JsonUtils.fromJson(dataMessage.getData(), SensorData.class); if (sensorData.getTimestamp() == 0) { sensorData.setTimestamp(dataMessage.getTimestamp()); } data.add(sensorData); } return data; }
Creates a Velocity template context that takes two arguments: a string and list containing SensorData data. This is the data that is being read into the table of the Velocity template file
Create Velocity template context@Override protected Map<String, Object> createVelocityParams(final Map<String, Object> context) { Map<String, Object> velocityParams = new HashMap<String, Object>(); velocityParams.put("data", getData()); return velocityParams; }
Step 04: Logger scheme gadget
This gadget displays the diagrammatic representation of the DHT22 sensor connection to Adafruit Feather M0 micro-controller. No data is displayed in this gadget.
Template file of this gadget indicates the location/path to the image displayed to the gadget. This is indicated in <img src = .... />. This path can be accessed through the download button as it is linked to the path of the image as indicated by href="$!{portalResourcesAttachmentUrl}/resources/images/dht22-logger.jpg".
logger-scheme.vm<br/> <img src="$!{portalResourcesUrl}/resources/images/dht22-logger.jpg" width="100%" alt="DHT22 logger scheme"/><br/> <br/> <a id="$!{gadget.key}_download-scheme" href="$!{portalResourcesAttachmentUrl}/resources/images/dht22-logger.jpg" class="aui-button" title="Download DHT22 logger scheme"> <span>Download scheme</span> </a><br/> <br/>
Gadget file simply sets the Velocity file as the gadget template and creates a Velocity context
LoggerSchemeGadget .javapackage com.unitycloudware.portal.tutorial.dhtlogger.gadget; import java.util.HashMap; import java.util.Map; import org.nsys.portal.gadget.AbstractGadget; public class LoggerSchemeGadget extends AbstractGadget { public static final String TEMPLATE = "/templates/gadget/logger-scheme.vm"; public LoggerSchemeGadget() { setTemplate(TEMPLATE); } @Override protected Map<String, Object> createVelocityParams(final Map<String, Object> context) { Map<String, Object> velocityParams = new HashMap<String, Object>(); return velocityParams; } }
Step 05: Sensor data generator
In the absence of real data from the DHT sensor, test data can be used to display on the dashboard. This test data is generated using SensorDataGeneratorJob.java file.
- Each job has to implement method execute(final Map<String, Object> jobDataMap) that is the entry point of the job and is called every cycle when the job runs.
- execute(final Map<String, Object> jobDataMap)
public void execute(final Map<String, Object> jobDataMap) { getLog().info("Executing sensor data generator..."); if (!jobDataMap.containsKey(ComponentName.DEVICE_MANAGER)) { getLog().error("Unable to find DeviceManager component in jobDataMap!"); return; } if (!jobDataMap.containsKey(ComponentName.DATA_MANAGER)) { getLog().error("Unable to find DataManager component in jobDataMap!"); return; } setDeviceManager((DeviceManager) jobDataMap.get(ComponentName.DEVICE_MANAGER)); setDataManager((DataManager) jobDataMap.get(ComponentName.DATA_MANAGER)); generateData(); }
Method generateData() generates random numbers of data type double, between 0 and 100 for both temperature and humidity. It does this by:
Gets a reference to the device and data stream created by TestDataUtils
Creating a SensorData object and passing the randomly created temperature, humidity, and timestamp values as an argument for a create method
Transforming SensorData object to JSON string (payload)
Storing payload to the data stream for device created by TestDataUtils
generateData()public void generateData() { Device device = getDeviceManager().getDeviceByName(TestDataUtils.PROJECT_KEY, TestDataUtils.DEVICE_NAME); if (device == null) { return; } DataStream dataStream = getDataManager().getDataStream(TestDataUtils.PROJECT_KEY, TestDataUtils.DATA_STREAM_NAME); if (dataStream == null) { return; } SensorData data = SensorData.create( RandomRange.getRandomDouble(0, 100), RandomRange.getRandomDouble(0, 100), TimeUtils.getNow().getTime()); String payload = JsonUtils.toJson(data); getLog().debugFormat("Storing generated sensor data... Payload: %s", payload); getDataManager().storeStream(dataStream, device, payload); }
Step 06: UCW Portal plugin
The DHTLoggerPlugin.java file handles the job scheduling of the SensorDataGeneratorJob.java.
Method scheduleJobs() creates a HashMap pairing a device manager and data manager with device manager and data manager objects respectively
scheduleJobs()protected void scheduleJobs() { // When the sensor data generator is disabled then skip adding job to scheduler. if (!DHTLoggerConfig.isGeneratorEnabled()) { return; } SchedulerService scheduler = ServiceProvider.getInstance().getServiceHost(SchedulerService.class); //Date delay1min = TimeUtils.addMinutes(TimeUtils.getNow(), 1); //long repeatInterval = (600 * 1000) * 2; // 2mins Date delay30sec = TimeUtils.addSeconds(TimeUtils.getNow(), 30); long repeatInterval = 10 * 1000; // 10sec Map<String, Object> jobDataMap = new HashMap<String, Object>(); jobDataMap.put(ComponentName.DEVICE_MANAGER, ComponentProvider.getInstance().getComponent(DeviceManager.class)); jobDataMap.put(ComponentName.DATA_MANAGER, ComponentProvider.getInstance().getComponent(DataManager.class)); scheduler.scheduleJob(SensorDataGeneratorJob.class, jobDataMap, delay30sec, repeatInterval); }
Method createTestData() creates a project, device and data stream (if non already exists) using the createData() method from TestDataUtils.java
createTestData()protected void createTestData() { TestDataUtils testDataUtils = new TestDataUtils(); testDataUtils.createData(); }
Step 07: Reporting
Temperature and humidity data displayed on the dashboard can be displayed as a graph for reporting purposes. Clicking the "DHT Logger" button on the navigation bar provides options for plotting temperature or humidity data as shown in the figure below.
In this example, you employed a line graph to represent temperature and humidity data through the procedure below:
Template File: #3 of Step03 details the functions of template files. A similar file is used to save temperature and humidity data.
template file: report - temperature$portalResourceManager.requireResourcesForContext("nsys.portal.chart") <div class="chart-center"> #foreach ($chart in $charts) <div class="aui-group"> <div class="aui-item"> $!{chart} </div> </div> #end <table class="aui aui-table-interactive aui-table-sortable"> <thead> <tr> <th id="data-temperature">Temperature</th> <th id="data-timestamp">Timestamp</th> </tr> </thead> <tbody> #foreach ($d in $data) <tr> <td headers="data-temperature"><span class="aui-lozenge aui-lozenge-current">$!{d.temperature}%</span></td> <td headers="data-timestamp">$date.format("yyyy-MM-dd HH:mm:ss", $!{d.timestamp})</td> </tr> #end </tbody> </table> <br/><br/><br/> </div>
template file: report - humidity$portalResourceManager.requireResourcesForContext("nsys.portal.chart") <div class="chart-center"> #foreach ($chart in $charts) <div class="aui-group"> <div class="aui-item"> $!{chart} </div> </div> #end <table class="aui aui-table-interactive aui-table-sortable"> <thead> <tr> <th id="data-humidity">Humidity</th> <th id="data-timestamp">Timestamp</th> </tr> </thead> <tbody> #foreach ($d in $data) <tr> <td headers="data-humidity"><span class="aui-lozenge aui-lozenge-current">$!{d.humidity}%</span></td> <td headers="data-timestamp">$date.format("yyyy-MM-dd HH:mm:ss", $!{d.timestamp})</td> </tr> #end </tbody> </table> <br/><br/><br/> </div>
Controller: TemperatureReport and HumidityReport java files provide methods that support setting chart parameters, data generation, and displaying reports. The various sections of the controller files are explained below:
a. Each file takes a list, cachedData, in which the elements are of data-type SensorData
private List<SensorData> cachedData;
b. sets the configuration for the chart
chart configurationpublic static final String REPORT_NAME = "UCW DHT logger Report - Temperature"; public static final String REPORT_TEMPLATE = "/templates/report/temperature-report.vm"; public static final String CHART_TITLE = "UCW Report - Temperature Chart (%)"; public static final int CHART_WIDTH = 640; public static final int CHART_HEIGHT = 480; public static final String CHART_DATA_URL = "/ucw-dhtlogger/temperature/chart-data"; @Override protected void configure() { setReportName(REPORT_NAME); setReportImageUrl("${portalResourcesUrl}/resources/images/ucw_logo.png"); setReportTemplate(REPORT_TEMPLATE); setChartTitle(CHART_TITLE); setChartType(ChartConfig.ChartType.LINE); setChartWidth(CHART_WIDTH); setChartHeight(CHART_HEIGHT); setChartDataUrl(CHART_DATA_URL); setChartLegend(true); setChartStack(false); setChartHoverDetail(true); setChartAxisXMode(ChartConfig.AxisXMode.TIME_SERIES); setChartAxisXTimeUnit(ChartConfig.TimeUnit.HOUR_2); }
c. showReport method assigns values to cacheData list if none already exists and creates a velocity context for the template file
showReport method@RequestMapping(method = RequestMethod.GET) @Override public ModelAndView showReport(final HttpServletRequest request, final HttpServletResponse response) { if (cachedData == null) { cachedData = getData(); } Map<String, Object> context = new HashMap<String, Object>(); context.put("data", cachedData); return render(context, request, response); }
d. getChartData method creates a list of data-type ChartSeries.This list holds the sensor data in the format to be used for making charts.This is done by calling the method getTemperatureData and getHumidityData for temperature and humidity plots respectively.
getChartData - temperature@RequestMapping(value = "/chart-data", method = RequestMethod.GET) @ResponseBody public List<ChartSeries> getChartData( final HttpServletRequest request, final HttpServletResponse response) { ChartData data = new ChartData(); //for plotting temperature readings data.addSeries(getTemperatureData("temperatureData", "DHT22 Sensor", ChartConfig.ChartColor.BLUE, cachedData)); return data.getSeries(); }
getChartData - humidity@RequestMapping(value = "/chart-data", method = RequestMethod.GET) @ResponseBody public List<ChartSeries> getChartData( final HttpServletRequest request, final HttpServletResponse response) { ChartData data = new ChartData(); data.addSeries(getHumidityData("humidityData", "DHT22 Sensor", ChartConfig.ChartColor.GREEN, cachedData)); return data.getSeries(); }
e. getTemperatureData method creates a lists of data-type ChartSeries. It contains only temperature and timestamp data from the cacheData list
getTemperatureData()protected ChartSeries getTemperatureData(final String key, final String name, final String color, final List<SensorData> data1) { ChartSeries temperatureData = new ChartSeries(); temperatureData.setKey(key); temperatureData.setName(name); temperatureData.setColor(color); for (SensorData td : data1) { temperatureData.addPoint(ChartPoint.<Long, Integer>create(td.getTimestamp() / 1000, (int) (td.getTemperature()))); } return temperatureData; }
f. getHumidityData method does the same function as a getTemperatureData method. It returns a list containing pairs of humidity data and timestamp
getHumidityData()protected ChartSeries getHumidityData(final String key, final String name, final String color, final List<SensorData> data1) { ChartSeries humidityData = new ChartSeries(); humidityData.setKey(key); humidityData.setName(name); humidityData.setColor(color); for (SensorData hd : data1) { humidityData.addPoint(ChartPoint.<Long, Integer>create(hd.getTimestamp() / 1000, (int)(hd.getHumidity()))); } return humidityData; }
Note that the name argument in this function is used in the legend of the chart to label each line-plot while color refers to the desired colour of the plot
g. getData method simply generates a list of data-type SensorData for cachedData, if none already exists
getData()protected List<SensorData> getData() { List<SensorData> sensorDataList = new ArrayList<SensorData>(); Date date = TimeUtils.getStartOfDay(TimeUtils.getNow()); for (int i = 0; i < 24; i++) { int temp = RandomRange.getRandomInt(1, 100); int hum = RandomRange.getRandomInt(1, 100); long timestamp = date.getTime() + TimeUtils.getTimezoneUTCAndDSTOffset(date); sensorDataList.add(SensorData.create(temp, hum, timestamp)); date = TimeUtils.addHours(date, 1); } return sensorDataList; }
Result
You can build and run the application, and the resulting page should look like this
When you click on Temperature Report or Humidity Report, you should get any of these charts