LabVIEW Simple Messaging Reference Library (STM)

Publish Date: Apr 02, 2015

Overview
For engineers who design distributed systems where hosts interact using a network, one of the main challenges is building an
effective communication protocol. The Simple Messaging Reference Library (STM) can enhance the performance, usability,
maintainability and scalability of a distributed system. Unlike raw communications APIs, using formatted packets makes data
manipulation more manageable and improves throughput by minimizing the transmission of repetitive data.

Table of Contents
1. Background
2. Theory of Operation
3. Implementation
4. Simple TCP/IP Messaging API
5. Using the STM API to Send Data
6. Using the STM API to Receive Data
7. Conclusion
8. Download
1. Background
For embedded LabVIEW applications, communication with a remote client is often a critical part of the project. Embedded
applications often function as 'data servers' because their primary role is to report information (status, acquired data, analyzed
data, etc.) to the client. They are also often capable of responding to commands from the client to perform application-specific
activities.
The complexity of these applications is reflected in the network communication requirements. The LabVIEW Real-Time
Communication Wizard provides a solution, but an alternative approach is needed for other embedded targets and for applications
that outgrow the capacity of wizard-based tools.
The TCP/IP protocol is the most common method for sharing information between hosts. The TCP/IP protocol provides a medium
for data sharing, but it is up to the application to implement the logic that optimizes performance and makes sense of the data. For
more sophisticated applications, the following additional functionality is desirable as part of the communication protocol:
Ability to send meta information for ease of data manipulation
Ability to send and receive data with a naming convention
Ability to send and receive many data types (string, I8, array of doubles, clusters, etc.)
This document describes a simple messaging protocol that addresses these requirements.
An installer for the Simple Messaging Reference Library (STM) component, which includes an application programming interface
(API) and example code, can be found at the link you will find at the end of this document. Some of the examples emphasize
LabVIEW Real-Time applications, but most will run on any LabVIEW platform.
This document discusses an approach to LabVIEW network communication using the TCP/IP protocol. STM also supports
UDP and serial as transport layers. You may also want to learn more about Shared Variables, which provide a higher-level
programming interface and abstract most of the implementation details described here.
2. Theory of Operation
The communication protocol should have the following characteristics:
Easily packages and parses data
Hides the transport layer (TCP/IP, UDP, etc.) implementation details
Minimizes network traffic by sending data only when it is needed
Minimizes impact in the overall overhead and throughput
Lends itself to communication with environments other than LabVIEW (C, C++, etc)
In every messaging protocol there is some overhead (meta data) associated with parsing the data stream on the receiving end.
Sending a complete set of meta data with every package adds significant overhead.

Since we are focusing on high-performance applications, we want to minimize the communication overhead by sending a reduced
set of meta information with each packet. The approach used by the STM protocol is to create a separate "meta data" message
containing an indexed list of meta information that is sent once whenever an STM connection is made between hosts. By doing
this, each host has all of the information that it needs to decode subsequent messages. When sending a message, the host
1/4 www.ni.com
containing an indexed list of meta information that is sent once whenever an STM connection is made between hosts. By doing
this, each host has all of the information that it needs to decode subsequent messages. When sending a message, the host
packetizes the data with a meta data index that the receiver can use to look up the decoding information.

You will see that by using this approach we are able to create an abstraction from the transport layer, where the user sends and
receives data by name. The underlying transfer mechanism is completely hidden.
3. Implementation
Now that we have explained the concept, we will discuss the LabVIEW implementation.

The meta data is implemented as an array of clusters. Each array element contains the data properties necessary to package and
decode one variable value. We have only defined the 'Name' property, but using a cluster allows you to customize the STM by
adding meta properties (such as data type) according to your application requirements. The meta data cluster is a typedef, so
adding properties should not break existing code.
The following figure shows an example of the meta data cluster configured for two variables: 'Iteration' and 'RandomData'.

Figure 1. Meta Data Array of Strings

Before each data variable is transmitted, a packet is created that includes fields for the Data Size, the Meta Data ID and the data
itself. Figure 2 shows the packet format.

Figure 2. Packet Format

The Meta Data ID field is populated with the index of the meta data array element corresponding to the data variable. The
receiving host uses the Meta Data ID to index the meta data array to get the properties of the message data. This mechanism is
very effective since it allows for ease of use (read and write to variable names) with minimum overhead.

Since only the Meta Data ID is transferred, notice how the meta data cluster can be extended to include more static variable
properties without impacting the transmission overhead.

The STM protocol is more efficient and has higher throughput when transmitting large data payloads. Each message includes 48
(32 + 16) bits of overhead for the data size and Meta Data ID. A boolean converted to string is represented by 1 character (8 bits).
When a Boolean value is sent, the payload efficiency is 8 bits / (48 + 8) bits, or 14%. When an array of 1000 doubles (1000*64
bits) is sent, however, the payload efficiency is much higher (64,000 bits / 64,048 bits = 99.9%).

When streaming a significant number of individual data values, we recommend aggregating them into an array before sending
them.

4. Simple TCP/IP Messaging API

The STM API is very simple. For basic operation it consists of a Read VI and a Write VI. There are also two supplemental VIs to
help with the transmission of the Meta Data, but their use is not mandatory. Each of the main VIs are polymorphic which allows for
the use with different transport layers. The API for each layer is very similar. The following is a brief description of the main API
VIs (only the TCP/IP versions are shown). Additional utility VIs are installed as well and are documented in the VI Help.

STM Write Message: Use this VI to send any type of data to a remote host. This VI creates a packet based on the data, the data
name and meta data information. When this VI is called, it retrieves the index of the variable specified by Name in the meta data
array. It then assembles the message packet and sends it to the remote host via TCP/IP using the connection ID.
The data must be in string format for transmission. Use the Flatten to String primitive to convert the message data to a string.

The Connection Info consists of a transport layer reference and the meta data array. This is an output from both the STM Read
Meta Data and STM Write Meta Data VIs.

2/4 www.ni.com
 STM Write Message (TCP).vi

STM Read Message: Use this VI to receive any type of data from a remote host. This VI reads and unpacks the meta data index
and flattened string data. It looks up the meta element and returns it along with the data string. The application can then convert
the flattened data to the message data type using the name or other meta properties as a guide. In the example below, the
variable named "RandomData" is always converted to an "Array of Doubles" data type).
This VI is usually used inside a loop. Since there is no guarantee that data will arrive at a given time, use the 'timeout' parameter
to allow the loop to run periodically and use the 'Timed Out?' indicator to know whether to process the returned values.

STM Read Message (TCP).vi

STM Write Meta Data: Use this VI to send meta data information to a remote host. For correct message interpretation, the meta
data must be consistent on both receiving and sending side. Instead of maintaining copies of the data on each host, we
recommend that you maintain the meta data on the server and use this VI to send it to clients as they connect.

STM Write Meta Data (TCP).vi

STM Read Meta Data: Use this VI to receive meta data information from a remote computer. This VI reads and unpacks the meta
data array, which can be passed to the read and write VIs.

STM Read Meta Data (TCP).vi

5. Using the STM API to Send Data
The following figure shows an example of a data server using the STM API. Notice that this program sends the meta data to a
remote host as soon as a connection is established. The example writes two values: the iteration counter and an array of doubles.
The meta data contains the description for these two variables (as shown in Figure 1).
Note that you only need to wire the variable name to the STM Write Message VI, which takes care of creating and sending the
message packet for you. This abstraction allows you to send data by name, while hiding the underlying complexity of the TCP/IP
protocol.
Also note that the application flattens the data to a string before it is sent. For simple data types (including those shown in Figure
3) it is possible to use a typecast, which is slightly faster than the Flatten to String VI. However Flatten to String also works with
complex data types such as clusters and waveforms.

[+] Enlarge Image

Figure 3. Basic STM Server Example
As you can see, the protocol can be customized and expanded to fit your application requirements. As you add variables, simply
add an entry to the meta data array and a corresponding STM Write Message VI for that variable.
6. Using the STM API to Receive Data
Receiving data is also very simple. The design pattern shown in Figure 4 waits for the meta data when the connection is
established with the server. It then uses the STM Read Message VI to wait for incoming messages. When a message is received,
it converts the data and assigns it to a local value according to the meta data name.
The case structure driven by the data name provides an expandable method for handling data conversion. As you add variables,
3/4 www.ni.com
The case structure driven by the data name provides an expandable method for handling data conversion. As you add variables,
simply create a case with the code to convert the variable to the right type and to send it to the right destination.
Note that an outer case structure handles timeout events.

[+] Enlarge Image

Figure 4. Basic STM Client Example
One advantage of this design pattern is that it centralizes the code that receives data and assigns it to local values.
Another advantage is that the STM Read Message VI sleeps until data is received (or a timeout occurs), so the loop is driven at
the rate of incoming data. This guarantees that no data is lost and no CPU time is wasted polling for incoming data.

Note: Since the client has no knowledge of the meta data until run time, the developer must be sure that the application handles
all possible incoming variables. It is a good practice to implement a 'Default' case to trap any 'unknown' variables as an error
condition.
7. Conclusion
The STM protocol can significantly improve productivity and client/server performance. The STM protocol abstracts the transport
layer, which enables you to create intuitive code that is easier to expand and maintain.
The STM reference library is suitable for many client/server applications, but it also provides a foundation for higher-level design
patterns. To read about a design pattern for command-based bidirectional communication, please see the article on
Command-based Communication (http://zone.ni.com/devzone/cda/tut/p/id/3098).
8. Download
You can download the STM (http://zone.ni.com/devzone/cda/epd/p/id/2739) reference library, which includes source code for the
above example here (http://zone.ni.com/devzone/cda/epd/p/id/2739).

PRODUCT SUPPORT COMPANY

Order status and history (http://www.ni.com/status/) Submit a service request ( About National Instruments (http://www.ni.com/compa
https://sine.ni.com/srm/app/myServiceRequests)
Order by part number ( Events (http://www.ni.com/events/)
http://sine.ni.com/apps/utf8/nios.store?action=purchase_form) Manuals (http://www.ni.com/manuals/)
Careers (http://www.ni.com/careers/)
Activate a product ( Drivers (http://www.ni.com/downloads/drivers/)
http://sine.ni.com/myproducts/app/main.xhtml?lang=en)
Alliance Partners (http://www.ni.com/alliance/)
Order and payment information (http://www.ni.com/how-to-buy/)

MISSION
NI equips engineers and scientists with systems that accelerate productivity, innovation, and discovery.

(http://twitter.com/niglobal) (http://www.facebook.com/NationalInstruments)

http://www.linkedin.com/company/3433?trk=tyah) (http://www.ni.com/rss/) (http://www.youtube.com/nationalins

Contact Us (http://www.ni.com/contact-us/)

(//privacy.truste.com/privacy-seal/National-Instruments-Corporation/validation?rid=bc6daa8f-7051-4eea-b7b5-fb24dcd96d95)

Legal (http://www.ni.com/legal/) | © National Instruments. All rights reserved. | Site map (

http://www.ni.com/help/map.htm)

4/4 www.ni.com