Working with Python Expression Tags

Python Expression Tags in Historian

A Python® expression is typically a single line of Python code which, when executed, evaluates to a single value. It can also be thought of as the right-hand side of a Python assignment operation.

Python Expression Tags are used in cases where you do not want to store a raw data value, but wish to store only derived or calculated values. Historian allows raw data to be pre-processed with Python expressions during collection, so that the data collected for the expression tags contain these derived values.

See www.python.org/ for complete Python documentation.

Note: This functionality is available only for the Enterprise edition of Historian.

To use a Python Expression Tag in Historian, you must configure a tag as a Python Expression Tag using either Historian File collector or Historian Excel Add-in. You can then use the Python expression on this tag to pre-process raw data before the result is inserted into Historian.

The following collectors support Python Expression Tags:

Simulation Data Collector
iFix Data Collector
OPC Data Collector
OSI PI Collector
Windows Performance Collector
Collector Toolkit

To enable Python Expression support for toolkit-based collectors, copy the following files to the collector executable path:

python34.dll
ihcpprest140_2_8.dll.dll
PythonExpressionExtension.dll

For 32-bit collectors, these files are located at C:\Program Files\Proficy\Proficy Historian\x86\Server. For 64-bit collectors, these files are located at C:\Program Files\Proficy\Proficy Historian\x64\Server.

Supported Python Modules

This table lists the available Python modules. Within the approved modules below, certain Python classes, functions, and keywords may not be available for use, for security reasons. For example, the exec function cannot be used.

Note: Python modules not listed in the table are not supported.


Standard Python Modules	Documentation	Needs Importing
`builtins`	Standard Python documentation	No
`datetime`	Standard Python documentation	Yes
`math`	Standard Python documentation	Yes
`statistics`	Standard Python documentation	Yes
Module part of Historian install `uom` (unit of measure conversion module)	Unit of Measure Python Module.pdf (This document can also be found in the help directory for your Historian installation.)	Yes

Constructing and Adding Python Expression Tags

About this task

To configure and use the Python Expression Tag in Historian:

Procedure

Construct a Python Expression.
Construct the JSON configuration.
Add the Python Expression Tag to Historian.
For examples, see the Python Expression Tag Examples.

Constructing a Python Expression

About this task

Procedure

Open a text editor.
Construct an expression in Python that takes raw data and produces a calculated value you want to store in Historian.
- The raw data in your expression is exposed in the Python script as an object with the following properties: value, is_quality_good, is_quality_bad, timestamp
- You can give this object any name you want.
Ensure that the expression you create is a valid Python expression (that is, it evaluates to a single value).
For example, we call our object temp and we use the value property. We create a simple, valid Python expression, taking raw data (temp.value) and create a calculated value.
```
temp.value + math.pow(10,temp.value/70)
```
To use this example, temp.value must be defined and the Python math module must be imported.

Constructing the JSON Configuration

About this task

Construct a JSON configuration object that contains the Python expression in the same text editor. This allows you to specify Python modules that you require and to link the variables (parameters) used in the expression to a raw data source.

It is easier to edit your JSON in the Historian Excel Add-In, which you will use later to add the tag to Historian, see Adding a Python Expression Tag to Historian.

The JSON needs to be “minified” and it needs to conform to a particular structure. Example of JSON containing a Python Expression in "Minified" format

{"imports":["math"],"script":"temp.value + 
math.pow(10,temp.value/70)","parameters":[{"name":"temp","source":{"add

Here is an example of JSON structure in “Beautified” format. The highlighted entries are placeholders.

{
"imports":["<imported_module0>", "<imported_module1>", 
"<imported_module2>",...], "script":"<python_expression>",
"parameters":[
{
"name":"<variable_name>", "source":{"address":"<source_address>",
"dataType":"<datatype_of_parameter_value_field>"}
},...
]
}

After "imports", list the Python modules you need to import for your Python expression. See Supported Python Modules.

After "script", enter the python expression that you created in Constructing a Python Expression.

Provide values for the following parameters:


Parameter	Description
name	The variable name you gave to the object representing the raw data which will be pre-processed by the expression you created. This variable is exposed in the Python script as an object with the following properties: `value`, `is_quality_good`, `is_quality_bad`, `timestamp`.
address	The source address used by the collector to access the raw data. This parameter is stipulated in the context of the chosen collector, which must be on the list of collectors supporting Python Expression Tags.
dataType	The datatype of the `\value` field for the variable representing the raw data. This allows the Python expression to know the data type on which it is operating. The datatype options are: `SingleFloat, DoubleFloat, SingleInteger, DoubleInteger, QuadInteger, UnsignedSingleInteger, UnsignedDoubleInteger, UnsignedQuadInteger, FixedString, VariableString, Byte, Boolean`. These are the same Historian data types that are supported by the File collector.

Example of JSON containing a Python Expression in “Beautified” format

This example uses the Simulation Collector. Your collector might use a different source address.

{
 "imports":["math"],
 "script":"temp.value+math.pow(10,temp.value/70)",
 "parameters":[{"name":"temp","source":
{"address":"Simulation00001","dataType":"SingleFloat" } } ]
}

It is important to check that your JSON is valid, since no validation will be performed on the JSON at tag creation.

Minified JSON

Once you have constructed this JSON, you need to format it as a “minified” string, so that it can be processed in later steps. Minified JSON has no newline characters or comments. There are tools which can help you minify JSON.

For our example, the minified JSON would look like this:

{"imports":["math"],"script":"temp.value + math.pow(10,temp.value/70)","parameters":
[{"name":"temp","source":{"address":"Simulation00001","dataType":"SingleFloat"}}]}

Pay attention to escape characters in your JSON. If your JSON contains a \ character, you need to escape it. So, \ becomes \\ (since \ is used to escape another \).

Adding a Python Expression Tag to Historian

About this task

You can now add a Python Expression Tag to Historian. Ensure the following for your Python Expression Tag:

CalcType is set to "PythonExpr".
SourceAddress contains the JSON configuration.

Adding the Python Expression Tag can be done using Historian File collector or Historian Excel Add-in in the same way that you would add a regular tag.

For more information on adding tags, refer to Historian File collector or Historian Excel Add-in.

File collector (especially the CSV File Formats and XML File Formats sections)
If you add your tag to the Historian via the File collector and using the CSV format, you must enclose the JSON in quotation marks (") to satisfy CSV requirements for a column value containing commas (,).
This means that you will also need to escape any quotation marks (") in the JSON. That is, " becomes "" (as " is used to escape another ").
Using the Historian Excel Add-in
The Historian Excel Add-In has the advantage of being easiest to use for editing your JSON.

Example of Adding a Tag Using the File collector

For this example, we choose to add a Python Expression tag to the Historian using the File collector to import a CSV file.

The file contents look like this:

[Tag]
Tagname,CollectorName,CalcType,SourceAddress,DataType,Description
ExampleTag,SimulationCollector,PythonExpr,
"{""imports"":[""math""],""script"":""temp.value + 
math.pow(10,temp.value/70)"",""parameters"":[{""name"":""temp"",""source"":{""address"":""Simulation00001"",""dataType"":""SingleFloat""}}]}",
SingleFloat,Python Expression Tag example

Note the following:

The CalcType header is included and set to PythonExpr.
The Source Address is set to the minified JSON created in the previous step.
The CollectorName is set to SimulationCollector, which is a Simulation Collector. This collector is on the list of collectors supporting Python Expression Tags. Your collector might be called by a different name.
The quotation marks within the JSON string are escaped with other quotation marks in the CSV file.
For more information, refer to File collector > CSV file format.

Viewing the Python Expression Tag

About this task

When viewing a Python Expression Tag using Historian Administrator, note that:

The Source Address field contains the full applicable JSON configuration, which includes an indication of the source address. Changing this is not recommended.
The Browse button for configuring the Source Address is disabled for Python Expression Tags.

Python Expression Tag Examples

This section provides examples for using Python® Expression Tags.

Using No Python Modules or Functions

In this example, we want to perform gross error detection on a signal "Signal" and clip the values to a range between 0 and 600.

Expression

To solve this, we create the following Python expression.


                    0 if Signal.value<0 else (600 if Signal.value>600 else Signal.value)


	Python Datatype	Name	Description
Expression Inputs	`SingleFloat`	`Signal`	Represents the signal value.
Expression Result	`SingleFloat`	Not Applicable	Represents the resulting expression value, with extreme outliers clipped.

Python Modules to Import for this Expression: None. (Only modules contained on the list of supported modules are available for this expression.)

Constructing the JSON:

Using the created expression, we construct the following JSON. For more information, refer to Constructing the JSON Configuration.

"script":"0 if Signal.value<0 else (600 if Signal.value>600 else Signal.value)", "parameters":[
                    {
                    "name":" Signal",
                    "source":{"address":"Simulation00001", "dataType":"SingleFloat"}
                    }
                    ]
                    }

Note that the address parameter is stipulated in the context of the chosen collector, which must be on the list of collectors supporting Python Expression Tags.

In this example, we have used the Simulation Collector. Your collector might use a different source address.

We then transform the above into a minified JSON string:

{"script":"0 if Signal.value < 0 else (600 if Signal.value > 600 else Signal.value)","parameters":
                    [{"name":"Signal","source":{"address":"Simulation00001",dataType":"SingleFloat"}}]}

Adding the Expression Tag to Historian

For this example, we choose to add a Python Expression tag to the Historian using the File collector to import a CSV file. (We could also have added the tag via the Historian Excel Add-In.)

The file contents look like this:

[Tag]
                    Tagname,CollectorName,CalcType,SourceAddress,DataType,DescriptionGEDSignalTag,SimulationCollector,PythonExpr,
                    "{""script"":""0 if Signal.value < 0 else (600 if Signal.value > 600 else Signal.value)"",""parameters"":
                    [{""name"":""Signal"",""source"":{""address"":""Simulation00001"",""dataType"":""SingleFloat""}}]}",
                    SingleFloat,Python Expression Tag example

Note the following:

The CalcType header is included and set to PythonExpr.
The Source Address is set to the minified JSON created in the previous step.
The CollectorName is set to SimulationCollector, which is a Simulation Collector. Your collector might be called by a different name.
The quotation marks within the JSON string are escaped with other quotation marks.

We then import the file, following the instructions specified in File collector.

Using a Bulit-In Python Function

In this example, we want to calculate the maximum of two temperature values to be collected.

Expression

To solve this, we create the following Python expression:

max(ThermocoupleA.value, ThermocoupleB.value)


	Python Datatype	Name	Description
Expression Inputs	`SingleFloat`	`ThermocoupleA`	Represents a temperature value.
	`SingleFloat`	`ThermocoupleB`	Represents a temperature value.
Expression Result	`SingleFloat`	Not Applicable	Represents the maximum of the two given temperature values

Python Modules to Import for this Expression: None. A built-in Python module from the Python Standard Library is used. (Only modules contained on the list of supported modules are available to this expression.)

Constructing the JSON

Using the created expression, we construct the following JSON:

{
 "script":"max(ThermocoupleA.value,ThermocoupleB.value)",
 "parameters":[
  {   
  "name":"ThermocoupleA",
  "source":{"address":"Simulation00001","dataType":"SingleFloat"}
  },
  {   
  "name":"ThermocoupleB",
  "source":{"address":"Simulation00002","dataType":"SingleFloat"}
  }
 ]
}

Note that the address parameter is stipulated in the context of the chosen collector, which must be on the list of collectors supporting Python Expression Tags. In this example, we have used the Simulation Collector. Your collector might use a different source address.

We then transform the above into a minified JSON string:

{"script":"max(ThermocoupleA.value,ThermocoupleB.value)","parameters":[{"name":"ThermocoupleA","source":
{"address":"Simulation00001","dataType":"SingleFloat"}},{"name":"ThermocoupleB","source":
{"address":"Simulation00002","dataType":"SingleFloat"}}]}

Adding the Expression Tag to Historian

For this example, we choose to add a Python Expression tag to the Historian using the Historian Excel Add-In. (We could also have added the tag by using via the File collector to import a CSV file. )

Ensure the following for your Python Expression Tag:

The CalcType is set to PythonExpr.
The SourceAddress contains the JSON configuration.
The CollectorName is set to the name of the chosen collector, which must be on the list of collectors supporting Python Expression Tags. Your collector might be called by a different name.
The quotation marks within the JSON string are escaped with other quotation marks in the CSV file.

Otherwise, the tag is added in exactly the same way as for a regular tag.

Using A Python Standard Library Module

In this example we want to calculate a result based on a specific time range for an expression input. We set a supply voltage to zero within prescribed time ranges.

Expression

To solve this, we create the following Python expression:

0 if (SupplyVoltage.timestamp.astimezone().time() &gt;= datetime.time(18) and 
SupplyVoltage.timestamp.astimezone().time() &lt;= datetime.time(20, 30)) else 
SupplyVoltage.value


	Python Datatype	Name	Description
Expression Inputs	`SingleFloat`	`SupplyVoltage`	Represents the value we wish to transform.
Expression Result	`datetime`	Not Applicable	Represents the resulting supply voltage, set to zero in the prescribed time ranges.

Python Modules to Import for this Expression: datetime module. This module is shipped with Historian.

Constructing the JSON

Using the created expression, we construct the following JSON:

{
 "imports":["datetime"],
 "script":"0 if (SupplyVoltage.timestamp.astimezone().time() >= datetime.time(18) and 
SupplyVoltage.timestamp.astimezone().time() <= datetime.time(20, 30)) else 
SupplyVoltage.value","parameters":[
  {   
  "name":" SupplyVoltage",
  "source":{"address":"Simulation00001", "dataType":"SingleFloat"}
  }
 ]
}

We then transform the above into a minified JSON string:

{"imports":["datetime"],"script":"0 if (SupplyVoltage.timestamp.astimezone().time() >= datetime.time(18) and 
SupplyVoltage.timestamp.astimezone().time() <= datetime.time(20, 30)) else 
SupplyVoltage.value","parameters":[{"name":"SupplyVoltage","source":
{"address":"Simulation00001","dataType":"SingleFloat"}}]}

Adding the Expression Tag to Historian

For this example, we choose to add a Python Expression tag to the Historian using the Historian Excel Add-In. (We could also have added the tag via the File collector to import a CSV file.)

Ensure the following for your Python Expression Tag:

The CalcType is set to PythonExpr.
The SourceAddress contains the JSON configuration.
The CollectorName is set to the name of the chosen collector, which must be on the list of collectors supporting Python Expression Tags. Your collector might be called by a different name.

Otherwise, the tag is added in exactly the same way as for a regular tag.

Using A Python Standard Library Module

In this example we want to calculate a result based on a specific time range for an expression input. We set a supply voltage to zero within prescribed time ranges.

Expression

To solve this, we create the following Python expression:

0 if (SupplyVoltage.timestamp.astimezone().time() &gt;= datetime.time(18) and 
SupplyVoltage.timestamp.astimezone().time() &lt;= datetime.time(20, 30)) else 
SupplyVoltage.value


	Python Datatype	Name	Description
Expression Inputs	`SingleFloat`	`SupplyVoltage`	Represents the value we wish to transform.
Expression Result	`datetime`	Not Applicable	Represents the resulting supply voltage, set to zero in the prescribed time ranges.

Python Modules to Import for this Expression: datetime module. This module is shipped with Historian.

Constructing the JSON

Using the created expression, we construct the following JSON:

{
 "imports":["datetime"],
 "script":"0 if (SupplyVoltage.timestamp.astimezone().time() >= datetime.time(18) and 
SupplyVoltage.timestamp.astimezone().time() <= datetime.time(20, 30)) else 
SupplyVoltage.value","parameters":[
  {   
  "name":" SupplyVoltage",
  "source":{"address":"Simulation00001", "dataType":"SingleFloat"}
  }
 ]
}

We then transform the above into a minified JSON string:

{"imports":["datetime"],"script":"0 if (SupplyVoltage.timestamp.astimezone().time() >= datetime.time(18) and 
SupplyVoltage.timestamp.astimezone().time() <= datetime.time(20, 30)) else 
SupplyVoltage.value","parameters":[{"name":"SupplyVoltage","source":
{"address":"Simulation00001","dataType":"SingleFloat"}}]}

Adding the Expression Tag to Historian

For this example, we choose to add a Python Expression tag to the Historian using the Historian Excel Add-In. (We could also have added the tag via the File collector to import a CSV file.)

Ensure the following for your Python Expression Tag:

The CalcType is set to PythonExpr.
The SourceAddress contains the JSON configuration.
The CollectorName is set to the name of the chosen collector, which must be on the list of collectors supporting Python Expression Tags. Your collector might be called by a different name.

Otherwise, the tag is added in exactly the same way as for a regular tag.

Using A Historian Python Module

In this example we want to convert a temperature value from Fahrenheit to Celsius.

Expression

To solve this, we create the following Python expression:

uom.to_Celsius(Thermocouple.value, uom.Temperature.Fahrenheit)


	Python Datatype	Name	Description
Expression Inputs	`SingleFloat`	`Thermocouple`	Represents the temperature value in degrees Fahrenheit
Expression Result	`SingleFloat`	Not Applicable	Represents the temperature value in degrees Celsius

Python Modules to Import for this Expression: uom module. This Python module is shipped with Historian.

(Only modules contained on the list of supported modules are available to this expression.)

Constructing the JSON

Using the created expression, we construct the following JSON:

{
 "imports":["uom"],
 "script":"uom.to_Celsius(Thermocouple.value, uom.Temperature.Fahrenheit)",
 "parameters":[
  {   
  "name":" Thermocouple",
  "source":{"address":"Simulation00001", "dataType":"SingleFloat"}
  }
 ]
}

Note that the address parameter is stipulated in the context of the chosen collector supporting Python Expression Tags. In this example, we have used the Simulation Collector. Your collector might use a different source address.

We then transform the above into a minified JSON string:

{"imports":["uom"],"script":"uom.to_Celsius(Thermocouple.value,uom.Temperature.Fahrenheit)","parameters":
[{"name":"Thermocouple","source":{"address":"Simulation00001","dataType":"SingleFloat"}}]}

Adding the Expression Tag to Historian

For this example, we choose to add a Python Expression tag to the Historian using the File collector to import a CSV file. (We could also have added the tag via the Historian Excel Add-In.)

The file contents look like this:

[Tag]
Tagname,CollectorName,CalcType,SourceAddress,DataType,DescriptionConvertedTempTag,SimulationCollector,PythonExpr,
"{""imports"":[""uom""],""script"":""uom.to_Celsius(Thermocouple.value,uom.Temperature.Fahrenheit)"",
""parameters"":[{""name"":""Thermocouple"",""source"":{""address"":""Simulation00001"",""dataType"":
""SingleFloat""}}]}",
SingleFloat,Python Expression Tag example
Note the following: The CalcType header is included and set to PythonExpr.

Note the following:

The CalcType header is included and is set to PythonExpr.
The SourceAddress contains the JSON configuration.
The CollectorName is set to the name of the chosen collector, which must be on the list of collectors supporting Python Expression Tags. Your collector might be called by a different name.
The quotation marks within the JSON string are escaped with other quotation marks in the CSV file.
For more information, see File collector > CSV File Formats.

We then import the file, following the instructions in the File collector section.

Using Array/Table Lookup

In this example we want to translate a string representing order of magnitude into a corresponding numerical value using array/table lookup.

This example will be explained by means of a hypothetical collector called PlantSensorCollector that is a Python Expression enabled collector. The collector collects a source tag with address TemperatureSetpoint of type VariableString, having values 'Low', 'Medium', and 'High'.

Expression

To solve this, we create the following Python expression:

{'Low':100, 'Medium':400, 'High':800}.get(Setpoint.value, 0)


	Python Datatype	Name	Description
Expression Inputs	`VariableString`	`Setpoint`	Represents the given ordinal string value we wish to transform.
Expression Result	`SingleFloat`	Not Applicable	Represents the numerical value corresponding to the ordinal string input.

Python Modules to Import for this Expression: None. (Only modules contained on the list of supported modules are available to this expression.)

Constructing the JSON: Using the created expression, we construct the following JSON:

{
                 "script":"{'Low':100,'Medium':400,'High':800}.get(Setpoint.value, 0)",
                 "parameters":[
                  {   
                  "name":" Setpoint",
                  "source":{"address":"TemperatureSetpoint", "dataType":"VariableString"}
                  }
                 ]
                }

We then transform the above into a minified JSON string:

{"script":"{'Low':100,'Medium':400,'High':800}.get(Setpoint.value,0)","parameters":
                [{"name":"Setpoint","source":{"address":"TemperatureSetpoint","dataType":"VariableString"}}]}

Adding the Expression Tag to Historian: For this example, we choose to add a Python Expression tag to the Historian using the File collector to import a CSV file. (We could also have added the tag via the Historian Excel Add-In.)

The file contents look like this:

[Tag]Tagname,CollectorName,CalcType,SourceAddress,DataType,DescriptionNumericalTagDerivedFromOrdinalVal,PlantSensorCollector,PythonExpr,
                "{""script"":""{'Low':100,'Medium':400,'High':800}.get(Setpoint.value,0)"",""parameters"":
                [{""name"":""Setpoint"",""source"":{""address"":""TemperatureSetpoint"",""dataType"":""VariableString""}}]}",
                VariableString, Python Expression Tag example

Note the following:

The CalcType is set to PythonExpr.
The SourceAddress contains the JSON configuration.
The CollectorName is set to the name of the chosen collector, which must be on the list of collectors supporting Python Expression Tags. Your collector might be called by a different name.
The quotation marks within the JSON string are escaped with other quotation marks in the CSV file.
For more information, see File collector > CSV File Formats.

We then import the file, following the instructions specified in File collector.