Retrieving Data from Historian

Important: You do not have the latest version of Historian! You are missing out on the newest capabilities and enhanced security. For information on all the latest features, see the Historian product page. For more information on upgrades, contact your GE Digital sales agent or e-mail GE Digital Sales Support. For the most up-to-date documentation, go here.

About Retrieving Data from Historian

After data collection, the Historian Server compresses and stores the information in a Data Archive or a *.iha file. Any client application can retrieve archived data through the Historian API. The Historian API is a client/server programming interface that maintains connectivity to the Historian Server and provides functions for data storage and retrieval in a distributed network environment.

You can retrieve data from Historian using any number of clients, including but not limited to:
  • Historian Analysis
  • Knowledge Center
  • iFIX
  • CIMPLICITY
  • Real-Time Information Portal
  • Dream Reports
  • Excel Add-In
  • Custom SDK Applications
  • OLE DB

Historian exposes various sampling and calculation modes that are used on retrieval of data that has already been collected to the archive. These modes do not effect data collection. Some sampling modes are suited to compressed data and should be used when collector compression or archive compression is used.

Sampling Modes

Sampling modes are used to specify how the data will be retrieved from Historian. Several modes are available, such as CurrentValue, Interpolated, Calculated and RawByTime. Sampling modes are specified in the client you use to retrieve data from Historian.

For more information, refer to the Advanced Topics section in the online help.

Sampling Mode Results
CurrentValue Retrieves the most recent data sample value received by the archiver, of any data quality. This does not necessarily produce the most recent raw data sample, as archive compression may not have stored the most recent raw sample.

The sample has a time stamp, a value, and a quality. The time stamp returned is not typically the current time; it is the time stamp as sent by the collector. If you have a slow poll rate or if collector compression is enabled, the time stamp may be much earlier than the current time.

RawByTime Returns all raw samples of all qualities with a time stamp greater than a specified start time, and less than or equal to a specified end time. The RawByTime sampling mode will not return a sample equal to the start time.
RawByNumber Returns a specified number of samples of all qualities with a time stamp greater than or equal to the start time. The RawByNumber sampling mode will return a sample with a time stamp equal to the start time if one exists

You must also specify a direction and number of samples when using this sampling mode.

Interpolated When archive or collector compression is used, a minimal number of actual data samples are stored in the archive. When this data is retrieved, interpolation can be performed to create an evenly spaced list of most likely real-world values (since the actual values have been removed from the archive through the compression algorithm).

The Interpolated sampling mode is also useful for data samples that haven't undergone archive compression. For example, you may want to plot data along an interval that doesn't match the collected raw samples. Using the Interpolated sampling mode would give you the most likely real-world values for the specified period.

Typically, you use the interpolated sampling mode when data is not collected on a set time schedule, or if you want to see the results returned in an interval that is slower than the collection rate. For example, these instances show when you can use interpolated mode to make evenly spaced values:
  • A tag is collected as unsolicited. In this case, we do not know what the time interval is between collected values.
  • The dead band or archive compression for a tag results in unevenly spaced collection intervals.
  • A tag is collected once per 8-hour shift, but you want to see it displayed in one hour intervals with a linear slope between points.
InterpolatedtoRaw When you request interpolated data, you specify an interval or number of samples. If the actual stored number of raw samples is greater than required, you will get interpolated data as described above. If the actual number of stored samples are less than the required, then you will get the raw samples. In this way, the needs of trending detail and application load are balanced.

This mode is best used when querying compressed data because the Data Archiver can switch to the more efficient raw data query.

Lab The Lab sampling mode only returns the collected values, without any interpolation of the value. The collected value is repeated for each interval until there is a change in the raw data sample's value.

Lab sampling is most often used to create a step chart rather than a smooth curve.

Use Lab sampling instead of interpolated if you only want true collected values returned. The Lab sampling mode is generally not useful on highly compressed data. Use interpolated sampling instead.

LabtoRaw LabtoRaw is an extension to Lab mode of sampling and similar to InterpolatedtoRaw mode where you will be switched to raw data or lab when the actual data samples are fewer than the requested samples.
Trend The Trend sampling mode was designed to produce maximum performance when retrieving data for plotting, particularly over long time periods.

The trend sampling mode returns the maximum and minimum value collected during each interval. When plotted, this makes it possible to display an accurate representation of the data that wont miss any extrema, by only retrieving a minimum of points.

For example, a trend of one year of data with a one-day interval will return 730 values consisting of the 365 minimums and 365 maximums for each day of the year.

TrendtoRaw The TrendtoRaw sampling mode almost always produces the same results as the Trend sampling mode. The exception is that, when more samples are requested than there are raw data points, the TrendtoRaw sampling mode returns all of the available raw data points with no further processing.

TrendtoRaw is used when the number of actual data samples are fewer than the requested number of samples. In that case, TrendtoRaw retrieves raw data in a given interval (between a selected raw minimum and raw maximum).

Trend2 The Trend2 sampling mode is a modified version of the Trend sampling mode.

Trend2 sampling splits up a given time period into a number of intervals (using either a specified number of samples or specified interval length), and returns the minimum and maximum data values that occur within the range of each interval, together with the timestamps of the raw values.

The key differences between Trend and Trend2 sampling modes are in:
  • How they treat a sampling period that does not evenly divide by the interval length:
    • For the Trend sampling mode, Historian ignores any leftover values at the end, rather than putting them into a smaller interval.
    • For the Trend2 sampling mode, Historian creates as many intervals of the interval length as will fit into the sampling period, and then creates a remainder interval from whatever time is left.
  • Spacing of timestamps returned:
    • For the Trend sampling mode, Historian returns evenly-spaced interval timestamps.
    • For the Trend2 sampling mode, Historian returns raw sample timestamps. These timestamps can be unevenly spaced, since raw data can be unevenly spaced.
  • Inclusion of start and end times entered:
    • The Trend sampling mode is start time exclusive and end time inclusive.
    • The Trend2 sampling mode is start time inclusive and end time inclusive.

Trend sampling mode is more suitable for plotting applications that prefer evenly-spaced data.

Trend2 sampling mode is more suitable for analysis of mins and maxes and for plotting programs that can handle unevenly spaced data.

TrendtoRaw2 The TrendtoRaw2 sampling mode is a modified version of the TrendtoRaw sampling mode.

The TrendtoRaw2 sampling mode almost always produces the same results as the Trend2 sampling mode. The exception is that, when more samples are requested than there are raw data points, the TrendtoRaw2 sampling mode returns all of the available raw data points with no further processing.

Calculated Returns samples based on a selected Calculation mode. Refer to Calculation Modes for more information.
RawByFilterToggle RawByFilterToggle returns filtered time ranges. The values returned are 0 and 1. If the value is 1, then the condition is true and 0 means false.

This sampling mode is used with the time range and filter tag conditions. The result starts with a starting time stamp and ends with an ending timestamp

Calculation Modes

Calculation modes are used when the sampling mode is set to Calculated. The data type of all calculated values will be DoubleFloat except for MinimumTime, MaximumTime, FirstRawTime and LastRawTime which will be a Date. The datatype of the values of FirstRawValue and LastRawValue will be the same as that of the selected tag.

Calculation Mode Results
Count Displays the number of raw samples in the specified interval. This only indicates the count and does not display the actual values or qualities of the samples.

The Count calculation mode is useful for analyzing the distribution of raw data samples. If you have a higher number of raw samples than expected, you may decide to implement collector or archive compression. If samples are missing, then you may want to slow your collection rates.

State Count Displays the number of times a tag has transitioned to another state from a previous state. A state transition is counted when the previous good sample is not equal to the state value and the next good sample is equal to state value.
State Time Displays the duration that a tag was in a given state within an interval.
Minimum Displays the minimum value in a specified interval with good data quality. This value may be raw or interpolated.
Note: The Minimum and MinimumTime calculation retrieve two additional samples per interval; one is interpolated at the interval start time and the other is interpolated at the interval end time. These samples are used to determine the min or max just like any raw value.
MinimumTime Displays the time stamp of the minimum value in a specified interval.

See the note in Minimum for additional information.

Maximum Displays the maximum value in a specified interval.
Note: The Maximum and MaximumTime calculation internally retrieve two additional samples per interval; one is interpolated at the interval start time and the other is interpolated at the interval end time. These samples are used in the min or max just like any raw or interpolated value.
MaximumTime Displays the time stamp of the maximum value in a specified interval.

See the note in Maximum for additional information.

RawAverage Displays the arithmetic average of the raw values in a specified interval with good data quality. This is useful only when a sufficient number of raw data values are collected.
Average Similar to RawAverage, but performs a special logic for time weighting and for computing the value at the start of the interval. This is useful for computing an average on compressed data.
OPCQOr and OPCQAnd The OPCQOr is a bit wise OR operation of all the 16 bit OPC qualities of the raw samples stored in the specified interval.

The OPCQAnd is a bit wise AND operation of all the 16 bit OPC qualities of the raw samples stored in the specified interval.

Total Retrieves the time-weighted total of raw and interpolated values for each calculation interval. The collected value must be a rate per 24 hours. This calculation mode determines a count from the collected rate.
RawTotal Displays the arithmetic sum of raw values in a specified interval.
StandardDeviation Displays the time-weighted standard deviation of raw values for a specified interval.
RawStandardDeviation Displays the arithmetic standard deviation of raw values for a specified interval.
TimeGood Displays the amount of time (in milliseconds) during an interval when the data is of good quality and matches filter conditions if the filter tag is used.
FirstRawValue Returns the first good raw value for a specified time interval.
FirstRawTime Returns the timestamp of the first good raw for a specified time interval.
LastRawValue Returns the last good raw value for a specified time interval.
LastRawTime Returns the timestamp of the last good raw for a specified time interval.
TagStats Allows you to return multiple calculation modes for a tag in a single query.
Note: You can also use INCLUDEBAD or FILTERINCLUDEBAD as query modifiers to include bad quality data. For more information, refer INLUDEBAD and FILTERINCLUDEBAD sections in Advanced Topics.

Query Modifiers

Query Modifiers are used for retrieving data that has been stored in the archive. They are used along with sampling and calculation modes to get a specific set of data.

Query Modifier Results
ONLYGOOD The ONLYGOOD modifier excludes bad and uncertain data quality values from retrieval and calculations. Use this modifier with any sampling or calculation mode but it is most useful with Raw and CurrentValue queries.

All the calculation modes such as minimum or average exclude bad values by default, so this modifier is not required with those.

INCLUDEREPLACED Normally, when you query raw data from Proficy Historian, any values that have been replaced with a different value for the same timestamp are not returned. The INCLUDEREPLACED modifier helps you to indicate that you want replaced values to be returned, in addition to the currently retrievable data. However, you cannot query only the replaced data and the retrievable values that have replaced the other values. You can query all currently visible data and get the data that has been replaced.

This modifier is only useful with rawbytime or rawbynumber retrieval. Do not use it with any other sampling or calculation mode.

INCLUDEDELETED The INCLUDEDELETED modifier retrieves the value that was previously deleted. Data that has been deleted from the archiver is never actually removed but is marked as hidden. Use the INCLUDEDELETED modifier to retrieve the values that were deleted, in addition to any non-deleted values during the query time period.

This modifier is only useful with rawbytime or rawbynumber retrieval. Do not use it with any other sampling or calculation mode.

ONLYIFCONNECTED

ONLYIFUPTODATE

The ONLYIFCONNECTED and ONLYIFUPTODATE modifiers can be used on any sampling or calculation mode to retrieve bad data if the collector is not currently connected and sending data to the archiver. The bad data is not stored in the IHA file but is only returned in the query. If the collector reconnects and flushes data and you run the query again, the actual stored data is returned in the following situations:
  • Collector loses connection to the archiver
  • Collector crashes
  • Collector compression is used and no value exceeds the dead band
ONLYRAW The ONLYRAW modifier retrieves only the raw stored samples. It does not add interpolated or lab sampled values at the beginning of each interval during calculated retrieval such as average or minimum or maximum.

Normally, a data query for minimum value will interpolate a value at the start of each interval and use that together with any raw samples to determine the minimum value in the interval. Interpolation is necessary because some intervals may not have any raw samples stored.

Use this query modifier with calculation modes only, not with raw or sampled retrieval like interpolated modes.

LABSAMPLING The LABSAMPLING modifier affects the calculation modes that interpolate a value at the start of each interval. Instead of using interpolation, lab sampling is used. When querying highly compressed data, you may have intervals with no raw samples stored. An average from 2 PM to 6 PM on a one-hour interval will interpolate a value at 2 PM, 3 PM, 4 PM, and 5 PM and use those in addition to any stored samples to compute averages. When you specify LABSAMPLING, then lab sampling mode is used instead of interpolated sampling mode to determine the 2 PM, 3 PM, 4 PM, and 5 PM values.

A lab sampled average would be used when querying a tag that never ramps but changes in a step pattern such as a state value or setpoint.

Use this query modifier with calculation modes only, not raw or sampled retrieval like interpolated modes.

ENUMNATIVEVALUE The ENUMNATIVEVALUE modifier retrieves the native, numeric values such as 1 or 2 instead of string values such as on/off for the data that has enumerated states associated with it.

You can use ENUMNATIVEVALUE with any sampling or calculation mode.

INCLUDEBAD Normally, when you query calculated data from Historian, only good data quality raw samples are considered. INCLUDEBAD modifier includes bad data quality values in calculations.

You can use INCLUDEBAD with any sampling or calculation mode.

FILTERINCLUDEBAD Typically, while filtering we use only good data quality values. When we use FILTERINCLUDEBAD, the bad data quality values are considered when filtering to determine time ranges.

This query modifier is not always recommended.

USEMASTERFIELDTIME The USEMASTERFIELDTIME query modifier is used only for the MultiField tags. It returns the value of all the fields at the same timestamp of the master field time, in each interval returned.
HONORENDTIME Normally, a query keeps searching through archives until the desired number of samples has been located, or until it gets to the first or last archive. However, there are cases where you would want to specify a time limit as well. For example, you may want to output the returned data for a RawByNumber query in a trend screen, in which case there is no need to return data that would be offscreen.

In cases where you want to specify a time limit, you can do this by specifying an end time in your RawByNumber query and including the HONORENDTIME query modifier. Since RawByNumber has direction (backward or forward), the end time must be older than the start time for a backward direction or newer than the start time for a forward direction.

Use this query modifier only with the RawByNumber sampling mode.

EXAMINEFEW Queries using calculation modes normally loop through every raw sample, between the given start time and end time, to compute the calculated values.

When using FirstRawValue, FirstRawTime, LastRawValue, and LastRawTime calculation modes, we can use only the raw sample near each interval boundary and achieve the same result. The EXAMINEFEW query modifier enables this. If you are using one of these calculation modes, you may experience better read performance using the EXAMINEFEW query modifier.

Using this query modifier is recommended when:
  • The time interval is great than 1 minute.
  • The collection interval is greater than 1 second.
  • The data node size is greater than the default 1400 bytes.
  • The data type of the tags is String or Blob.

Query performance varies depending on all of the above factors.

Use this query modifier only with FirstRawValue, FirstRawTime, LastRawValue, and LastRawTime calculation modes.

EXCLUDESTALE

Stale tags are tags that have no new data samples within a specified period of time, and which have the potential to add to system overhead and slow down user queries.

The EXCLUDESTALE query modifier allows for exclusion of stale tags in data queries.

Unless permanently deleted, stale tags from the archiver are not removed but are simply marked as stale. Use the query without this query modifier to retrieve the sample values.

Data is not returned for stale tags. An ihSTATUS_STALED_TAG error is returned instead.

Filtered Data Queries

Filtered data queries enhance Historian by adding filter tags and additional filtering criteria to standard queries. Unfiltered data queries in Historian allow you to specify a start and end time for the query, then return all data samples within that interval. A filtered data query, however, will allow you to specify a condition to filter the results by, as well as calculation modes to perform on the returned data. Filtered data queries are performed on the Historian server.

For example, a filtered data query is useful when trying to retrieve all data for a specific Batch ID, Lot Number, or Product Code and for filtering data where certain limits were exceeded, such as all data where a temperature exceeded a certain value. Rather than filtering a full day's worth of process data in the client application, you can filter data in the Historian archiver, and only return the matching results to the client application. The result is a smaller, more relevant data set.

You can use filter criteria with raw, interpolated, and calculated sampling modes. You cannot use it with current value sampling. The logic of selecting intervals is always interpolated, even when the data retrieval is raw or calculated. The value that triggers a transition from false to true can be a raw value or interpolated value.

You cannot use a filtered data query in an iFIX chart. For more information, refer to Advanced Topics section in the online help.

Filter Parameters for Data Queries

Use of filter parameters with a data query is optional.

Parameter Description
Filter Tag The single tag name used when applying the filter criteria.

You can enter your filter conditions using Filter tag, Filter Comparison Mode, and Filter Comparison Value or you can put that all that information in a single FilterExpression.

Filter Expression An expression which includes one or more filter conditions. The type of conditions used are:
  • AND Condition
  • OR Condition
  • Combination of both AND and OR

Filter Expression can be used instead of FilterTag, FilterComparisonMode and FilterValue parameters. While using FilterExpression, the expression is passed within single quotes and for complex expressions we write the conditions within a parenthesis. There is no maximum length for a filter expression, but if it is called using OLEDB or Excel, they may have their own limitations.

Filter Mode The type of time filter.

The Filter Mode defines how time periods before and after transitions in the filter condition should be handled.

For example, AfterTime indicates that the filter condition should be True starting at the timestamp of the archive value that triggered the True condition and leading up to the timestamp of the archive value that triggered the False condition.

ExactTime
Retrieves data for the exact times that the filter condition is True (only True).
BeforeTime
Retrieves data from the time of the last False filter condition up until the time of the True condition (False until True).
AfterTime
Retrieves data from the time of the True filter condition up until the time of next False condition (True until False).
BeforeAndAfterTime
Retrieves data from the time of the last False filter condition up until the time of next False condition (While True).
Filter Comparison Mode Filter Comparison Mode is only used if Filter Tag is filled in.

The Filter Comparison Mode defines how archive values for the Filter Tag should be compared to the Filter Value to establish the state of the filter condition. If a Filter Tag and Filter Comparison Value are supplied, time periods are filtered from the results where the filter condition is False.

The type of comparison to be made on the filter comparison value:
Equal
Filter condition is True when the Filter Tag is equal to the comparison value.
EqualFirst
Filter condition is True when the Filter Tag is equal to the first comparison value.
EqualLast
Filter condition is True when the Filter Tag is equal to the last comparison value.
NotEqual
Filter condition is True when the Filter Tag is NOT equal to the comparison value.
LessThan
Filter condition is True when the Filter Tag is less than the comparison value.
GreaterThan
Filter condition is True when the Filter Tag is greater than the comparison value.
LessThanEqual
Filter condition is True when the Filter Tag is less than or equal to the comparison value.
GreaterThanEqual
Filter condition is True when the Filter Tag is greater than or equal to the comparison value.
AllBitsSet
Filter condition is True when the binary value of the Filter Tag is equal to all the bits in the condition. It is represented as ^ to be used in Filter Expression.
AnyBitSet
Filter condition is True when the binary value of the Filter Tag is equal to any of the bits in the condition. It is represented as ~ to be used in Filter Expression.
AnyBitNotSet
Filter condition is True when the binary value of the Filter Tag is not equal to any one of the bits in the condition. It is represented as !~ to be used in Filter Expression.
AllBitsNotSet
Filter condition is True when the binary value of the Filter Tag is not equal to all the bits in the condition. It is represented as !^ to be used in Filter Expression.
Alarm Condition
Specifies an alarm condition to filter data by. For example, Level.
Alarm SubCondition
Specifies an alarm sub-condition to filter data by. For example, HIHI.
Filter Comparison Value Filter Comparison Value is only used if Filter Tag is filled in.

The value to compare the filter tag with when applying the appropriate filter to the data record set query (to determine the appropriate filter times).

Filtered Queries in the Excel Add-in Example

This example shows how a filtered data query returns specific data from the Historian archive. The example uses two tags: batchid and ramp. The batchid tag is updated before a new batch is produced with the new batch's ID. The ramp tag contains raw data sent by a device in the process. In this example, it is requested that Historian return data samples at ten second intervals for the ramp tag during the period that the batchid tag is set to B1.
A standard query in Historian for the ramp tag's values between 08:00 and 08:01, at ten second intervals, would look like this:
Time Stamp Value Data Quality
07/30/2003 08:00:10 16 Good
07/30/2003 08:00:20 22 Good
07/30/2003 08:00:30 34 Good
07/30/2003 08:00:40 46 Good
07/30/2003 08:00:50 50 Good
07/30/2003 08:01:00 55 Good
If we perform a query against the batchid tag for the same time interval, we would receive the following results:
Time Stamp Value Data Quality
07/30/2003 08:00:00 B0 Good
07/30/2003 08:00:20 B1 Good
07/30/2003 08:00:45 B2 Good

Filtering Data Queries in the Excel Add-in

You can enter your filter conditions using Filter tag, Filter Comparison Mode, and Filter Comparison Value or you can put that all that information in a single FilterExpression. You can enter the filter conditions in the FilterExpression field of the Historian Data Query dialog box. The filter conditions are passed within single quotes.

To find the values of the ramp tag for the B1 batch, enter the following values into the Historian Filtered Data Query dialog box:

  1. In the Tag Name(s) field, enter the tag you want to receive results from - the ramp tag in this example.
  2. Select a start and end time for your query.
  3. In the Filter Tag field, enter the tag you want to enable filtering with - batchid in this example.
  4. In the Filter Comparison field, select your comparison condition.
  5. n the Include Data Where Value Is field, enter your filter condition value.
  6. In the Include Times field, select your filter mode.
  7. In the Sampling Type field, select your sampling mode.
  8. In the Calculation field, select your calculation mode.
  9. Select your Sampling Interval.
  10. In the Output Display field, select the tag values you want to display.