Data Input

Important: You do not have the latest version of Historian! You are missing out on the newest capabilities and enhanced security. For information on all the latest features, see the Historian product page. For more information on upgrades, contact your GE Digital sales agent or e-mail GE Digital Sales Support. For the most up-to-date documentation, go here.

Calculation and Server-to-Server Collectors

The Calculation and Server-to-Server collectors have some unique behavior not found in other standard collectors. This section provides details about Recovery and Manual Recalculation.

Recovery

This feature is unique to Calculation and Server-to-Server collectors. If the calculation engine is not running for a period of time, recovery makes it look like it was running. Recovery can also be used to fill in a hole of time where the collector was not able to communicate with the source archiver.

Recovery is applicable to both unsolicited and polled tags. Messages are also recovered. Comments are not recovered.

Normally, it is impossible to go back to the past and collect data. However, since these collectors are 'deriving' data instead of 'collecting' data, it is possible to recover past data, especially since the source of the derived data is archived in the Historian. It is important to understand that while recovery is possible in the calculation and Server-to-Server collectors, it only makes sense for certain types of calculation formulas.

Intended candidates for data recovery are formulas whose only inputs are Historian tags, since past data for these tags can be interpolated. Formulas that use data from external text files or from ADO via CreateObject will most likely not recover correct data because the inputs are not historized. If you are using these types of formulas, you should turn off recovery for the whole collector or insert VBScript code in the formula of individual tags to detect recovery. An example of this is given in the Historian documentation. A similar approach can be used to set a Max Recovery Time on a tag basis, overriding the collector wide setting.

Even calculation tags using only Historian tags as inputs have some caveats for recovery. If you are deriv- ing calculated data from other calculated data, be sure to set up a trigger tag for each of the tags used in your formula. This way the tags will be processed in chain order. All tags are processed in time order.

The recovery logic is not intended to overcome polled collection overruns. If you configure too much collection, then you will get overruns.

You can control the amount of recovered data using Max Recovery Time configuration setting. You can turn off the recovery by setting it to zero.

Manual Recalculation

The Manual re-calc/re-replicate option is often the best choice for generating past derived data.

How Data Recovery works:

  • When the recovery logic is executed, the collector will setup subscriptions for all the trigger tags.
  • Next, it will recover data. The collector first determines how long it has been since the last write. It compares the current time to data in the registry key LastCalcRepWriteTime, which stores the last time data was written to the archive. The collector compares this to the Max Recovery Time that is specified in the user settings and performs a raw data query on the shorter of these two periods. Then it will take the shorter of these two and do a raw data query for all trigger tags. It will then process the returned samples in sequential order based on time. For example, if the collector was shut down for 8 hours, bnut Max Recovery Time was 4 hours, only 4 hours of data would be recovered.
  • Recovery is performed before real time processing. Once recovery is complete, it will start polling and processing subscriptions in real time. The subscriptions in real time are queued up till the recovery is done.
  • Recovery logic will place an end-of-collection marker at the point in time where the collector was shut down. This end-of-collection marker may or may not be there once the recovery is complete. As part of recovery logic, if it calculates a data point exactly at that timestamp where the end-of-collection marker is there, then it will be overwritten with the calculated good data.
  • The recovery logic does not write samples to trigger tags or tags that are just in the formula. It is intended to write samples to the calculation tags.
  • Messages are added to the log file that indicate when entering and exiting recovery mode.

Examples

The examples below assume the following tag configuration.
  • Machine 1:

    Runs Data Archiver, iFix Collector (Collector 1), and Calculation Collectors.

  • Machine 2:

    Runs iFix Collector (Collector 2), which collects and sends data to the archiver in Machine 1 (as a Remote Collector).

    TagA and TagB are the iFix tags coming from Collector1 and Collector2, respectively. Both of these tags are scanned at a 1-minute poll rate.

Example 1

The following example demonstrates the recovery function for an unsolicited 1-minute interval calculation tag that has a simple current value function.

Create an event based 1-minute interval Calculation Tag (CalcTag1) in Machine 1 consisting of the following calculation: Result=CurrentValue (TagA)

Stop the calculation collector for 5 minutes and then restart it to trigger data recovery for the 5-minute shutdown period. For the following example, the Calculation Collector was stopped at 2002-12-27 17:05:36 and started at 2002-12-27 17:10:48.

Since there is no interruption for the iFIX Collector, the raw data query for TagA results the following output:

Raw Data Query for TagA during shutdown period

114) 81 [2002-12-27 17:02:00:00000] Good NonSpecific

115) 72 [2002-12-27 17:03:00:00000] Good NonSpecific

116) 64 [2002-12-27 17:04:00:00000] Good NonSpecific

117) 56 [2002-12-27 17:05:00:00000] Good NonSpecific

118) 39 [2002-12-27 17:06:00:00000] Good NonSpecific

119) 31 [2002-12-27 17:07:00:00000] Good NonSpecific

120) 22 [2002-12-27 17:08:00:00000] Good NonSpecific

121) 14 [2002-12-27 17:09:00:00000] Good NonSpecific

122) 6 [2002-12-27 17:10:00:00000] Good NonSpecific

A raw data query for CalcTag1 during the shutdown period generates the following:

Raw Data Query for CalcTag1 (before recovery)

96) 81 [2002-12-27 17:02:00:00000] Good NonSpecific

97) 72 [2002-12-27 17:03:00:00000] Good NonSpecific

98) 64 [2002-12-27 17:04:00:00000] Good NonSpecific

99) 56 [2002-12-27 17:05:00:00000] Good NonSpecific

100) 0 [2002-12-27 17:05:36:00000] Bad OffLine

Note that an end-of-collection marker is placed at the shutdown point (that is, at 17:05:36) with a bad data quality.

Once the recovery is complete, this is what we see for the recovered CalcTag1. Note that data during the shutdown period is recovered completely. Compare this result set with the one for TagA. Both are the same.

Raw Data Query for CalcTag1 (after recovery)

96) 81 [2002-12-27 17:02:00:00000] Good NonSpecific

97) 72 [2002-12-27 17:03:00:00000] Good NonSpecific

98) 64 [2002-12-27 17:04:00:00000] Good NonSpecific

99) 56 [2002-12-27 17:05:00:00000] Good NonSpecific

100) 0 [2002-12-27 17:05:36:00000] Bad OffLine

101) 39 [2002-12-27 17:06:00:00000] Good NonSpecific

102) 31 [2002-12-27 17:07:00:00000] Good NonSpecific

103) 22 [2002-12-27 17:08:00:00000] Good NonSpecific

104) 14 [2002-12-27 17:09:00:00000] Good NonSpecific

105) 6 [2002-12-27 17:10:00:00000] Good NonSpecific

Also note that the end-of-collection marker is not overwritten by the recovery logic here. If it calculated a data point exactly at the end-of-collection marker, then it would have been overwritten by the calculated good value.

Example 2

The following example demonstrates the recovery function for an unsolicited calculation tag that has multiple triggers.

Create an event based Calculation Tag (CalcTag2) in Machine 1 consisting of the following calculation:

Result=CurrentValue (TagA) + CurrentValue (TagB)

where TagA and TagB are both trigger tags, coming from Collector1 and Collector2 respectively. Set the collection offset of 5 seconds for TagA and 10 seconds for TagB, forcing the calculation to be performed twice per minute.

Stop the calculation collector for 5 minutes, and then restart it to trigger data recovery for this 5-minutes shutdown period. For the following example, the Calculation Collector was stopped at 02/18/2003 12:15:33 and started at 02/18/2003 12:21:53.

Since the iFIX Collector was not interrupted, a raw data query for TagA and TagB values generates the following output:

Raw Data Query for TagA during the shutdown period

10) 13 [2003-02-18 12:10:05:00000] Good NonSpecific

11) 12 [2003-02-18 12:11:05:00000] Good NonSpecific

12) 11 [2003-02-18 12:12:05:00000] Good NonSpecific

13) 11 [2003-02-18 12:13:05:00000] Good NonSpecific

14) 10 [2003-02-18 12:14:05:00000] Good NonSpecific

15) 18 [2003-02-18 12:15:05:00000] Good NonSpecific

16) 17 [2003-02-18 12:16:05:00000] Good NonSpecific

17) 16 [2003-02-18 12:17:05:00000] Good NonSpecific

18) 16 [2003-02-18 12:18:05:00000] Good NonSpecific

19) 15 [2003-02-18 12:19:05:00000] Good NonSpecific

20) 14 [2003-02-18 12:20:05:00000] Good NonSpecific

21) 13 [2003-02-18 12:21:05:00000] Good NonSpecific

Raw Data Query for TagB during the shutdown period

10) 35 [2003-02-18 12:10:10:00000] Good NonSpecific

11) 34 [2003-02-18 12:11:10:00000] Good NonSpecific

12) 33 [2003-02-18 12:12:10:00000] Good NonSpecific

13) 32 [2003-02-18 12:13:10:00000] Good NonSpecific

14) 31 [2003-02-18 12:14:10:00000] Good NonSpecific

15) 31 [2003-02-18 12:15:10:00000] Good NonSpecific

16) 39 [2003-02-18 12:16:10:00000] Good NonSpecific

17) 38 [2003-02-18 12:17:10:00000] Good NonSpecific

18) 37 [2003-02-18 12:18:10:00000] Good NonSpecific

19) 36 [2003-02-18 12:19:10:00000] Good NonSpecific

20) 36 [2003-02-18 12:20:10:00000] Good NonSpecific

21) 35 [2003-02-18 12:21:10:00000] Good NonSpecific

A raw data query for CalcTag2 during the shutdown period generates the following:

Raw Data Query for CalcTag2 (before recovery)

12) 50 [2003-02-18 12:09:05:00000] Good NonSpecific

13) 50 [2003-02-18 12:09:10:00000] Good NonSpecific

14) 49 [2003-02-18 12:10:05:00000] Good NonSpecific

15) 48 [2003-02-18 12:10:10:00000] Good NonSpecific

16) 47 [2003-02-18 12:11:05:00000] Good NonSpecific

17) 46 [2003-02-18 12:11:10:00000] Good NonSpecific

18) 45 [2003-02-18 12:12:05:00000] Good NonSpecific

19) 44 [2003-02-18 12:12:10:00000] Good NonSpecific

20) 44 [2003-02-18 12:13:05:00000] Good NonSpecific

21) 43 [2003-02-18 12:13:10:00000] Good NonSpecific

22) 42 [2003-02-18 12:14:05:00000] Good NonSpecific

23) 41 [2003-02-18 12:14:10:00000] Good NonSpecific

24) 49 [2003-02-18 12:15:05:00000] Good NonSpecific

25) 49 [2003-02-18 12:15:10:00000] Good NonSpecific

26) 0 [2003-02-18 12:15:11:00000] Bad OffLine

Once data recovery is complete, this is what we see for the recovered data for CalcTag2. Note that data during the shutdown period is completely recovered:

Raw Data Query for CalcTag2 (after recovery)

12) 50 [2003-02-18 12:09:05:00000] Good NonSpecific

13) 50 [2003-02-18 12:09:10:00000] Good NonSpecific

14) 49 [2003-02-18 12:10:05:00000] Good NonSpecific

15) 48 [2003-02-18 12:10:10:00000] Good NonSpecific

16) 47 [2003-02-18 12:11:05:00000] Good NonSpecific

17) 46 [2003-02-18 12:11:10:00000] Good NonSpecific

18) 45 [2003-02-18 12:12:05:00000] Good NonSpecific

19) 44 [2003-02-18 12:12:10:00000] Good NonSpecific

20) 44 [2003-02-18 12:13:05:00000] Good NonSpecific

21) 43 [2003-02-18 12:13:10:00000] Good NonSpecific

22) 42 [2003-02-18 12:14:05:00000] Good NonSpecific

23) 41 [2003-02-18 12:14:10:00000] Good NonSpecific

24) 49 [2003-02-18 12:15:05:00000] Good NonSpecific

25) 49 [2003-02-18 12:15:10:00000] Good NonSpecific

26) 0 [2003-02-18 12:15:11:00000] Bad OffLine

27) 48 [2003-02-18 12:16:05:00000] Good NonSpecific

28) 56 [2003-02-18 12:16:10:00000] Good NonSpecific

29) 55 [2003-02-18 12:17:05:00000] Good NonSpecific

30) 54 [2003-02-18 12:17:10:00000] Good NonSpecific

31) 54 [2003-02-18 12:18:05:00000] Good NonSpecific

32) 53 [2003-02-18 12:18:10:00000] Good NonSpecific

33) 52 [2003-02-18 12:19:05:00000] Good NonSpecific

34) 51 [2003-02-18 12:19:10:00000] Good NonSpecific

35) 50 [2003-02-18 12:20:05:00000] Good NonSpecific

36) 50 [2003-02-18 12:20:10:00000] Good NonSpecific

37) 49 [2003-02-18 12:21:05:00000] Good NonSpecific

38) 48 [2003-02-18 12:21:10:00000] Good NonSpecific

39) 47 [2003-02-18 12:22:05:00000] Good NonSpecific

40) 46 [2003-02-18 12:22:10:00000] Good NonSpecific

Example 3

The following example demonstrates the recovery function for an unsolicited calculation tag that has multiple triggers, but for which none of the triggers is in the formula.

TagA and TagB are the iFix tags coming from Collector1 and Collector2, respectively. Both tags are scanned at a 1-minute poll rate. This example uses two more iFix tags, TagC and TagD, coming from Collector1.

Create an event-based Calculation Tag (CalcTag3) in Machine 1 consisting of the following calculation:

Result=CurrentValue (TagA) + CurrentValue (TagB)

Make sure that the trigger tags for this calculation tag are TagC and TagD, which are not in the formula. Set the collection offset of 5 seconds for TagC and 10 seconds for TagD, forcing the calculation to be performed twice per minute.

Stop the calculation collector for 5 minutes, and then restart it to trigger data recovery for this 5-minutes shutdown period. For the following example, the Calculation Collector was stopped at 02/18/2003 02:24:37 and started at 02/18/2003 02:31:44.

Since the iFIX Collector was not interrupted, a raw data query for TagA and TagB values generates the following output:

Raw Data Query for TagA during shutdown period

56) 13 [2003-02-18 14:21:05:00000] Good NonSpecific

57) 12 [2003-02-18 14:22:05:00000] Good NonSpecific

58) 11 [2003-02-18 14:23:05:00000] Good NonSpecific

59) 11 [2003-02-18 14:24:05:00000] Good NonSpecific

60) 10 [2003-02-18 14:25:05:00000] Good NonSpecific

61) 19 [2003-02-18 14:26:05:00000] Good NonSpecific

62) 18 [2003-02-18 14:27:05:00000] Good NonSpecific

63) 17 [2003-02-18 14:28:05:00000] Good NonSpecific

64) 16 [2003-02-18 14:29:05:00000] Good NonSpecific

65) 16 [2003-02-18 14:30:05:00000] Good NonSpecific

66) 15 [2003-02-18 14:31:05:00000] Good NonSpecific

Raw Data Query for TagB during shutdown period

141) 36 [2003-02-18 14:20:10:00000] Good NonSpecific

142) 36 [2003-02-18 14:21:10:00000] Good NonSpecific

143) 35 [2003-02-18 14:22:10:00000] Good NonSpecific

144) 34 [2003-02-18 14:23:10:00000] Good NonSpecific

145) 33 [2003-02-18 14:24:10:00000] Good NonSpecific

146) 32 [2003-02-18 14:25:10:00000] Good NonSpecific

147) 31 [2003-02-18 14:26:10:00000] Good NonSpecific

148) 31 [2003-02-18 14:27:10:00000] Good NonSpecific

149) 39 [2003-02-18 14:28:10:00000] Good NonSpecific

150) 38 [2003-02-18 14:29:10:00000] Good NonSpecific

151) 37 [2003-02-18 14:30:10:00000] Good NonSpecific

152) 36 [2003-02-18 14:31:10:00000] Good NonSpecific

A raw data query for CalcTag3 during the shutdown period generates the following:

Raw Data Query for CalcTag3 (before recovery)

6) 49 [2003-02-18 14:21:05:00000] Good NonSpecific

7) 49 [2003-02-18 14:21:10:00000] Good NonSpecific

8) 48 [2003-02-18 14:22:05:00000] Good NonSpecific

9) 47 [2003-02-18 14:22:10:00000] Good NonSpecific

10) 46 [2003-02-18 14:23:05:00000] Good NonSpecific

11) 45 [2003-02-18 14:23:10:00000] Good NonSpecific

12) 45 [2003-02-18 14:24:05:00000] Good NonSpecific

13) 44 [2003-02-18 14:24:10:00000] Good NonSpecific

14) 0 [2003-02-18 14:24:11:00000] Bad OffLine

A data query for the recovered CalcTag3 values once data recovery is complete generates the following. Note that data during the shutdown period is completely recovered:

Raw Data Query for CalcTag3 (after recovery)

6) 49 [2003-02-18 14:21:05:00000] Good NonSpecific

7) 49 [2003-02-18 14:21:10:00000] Good NonSpecific

8) 48 [2003-02-18 14:22:05:00000] Good NonSpecific

9) 47 [2003-02-18 14:22:10:00000] Good NonSpecific

10) 46 [2003-02-18 14:23:05:00000] Good NonSpecific

11) 45 [2003-02-18 14:23:10:00000] Good NonSpecific

12) 45 [2003-02-18 14:24:05:00000] Good NonSpecific

13) 44 [2003-02-18 14:24:10:00000] Good NonSpecific

14) 0 [2003-02-18 14:24:11:00000] Bad OffLine

15) 43 [2003-02-18 14:25:05:00000] Good NonSpecific

16) 42 [2003-02-18 14:25:10:00000] Good NonSpecific

17) 51 [2003-02-18 14:26:05:00000] Good NonSpecific

18) 50 [2003-02-18 14:26:10:00000] Good NonSpecific

19) 49 [2003-02-18 14:27:05:00000] Good NonSpecific

20) 49 [2003-02-18 14:27:10:00000] Good NonSpecific

21) 48 [2003-02-18 14:28:05:00000] Good NonSpecific

22) 56 [2003-02-18 14:28:10:00000] Good NonSpecific

23) 55 [2003-02-18 14:29:05:00000] Good NonSpecific

24) 54 [2003-02-18 14:29:10:00000] Good NonSpecific

25) 54 [2003-02-18 14:30:05:00000] Good NonSpecific

26) 53 [2003-02-18 14:30:10:00000] Good NonSpecific

27) 52 [2003-02-18 14:31:05:00000] Good NonSpecific

28) 51 [2003-02-18 14:31:10:00000] Good NonSpecific

29) 49 [2003-02-18 14:32:05:00000] Good NonSpecific

30) 49 [2003-02-18 14:32:10:00000] Good NonSpecific

31) 48 [2003-02-18 14:33:05:00000] Good NonSpecific

32) 47 [2003-02-18 14:33:10:00000] Good NonSpecific