Migrating iFix Data to Historian

Important: You do not have the latest version of Historian! You are missing out on the newest capabilities and enhanced security. For information on all the latest features, see the Historian product page. For more information on upgrades, contact your GE Digital sales agent or e-mail GE Digital Sales Support. For the most up-to-date documentation, go here.

About Migrating iFix to Historian

The iFIX migration tools are intended for users who are responsible for migrating their Classic or Advanced Historian systems and iFIX alarm and event data to Historian. This manual assumes that you are familiar with the iFIX and the Advanced Historian or Classic Historian environments.

This manual describes the steps for planning your Historian migration and performing the migration of your data from both the Advanced Historian and Classic Historian environments.

For more information on running iFIX, refer to the iFIX Help. If you are planning to implement Electronic Signatures and Electronic Records or Historian Security, it is recommended that you perform migration prior to setting up or initializing these features. If the migration will be a gradual process over time, please refer to Using Historian in a Regulated Environment, Using Historian and Implementing Security During Migration.

Before You Begin

Before you start migrating your Classic or Advanced Historian data, be aware of the following:

Confirm that your Historian environment is set up. For more information, refer to the Getting Started with Historian manual.
Confirm that you have ample disk space on the archive machine. Assume that you need to have at least as much free space as the amount of data that you intend to migrate. For example, if you have 300MB of Advanced Historian data, you need at least 300MB free disk space on the Historian Server to accept that data.
Do not uninstall Advanced Historian until you have migrated all the data to your Historian. Advanced Historian must be running and must have all its components in place for migration. The Historian Server that you are migrating to does not need to be on the same computer as the Advanced Historian or Classic data that you are migrating.
Migrating a large amount of data from Advanced or Classic Historian into Historian may take a substantial amount of time, depending on the size of the database and the processing power of your machines. For more information, refer to Estimating Migration time.
When migrating Historical data from a SCADA node to an Historian Server, it is recommended that you have an iFIX database loaded for the corresponding iFIX historical tags (on the source machine) prior to migration so that descriptions and EGUs can be retrieved.
When migrating Historical data to Historian, it is recommended that you migrate the data in chronological order from the oldest to the newest. This prevents adding data out of order, which may impact migration performance and disk space usage.
Migrate collection groups before historical data to maintain collection rates and deadbands.
For migrating alarms and events, determine the location and time range of your iFIX alarms logged via Alarm ODBC.

Historian Migration Utilities

Historian provides three utilities that allow you to migrate your existing Classic and Advanced Historian configuration and data into the Historian environment and migrate alarm and event data from iFIX. The Historian migration tools are designed to help you migrate your data quickly and easily. The tools allow you to:

Select which Advanced Historian tags or Classic Historian archive files to migrate.
Set up configuration options.
Migrate data into your Historian system.
Migrate alarms and events logged via iFIX Alarm ODBC into your Historian system.

Plan Your Migration Startegy

In order to successfully migrate your data and alarms from existing iFIX or Advanced Historian applications into the Historian system, you should first plan out a sound migration strategy. Consider the following questions when planning your migration:

Questions	Refer to section
How should I be adding tags to the archive?	Adding Tags to the Archive
Should I compress my data?	Planning Compression
In what order should I migrate my data?	Recommended Migration Order
Do I need any security rights to migrate my data?	Implementing Security During Migration
What if my data is being migrated into an already active Historian system?	Planning Migrations with Online Systems
Should I account for Daylight Savings Time or Time Zone differences?	Planning Daylight Savings Time
How long will my migration take?	Estimating Migration Time
What if my ADH archives have been moved around or I'm attempting to migrate backups of older ADH archives?	Registering Advanced Historian Archives
Is there anything I should do after migrating?	After Migrating Your Data

Adding Tags to the Archive

You must have tags in Historian to hold the migrated data. You can add tags automatically during group migration, data migration, or manually prior to migration using the Historian Administrator. When you add tags by migration, you can set the maximum number of tag properties when the migration program has access to the real time database. Having access to the iFIX real time database allows the program to retrieve the Description, HI and LO Engineering Units, and Engineering Unit descriptions. These are not stored in the Classic Historian data or group files.

Migrating group files will add tags for immediate collection. Immediate collection occurs as a result of migrating the collection interval during the process of migrating the group files. Qualifier and phase values are not preserved, deadband values are preserved.

Planning Compression

Collector compression is automatically configured for collection groups that have a configured deadband in HTA when those collection groups are migrated to Historian.

By default, tags are migrated with archive compression turned off. Only archive compression has any effect on migration.

To enable archive compression during migration:

Add the tags through the Historian Administrator, or, Migrate the historical groups.
After the tags are added to Historian, enable archive compression for each tag through the Historian Administrator before migrating the Classic Historian data.
Subsequent migrated data can then pass through archive compression during migration.

Recommended Migration Order

When migrating historical data to Historian, it is recommended that you migrate the data in a chronological order from oldest to newest. This prevents adding data out of order, which may impact migration performance. It is also recommended that you migrate collection groups before data.

Planning Migrations with Online Systems

Typically, migration is performed after configuring and running new collection. Depending on the amount of data migrated, the process may take hours or days.

Alarm and event migration can take a significant amount of time. You can mitigate the risk of data loss by configuring the alarm collectors before starting the migration. You may also choose to migrate your alarm and event data in blocks of time ranges.

If you are migrating Classic Historian data and decide to select the Overwrite Existing Tags option in the Migration Options dialog box, the existing Historian tag properties are replaced by the migrated Classic Historian tag properties and some configuration properties are overwritten. If you decide to clear the Overwrite Existing Tags option, tag information will not migrate into the Historian tag database. If you are migrating your Historian data into an existing Historian tag database, you may discover that tags with the same name exist in both the Historian database and the iFIX or Advanced Historian database.

Note: Do not attempt to migrate the currently collecting Classic Historian file. If you do, you may receive an Error -9.

If you are migrating Advanced Historian data and decide to select the Allow Updates to Existing Data option in the Advanced Historian to Historian Migration Utility, the existing Historian tag data will be replaced by migrated Advanced Historian tag data if the data points have the same timestamps. If you decide to clear the Allow Updates to Existing Data option, the data will not be migrated (replaced) where there is existing data with duplicate timestamps in Historian.

Limit Processing Load on Server During Migration

Both data migration utilities provide you with the ability to limit the amount of processing load on the server during migration. This allows your online Historian Server to continue processing data efficiently while migration occurs. By modifying the Events rate/second field in the Classic Historian Migration Options dialog box or the Max values/sec field in the Advanced Historian to Historian Migration Utility, you can specify how much data is sent and therefore how much processing power the Migration Utility receives on your system. The minimum you can set this rate to is 10,000 (or 0, meaning no throttle); the maximum you can set this field to is 100,000.

If you are performing a migration on a computer that is also processing other data or applications, you may want to set the event processing speed lower to allow your other applications to process well. The Historian Classic Migration and Advanced Historian Migration defaults to 10,000 events per second. The default value allows you to throttle back the Migration Utility to allow for other applications processing, as well as continue running your Migration Utility at a reduced speed. If you are not running other data or applications on your machine and you want to run the Migration Utility at maximum speed, set the Events rate/second or Max Values/sec fields to 0.

Tip: It is recommended that you do not set the events per second past 25,000 or below 10,000 (unless to 0). The higher the events per second, the faster your Migration Utility processes the migration. For example, the migration may take twice as long at 10,000 as it will at 20,000 but it will take less CPU time.

Note: You cannot change this entry while migrating. It must be set prior to or changed after migration.

Implementing Security During Migration

In order to migrate data and alarms into the Historian System, you must be a member of the appropriate predefined Historian Security Groups if you have implemented Historian security. Refer to the Historian Group Rights section for information on the individual groups required for each task.

If any existing Historian Security groups are defined, you must supply a username and password before migrating your data in either the Migration Options dialog box (for Classic Historian) or the Advanced Historian to Historian Migration Utility dialog box (for Advanced Historian) or in the Alarm Destination dialog box (for alarm migration).

Applying Daylight Savings Time

When migrating alarms or Classic Historian data, you can select whether or not you want to apply Daylight Savings Time (DST) bias to timestamps. The Migration Utility converts the timestamps of migrated samples to UTC time (universal time format for storing timestamps) before writing the data to Historian. If you select this option, the Migration Utility will apply the DST offset before converting to UTC time. The timestamps are converted to UTC time by adjusting the time based on the local computer time zone offset and DST setting.

If you enabled DST in your operating system during data or alarm logging, you must select the Treat as DST Timestamp option. With Classic Historian, a switch from Daylight Time to Standard Time results in a loss of one hour of data between 1:00:00 and 1:59:59 AM. With Historian, no loss of data occurs when you switch from Daylight Time to Standard Time.

Note: The migration utilities assume that the computer you are migrating your data from and the computer that you are migrating your data to are in the same Time Zone.

Estimating Migration Time

Migrating a large historical database from either Advanced or Classic Historian into Historian may take hours or days, depending on the size of the database and the processing power of your machines.

To estimate migration time, refer to the total file size of the migration files. For Classic Historian migration, it takes approximately one minute to migrate 3 to 8.3 MBs of data. For the quickest migration, it is recommended that you clear the Migrate Current Alarm and the Readback Values option, if they are not required, in the Classic Historian Migration Options dialog box.

For Advanced Historian data, it takes approximately 30 minutes to migrate 100 MB of data. This estimation assumes a default throttling of 10,000 events per second. If you have modified your Max Values/sec field in Advanced Historian, your migration time may vary. For more information, refer to Limit Processing Load on Server During Migration.

Note: In addition to migration file size, migration time depends upon the processing power of the computer running the migration utility. It is highly recommended that you select a computer with high processing power to run the migration program.

Alarm migration performance is bounded to the CPU power of both the migrating machine and the alarm archiver machine.

Register Your Advanced Historian Archives

If you have changed the original location of your archives or you have backups of older archives that you wish to migrate, you must enable those archives by registering them.

Use the following guidelines when registering your archives:

No other archive can hold data in the same time frame.
Archives must be registered using PIARTOOL -AR {Full Path of file}.
PIARTOOL - AL must show that the archive is registered prior to migration.

Note that any unregistered archive will not be migrated.

For more information on registering your archives, refer to step 2 of Migrating Remote Advanced Historian Data.

After Migrating Your Data

After migrating your data, ensure that you are no longer running Historical Collect (HTC). If you have HTC configured to start automatically from the iFIX WorkSpace, remove HTC from the SCU task startup list and replace it with the iFIX Collector.

You may choose to do a Historian backup of your newly created Historian archives containing the migrated data.

After migrating your alarms, you may choose to change the data source name of the alarms so they match the data collector and appear as if they were collected along with the real time collector. For more information, see Alarms and Events Collector.

Adding the Historian Toolbar

To configure the toolbar to appear in the WorkSpace:

Start iFIX v4.0 or greater. The iFIX WorkSpace opens, if configured so.
Select Toolbars from the WorkSpace menu. The Toolbars dialog box appears.
Click the Customize button. The Customize Toolbars dialog box appears.
Click the Import button. The Import Toolbars dialog box appears.
Select Historian 7.1.
Click the Import button. The Historian Toolbar appears.
Click the Close button to close the Import Toolbars dialog box.
Click the Close button to return to the WorkSpace.

Configure Historian Server Buttons

The Configure Historian Server button in the Historian Toolbar specifies the location of historical data retrieval for the WorkSpace, not the location of the historical data storage. You can view/retrieve data stored on these listed servers while you select a pen for a chart display. The Configure Historian Servers dialog box also determines where HDA programs and historical ODBC retrieve data from, which is always the default server.

Migration Checklist

The following is a list of general tasks for migrating your data to an online or new Historian system.

Verify that you are familiar with the setup recommendations. How?
Estimate the number of archives that Historian will create during migration and verify that you have enough disk space to accommodate the new archives and backups of those archives. How?
Migrate all collection groups. How?
If you are migrating to an online Historian system and tags exist in the iFIX database, the collector begins collecting on those tags.
Export tag configuration information.
Migrate Classic Historical data or, How?
Migrate Advanced Historical data or, How?
Backup the newly created archives. How?
Start your collectors if they are not already running.
Verify migrated data through one of the following options:
- iFIX chart
- Raw data dump into OLE DB
- Classic Historian log file
- Data Readback Verification option in the Classic (optional) and Advanced (automatic) Migration utilities.