{"id":4706,"date":"2016-04-04T14:55:44","date_gmt":"2016-04-04T13:55:44","guid":{"rendered":"https:\/\/etlsolutions.com\/?p=4706"},"modified":"2021-04-29T10:38:34","modified_gmt":"2021-04-29T09:38:34","slug":"how-to-prepare-your-data-before-a-data-migration","status":"publish","type":"post","link":"https:\/\/etlsolutions.com\/how-to-prepare-your-data-before-a-data-migration\/","title":{"rendered":"How to prepare your data before a data migration"},"content":{"rendered":"

Data migration is a complex undertaking: according to a study by Bloor Research, 30% of data migration projects fail.\u00a0 Thorough preparation of data and systems before a migration takes place helps to reduce the risks involved.\u00a0 (For information on how to plan an entire data migration project, visit this post<\/a>.)<\/p>\n

Landscape analysis<\/h4>\n

Landscape analysis is an important part of preparing for a data migration.\u00a0 It provides an overview of the source and target systems, enabling the project team to understand how each system works and how the data within each system is structured.\u00a0 For detailed information on how to conduct a landscape analysis, take a look at this useful article<\/a> at Data Migration Pro.<\/p>\n

Data assurance<\/h4>\n

Another important component is data assurance.\u00a0 This procedure validates the information discovered in the landscape analysis and ensures that all data is fit for purpose.\u00a0 By validating the data, the migration team are then free to focus solely on structural manipulation and movement.\u00a0 Data assurance has several phases: data profiling; data quality definition; and data cleansing.
\n\"Data-migration-eGuide\"<\/a><\/p>\n

Data profiling<\/h4>\n

The aim of the data profiling phase is to ensure that any historical data due to be migrated is suitable for the changes that are taking place in the organisation.\u00a0 Profiling should be carried out to identify areas of the data which may not be of sufficient quality.\u00a0 It should include comprehensive checks of existing model structure, data format and data conformance.<\/p>\n

A retirement plan should be used to define the data no longer required.\u00a0 Any data to be retired should be recorded, along with a description of what replaces it or why it can be removed.\u00a0 The data that is no longer needed may have to be archived for tax purposes or to meet the requirements of an industry\u2019s governing bodies.<\/p>\n

Data quality definition<\/h4>\n

Data quality definitions state the quality that must be attained by elements, attributes and relationships in the source system.\u00a0 The definitions or rules should be used during profiling to identify whether or not the data is of the correct quality and format.\u00a0 All data quality rules should be listed at element level, such as data table or flat file. All data quality issues and queries should be tracked and stored.<\/p>\n

Data cleansing<\/h4>\n

The first stage in data cleansing is to define which cleansing rules will be carried out manually and which will be automated.\u00a0 Splitting the rules into two enables the organisation\u2019s domain experts to concentrate on the manual process, while the migration experts design and develop the automated cleansing.\u00a0 Typically, the manual cleansing will be carried out before the migration, while the automated cleansing may be carried out before the migration or as part of the migration\u2019s initial phase.<\/p>\n

Data verification is the part of the data cleansing process that checks that the data is available, accessible, complete and in the correct format.\u00a0 Our consultants often continue to carry out verification once a migration has begun, ensuring that the information is optimised prior to each stage of the migration.<\/p>\n

We find that data impact analysis is a crucial part of data cleansing.\u00a0 Because cleansing data adds or alters values, data impact analysis ensures that these changes do not have a knock-on effect on other elements within the source and target systems.\u00a0 It also checks the impact of data cleansing on other systems which currently use the data, and on systems which may use the data once the migration is complete.<\/p>\n

All these areas should be reviewed systematically to ensure that potential errors are identified in advance of the migration.<\/p>\n

Further reading<\/h3>\n\n
\n

<\/p>