Technical Data Migration Specification.pdf

Data migration is the process of moving data from one environment to a new one; it may be used to support migration from one database to another or between major upgrades of a database. The implementation of master data management may also require data migration. The data integration, ETL,ELT and replication, which are primarily concerned with moving data between existing environments, may be used in order to support the migration process .Data migration is often undertaken as a part of a broader application migration (for example: migrating from SAP to Oracle, consolidating SAP environments or migrating from one version of SAP to another. or when migrating to SaaS (software as a service) environments it is important for data migration to be automated as much as possible, especially where these applications have been acquired directly by the business rather than via IT. Data migration projects are undertaken because they support business objectives. There are costs to the business if it goes wrong or if the project is delayed, and the most important factor in ensur¬ing the success of such projects is close collaboration between the busi¬ness and IT. Whether this means that the project should be owned by the business— treated as a business project with support from IT—or wheth¬er it should be delegated to IT with the business overseeing the project is debatable, but what is clear is that it must involve close collaboration. This white paper is about the acquisition of FMG(Fast Moving Goods)business of one company(COM-1) by another company (COM-II) resulting in the merger of the FMG business of COM-I into COM-IIs. It involves migration of huge amount of data from one company to the other resulting from partial M&A between COM-I and COM-II keeping the following parameters in check -Data integrity, Time(duration of engagement),Cost of technology, Man hours sent, Downtime, Maintaining availability of application. Huge data migration is not only cumbersome but requires special tools and techniques for maintaining integrity of the data. Migration of data from one source (company) to the other (company) requires time and effort and has huge cost implications that are not visible on the surface and hence extensive design, planning and funding are needed. Various tools are evaluated for migration of data but owing to the complexity of the existing system which involves Open road as front end, tuxedo at middle tier and Oracle 10g at the backend and there were many critical business rules applied at all the three tiers, that needs to be taken into account while migrating the data. This involved lots of study and research in term of determining the best methodology for migration data from one landscape to another landscape. Please note the two landscapes may be on two entirely different platforms involving lots of complexity and contradictions. There was a need to study in details the application and hardware architecture of both the systems for the purpose of data migration /integration. For the purpose of data migration from one environment to another, all the validation (including Biz validation at front end and middleware and data referential and integrity validations at backend) should be considered and cannot be bypassed for the sake of migration.

—Data migration is one of the vital tasks of Data integration process. It is always assumed to be most tedious as there will never be a systematic defined procedure. Each migration process is to be treated as unique as the input data sets will be different and the output format required is always unique based on the services provided as well as the user and data handler requirements. In the recent years data migration became the most vital process in various departments of public and private services due to technological advancements and big data handling requirements caused by the increase in acquired data volume. This paper discusses about data migration requirement, data migration strategy finalization and various stages of data migration process discussion of each stage and why complete automation of data migration is not feasible etc.

2008

Relational DataBases (RDBs) are dominant in the market place yet they have limitations in the support of complex structure and user-de ned data types provided by relatively recent database technologies (i.e., object-based and XML databases). Such a mismatch inspires work on migrating an RDB into these technologies. The problem is how to effectively migrate existing RDBs, as a source, into the recent database technologies, as targets, and what is the best way to enrich RDBs' semantics and constraints in order to meet the characteristics of these targets? Existing work does not appear to provide a solution for more than one target database. We tackle this question by proposing a solution for migrating an RDB into these targets based on available standards. The solution takes an existing RDB as input, enriches its metadata representation with as much semantics as possible, and constructs an enhanced Relational Schema Representation (RSR). Based on the RSR, a canonical data model is generated, which captures essential characteristics of the target data models that are suitable for migration. A prototype has been implemented, which successfully migrates RDBs into object-oriented, object-relational and XML databases using the canonical data model.

Lecture Notes in Computer Science, 2008

This paper presents an investigation into approaches and techniques used for database conversion. Constructing object views on top of a Relational DataBase (RDB), simple database integration and database migration are among these approaches. We present a categorisation of selected works proposed in the literature and translation techniques used for the problem of database conversion, concentrating on migrating an RDB as source into object-based and XML databases as targets. Database migration from the source into each of the targets is discussed in detail including semantic enrichment, schema translation and data conversion. Based on a detailed analysis of the existing literature, we conclude that an existing RDB can be migrated into object-based/XML databases according to available database standards. We propose an integrated method for migrating an RDB into object-based/XML databases using an intermediate Canonical Data Model (CDM), which enriches the source database's semantics and captures characteristics of the target databases. A prototype has been implemented, which successfully translates CDM into object-oriented (ODMG 3.0 ODL), object-relational (Oracle 10g) and XML schemas.

Proceedings of the 2004 ACM SIGMOD international conference on Management of data - SIGMOD '04, 2004

Lecture Notes in Computer Science, 2010

Information systems often hold data of considerable value. Their continuing development or maintenance will often necessitate evolution of the system and migration of the data from one version to the next: a process that may be expensive, time-consuming, and prone to error. That such a process remains a source of challenges, is recognised by both academia and industry. In current practice, data migration is often considered only in the later stages of development, leaving critical data to be transformed and loaded by handwritten scripts, long after the design process has been completed. The advent of model-driven engineering offers an opportunity to consider the question of information system evolution and data migration earlier in the development process. A precise account of the proposed changes to an existing system model can be used to predict the consequences for existing data, and to generate the necessary data migration implementation. This dissertation shows how automatic data migration can be achieved by extending the definition of a data modeling language to include modellevel operations, each of which corresponds to the addition, modification, or deletion of a model component. Using the Unified Modeling Language (UML) notation as an example, we show how the specification of these operations may be translated into an abstract program in the Abstract Machine Notation (AMN), employed in the B-method, and then formally checked for consistency and applicability prior to translation into a concrete programming notation, such as Structured Query Language (SQL). Employee PersonalFile file employee Department employee department manages managedBy Figure 2.2: An ORM model diagram for a simplified Employee Information System the diagram does not explicitly state whether the employee works for the department, or the department works for the employee). Outside academia, Chen's notation seems to be rarely used nowadays. One important reason may be that there are so many ER notation versions, with no single standard. 2.1.2 Fact-Oriented modeling Object Role Modeling (ORM) is one of the most commonly cited Fact-Oriented modeling method [27]. It began in the early 1970s as a semantic modeling approach that views the world simply in terms of objects playing roles (taking parts in relationships). ORM has appeared in a variety of forms such as the natural-language information analysis method (NIAM) [74]. ORM includes various procedures to assist in the creation and transformation of data models. A key step in its design procedure is the verbalization of information examples relevant to the application, such as sample reports expected from the system. ORM sentence types (and constraints) may be specified either textually or graphically. Domain concepts are shown in ORM as named ellipses and must have a reference scheme, as an abbreviation of the relevant association (e.g., Employee has a name). In ORM, a role is a part played in a relationship or association. A relationship is shown as a named sequence of one or more role boxes, each connected to the object type that plays it.

Every type of system may replace or enhance the functionality currently delivered by legacy systems to new system, regardless of the type of project/application; some data conversion may take place. Difficulties arise when we take the information currently maintained by the legacy system and transform it to fit into the new system. We refer to this process as data migration. Data migration is a common element among most system implementations. It can be performed once, as with a legacy system redesign, or may be an ongoing process as in storage of historical data in the form of a data warehouse. Some legacy system migrations require ongoing data conversion if the incoming data requires continuous cleansing. It should be that any two systems that maintain the same sort of data must be doing very similar things and, therefore, should map from one to another with ease. Legacy systems have historically proven to be far too lenient with respect to enforcing integrity at the atomic level of data. Another common problem has to do with the theoretical design differences between hierarchical and relational systems. In data migration one method apply in twice (i.e. automated and manual). This paper explores the steps to migrate date in form of manual, i.e. process of data migration without the help of any special tool those made for data migration. Manual data cleaning is commonly performed in migration to improve data quality, eliminate redundant or obsolete information, and match the requirements of the new system in correct and efficient form.

International Journal of INTELLIGENT SYSTEMS AND APPLICATIONS IN ENGINEERING, 2024

In this work, a procedure for converting a relational database (RDB) into an XML document is proposed. Database migration is the process of moving schema and data from a source RDB towards the destination database of XML script, which makes to achieved and moved through new context. The home schema is semantically enhanced and translated into a target schema, and the data in the source database is transformed into a target database based on the new schema. The semantic enrichment procedure is required to create an improved metadata model from the source database and captures key elements of the destination XML schema, making it appropriate for turning RDB data into an XML document. Algorithms are designed for constructing the target database based on a set of migration rules to translate all RDB constructions into an XML Schema, from which RDB data is subsequently transformed. A prototype system has been developed and experimentally assessed by testing its outcomes, looking at our accomplishments, and commenting on the findings. The recommended remedy is found to be effective and accurate after the review of the outcomes.

International Journal for Research in Applied Science & Engineering Technology (IJRASET), 2022

The data migration process is usually needed when there is a change in system, format, or storage type. Currently, there are several techniques and tools for migrating data, for example, CSV files, ODBC, SQL Dump, and so on. Inappropriately not all of these techniques can be implemented for migrating data between two different DBMS. This research describes a data migration technique that can be used to migrate data across DBMS. The data migration technique described makes use of existing metadata in each DBMS. The data migration process described here goes through three stages, namely capture, convert and construct. A prototype was built to test this data migration technique. By using the HR schema, cross-DBMS data migration trials were carried out between Oracle and MySQL. Using this technique, full-schema data migration takes an average of 20.43 seconds from Oracle DBMS to MySQL and 12.96 seconds for the reverse scenario. As for partial data migration, it takes an average of 5.95 seconds from Oracle DBMS to MySQL and 2.19 seconds for the reverse scenario