We’ve conducted a series of live demos on E&P data management in the last month and would like to thank the attendees for all the questions and comments during the sessions. As promised, we’ve pulled together a round-up of some of the more popular questions below in case there’s anything you’ve missed. Join our mailing list at the bottom of this page to find out about future demos.

Q: Who is the Transformation Manager software aimed at?

A: Our Transformation Manager data movement software is designed for use by developers. However, it provides a neutral environment in which the developer can work closely with other colleagues, such as domain experts.

Q: Does the user have to do anything manually when importing models into the software?

A: This depends on the model. There are typically a range of options to select from. Data sources with rich metadata, for example relational databases, need little or no manual intervention, whereas files can require some manual intervention. This is where Transformation Manager’s developer kits are valuable, such as those for LAS and DLIS. Our developer kits include  specific model packs and adapters to correctly load and display the models, without the user having to carry out any work.

Q: Can you add anything to the models in Transformation Manager, such as new attributes or new relationships?

A: Transformation Manager has a change model feature. There are also model editing facilities, enabling the user to add or modify elements, attributes and relationships. Incompatible changes are detected at build time, enabling you to adapt the project to the new model immediately.

Q: How does Transformation Manager handle errors?

A: The user writes an error handling project in Transformation Manager. When an error occurs and an error handling project is attached, the user can map the error to any desired target model, for example an Excel workbook. Information can be logged about what was happening when the error occurred, and this can be passed on to the user’s data quality team to assess or fix in the source.

The user can choose to stop an error or continue the task. For instance, if the movement involves fifty LAS files and the 30th has an error in it, the user can stop and roll back, or continue to process the 49 correct files and then report the error.

Q: How are transformations documented?

A: As a transform is built, Transformation Manager automatically creates documentation. There’s a page of documentation for each transform, making it straightforward to pass documentation on to colleagues or customers.

The screenshot below shows an example of a documentation page. It illustrates how the user can see the actual text of a transform.

Q: How can Transformation Manager help to cut development times in large scale projects?

A: Transformation Manager has been proven in difficult or complex data movements, in large scale data models and with challenging mapping rules. It is an open toolset with additional E&P data management value such as developer kits, including mappings constructed using E&P model terminology.

Transformation Manager generates Java jars, which are very flexible. The user has freedom in terms of deployment and the software won’t conflict with the technologies planned for the project. So it’s a toolset which can be deployed in any way to best suit the user’s particular environment.

Q: What technology is Transformation Manager based on?

A: Transformation Manager is a Java application with a flat file based repository. The development application doesn’t come with a heavyweight server engine; instead it sits on a laptop or desktop PC. As mentioned earlier, the software generates Java jars which can be deployed in batch mode or as part of the messaging infrastructure. Essentially, Transformation Manager creates Java code as if the user had written it, which can then be deployed anywhere where Java can be run.

Q: So Transformation Manager has a front end that builds the appropriate code in the background to create the transform?

A: Yes, the software allows the user to import the metadata of the sources and targets, and then to navigate through them. It brings a lot of rich features to make the user’s task a lot easier. The user writes the source and target mappings in Transformation Manager’s straightforward GUI, and then builds the project. This analyses the mapping rules, checks them against the metadata, and if there are no errors generates an implementation of the mapping. The implementation can be executed using the TM Migrator tool, a GUI application used during development and for manually initiated deployment. You can also generate a deployment pack, which allows you to run the project as a command line application on any supported platform, including Windows, Unix and OSX. Finally, as the generated code is pure Java, it can be invoked using our deployment API, allowing it to be embedded in any Java environment: application servers, client applications, workflow engines and so on.

Q: Can Transformation Manager read or write against vendor APIs?

A: It is often possible to read data directly from an underlying database, for which the standard database functionality of Transformation Manager can be used. The software has an open interface which allows us to quickly build custom loaders and adapters for non-standard data access layers and APIs, supporting all the features of the Transformation Manager tool set at design and run time. Alternatively, data can often be exported or imported using a format directly supported, such as text files or XML documents. A custom adapter framework will soon be available, enabling users to develop API interfaces themselves.

Q: Can Transformation Manager use existing PLSQL and Java code?

A: PLSQL or Java procedures and functions can be called using the external function feature of Transformation Manager. Alternatively, where the code is used as the data access layer, a custom adapter can be built as described in the previous question.

Make sure you don’t miss the next public online demo by signing up for our email newsletter (on the right hand side of this page).

Further reading

    Did you like this article? Get our new articles in your inbox with our occasional email newsletter.

    We will never share your details with anyone else, except that your data will be processed securely by EmailOctopus (https://emailoctopus.com/) in a third country, and you can unsubscribe with one click at any time. Our privacy policy: https://www.etlsolutions.com/privacy-policy/.