Corneliu Chirnoaga

Export-Import in Trading Partner Manager (TPM) 9.1 Scope Being able to export and import data from one system to another is a feature that each modern data system should be able to provide in order to support automation of certain processes like CI/CD. In versions prior to 9.1 TPM provided an export-import feature allowing the export/import of a partner along with its connected routes and partners. This approach covered a specific case thus being limited in scope, hence the need of a more generic and comprehensive export/import mechanism. TPM 9.1 import/export is redesigned along new guidelines: allow for granular exports/imports: Granularity is important especially in situations where you want to control smaller sets of data all the way down to the last entity (a partner/route/system etc. only with its own properties and the associated metadata). Splitting the exports and imports in smaller sets gives more control over what is exported and imported. In this sense such approach creates the premises for composite exports/imports where any data structure can be exported an imported. potential to accept very large data sets to adapt to situations where database needs to be initialized with a pre-existing set of data, or simply to transfer a large set of data from one system to another. import entity child sets merges and/or synchronizations. merging data structures is one of the basic functionalities an import should satisfy. In this sense when importing a partner for instance, you want to have in your target database all the properties of the target partner be similar with the imported partner, including the metadata associated with the specific instance. With regards to entity/domain properties the import works like PUT, where the direct and metadata properties of the target entity are synchronized with the ones belonging to the imported entity. However, if a TPM entity has children (eq a Partner has multiple PartnerSystems), then if a new child is imported this is going to be created in the target TPM, and any existing child properties are synchronized with the corresponding imported child. If a child in the target system does not have a correspondent in the imported set, then this is kept as is. synchronization refers to situations where you want the list of children in the target database to be similar to the one in the import set. For instance, if a Partner has a PartnerSystem that is not in the import set, then this PartnerSystem along with its children is removed. However, this removal will not occur if this if this Partner System is already referenced directly or indirectly by other entities in the system (a PartnerSystem can be referenced via PartnerSystemMessage and a BusinessChannel by an advanced BusinessRoute). Note that this feature is not enabled by default since it can lead to data removal. improved performance enable export/import for all entities of interest in TPM. TPM 9.1 covers all entities used in the system except basic routes and associated children which will be introduced in 9.2. For a complete list please read farther. Exports and imports are grouped in two large categories: elementary and complex (or composite). This article talks about the elementary/domain exports. Export/Import Domains (or Areas) The TPM data structure is split in functional export/import domain (or areas) depicted in the following diagram where each dotted border represents such domain/area: An import/export domain contains either terminal entities, parent entities or parent with child entities. Thus, for example, a Partner can be optionally exported without its dependent children (as a standalone entity), or along with all its dependent children. The dependent entities are all entities in the same dotted rectangle with Partner: Contact, Partner System, Partner System Messages and BusinessChannel. An import/export domain can have subdomains. For instance, Partner has the following direct subdomains: Contact and PartnerSystem. PartnerSystem has PartnerSystemMessages/BusinessChannels as direct subdomain. In order to determine what entities need to be updated or created during import it is necessary to fully resolve the dependencies they rely on. For instance, a Partner System depends on a System by referencing the ID to the respective system. However, in the target database this system ID value might not exist, or if it does, it likely points to another system. In order to resolve this situation, when exporting a certain entity, all direct dependencies are also exported to be able to remap based on name during the import the source dependency ID to the real target dependency ID. This ID remapping is called dependency resolution. As explained before, since the ID of an entity can be different in two different TPM systems, the proper mapping is realized using names: name-based resolution. For example, partner Foo in the source system can have the ID 12 and in the target system the ID 6. On import the target partner ID is determined based on the partner 's name. Therefore, if partner named Foo exists in the target system, then its ID will be used, else a new partner named Foo is created and a new ID is assigned to it. It is important to understand that the name of an entity (for named entities) is paramount in establishing the identity of the entity during the import. Hence, treating the name of the entity as immutable on both source and target system ensures the proper update of the target. Of course, scenarios where the name can change either on source or on target could also be valid, however you have to be aware of the consequences of the changes you make. In the above diagram the dependencies can be determined by all outgoing arrows crossing the export dotted rectangle. Thus, Contact depends on Partner and DomainSubtype. As a consequence, when exporting contacts, you will also see in the export file the partners the contact belongs to and the domain subtype extending the exported contacts. NOTE: exported dependency entities are only used for dependency verification and proper internal ID mapping. They do not contribute to any data changes. Currently there are six major export/import domains: DataType DomainSubtype MetaData Partner Contact PartnerSystem PartnerSystemMessages & BusinessChannels (are always exported together since there is no UI for business channels, and the identity of a PartnerSystemMessages cannot be uniquely established without the connected BusinessChannel) Environment System Standard CodeSubs MessageFormat MessageType BusinessRoute RouteContact RouteCodeSubs The BasicRoutes domain (not depicted in the diagram) along with BasicRouteContacts and BasicRouteCodeSubstitutions will be added in version 9.2. Parent-Child Export/Import Modes Currently there are two supported modes: Export Parent Entities Without Children this mode enables import only for parent entity information: direct properties synchronization TpaValues synchronization (TpaValues pointing to the parent entity found in the target system but not in the import set will be removed from DB) Export Parent Entities with children this mode exports the parent entity along with its child hierarchy. It enables on import: parent and children direct property synchronization parent and children TpaValues synchronization (TpaValues found in the target system but not in the import set will be removed from DB) optionally allows child synchronization (DB children pointing to the parent which are not in the import set will be removed from DB) optionally, on import, only the parent can by picked from the export set and imported into the target database. Data Set Size Import/Export Mode This mode controls how data is aggregated in the export file with regards to memory requirements. Currently two modes are supported: flat data (or streaming) mode and memory mode. Flat Data Mode - Flatten Mode By default, the export is designed in such a way as to scale by optionally flattening the entities in lists that can be streamed to files and thus scalable. In other words, all Partners (without children) will be in a list, the PartnerSystems (without children) in a separate list etc. Memory Mode - Children Entities Generated Under the Parent Entities In instances where TpaValue nodes/data need to be exported under the entity it extends (instead of a separate flat list), will limit the size of such exports due to the fact that the data is generated all in memory. Theoretically, even in these instances data can be generated to files thus allowing to some extent to increase the size of the exported data but with important sacrifices in terms of performance. NOTE: currently TPM is only providing the memory mode, however it is equipped with the proper set of interfaces to be able to stream data to files (to scale to any import size), but no implementation for these interfaces is yet provided. This will be approached in a future release. At this stage the export will only generate the JSON object containing all exported data in memory. Upon import this is fully loaded in memory. NOTE: Internally, after the data is normalized, both modes use the same code base for importing the data. Parent-Child Import Synchronization Mode By default, all imports merge the set of children that belong to a specific parent: a child is identified by name in both source and target database, then the target child properties and TpaValues are synchronized. a child is present in the source but not in the target, this will be created, and all its dependent children (direct or indirect) will be added. a child is found in the target database bout not in the source, it will be not touched. The parent-child synchronization mode targets item 3 above. In this sense, on import an extra option to synchronize the children is provided as a way to remove all children from the target database that are not in the import set. This option is disabled by default because it leads to data removal, so enabling it should be done conscientiously. Therefore, an import that does not enable synchronization will result in a merger for the target with the source (imported) entities. NOTE: if synchronization is enabled, and there are child entities that need to be removed from the database because are not in the import set, the removal of these children still depends on whether they are dependencies for other entities. For instance, if a Contact needs to be removed from a Partner because it is not in the imported set, but there is a Route Contact referencing it, the contact will not be removed. This can be possibly added as a feature in the future, although it would be quite dangerous since it leads to hidden dependency removals. Export Mode Parameters Domain exports can be logically split into to two groups: entities with dependents (children) terminal entities (entities without dependents) In both situations the REST endpoint accepts: a request object containing the entity selection criteria. flattenMetadata parameter which if true (default) will generate data in flatten mode where all metadata is generated as a separate entry in the export file split in buckets that are keyed by DomainType and DomainCode. If this parameter is false, the metadata will be attached to the entity if belongs to. For entities with dependents only, there is a third option includeDependents which dictates if the dependents (direct and indirect) are exported along with the parent. By default, this value is true. The export file will contain the flags indicating what were the parameters used to generate it so on import these are used for import validation. Export Request Object For most of the export endpoints the JSON schema of the request object is the following: { "$schema": "http://json-schema.org/draft-07/schema#", "title": "Generated schema for Root", "type": "object", "properties": { "ids": { "type": "array", "items": { "type": "number" } } }, "required": [ "ids" ] } An example of such object is: { "ids": [127, 137, 147, 157] } In some instances, the ids can either be empty or null. In such instances this will be assimilated with a request to return all objects of the specified entity. In a few situations, this can be null if another selection field which takes precedence is not null, and in these instances ids is ignored (even if populated). NOTE in the future more the request objects could be more complex depending on use cases. However, any such complexity will ultimately result in the above list of IDs. This way, any extension of the selection criteria is easily achievable. Objects where ids can be empty/null and considered to mean 'return all objects in the system'. Usually, these domains are quasi-static in the sense that they foundational blocks for defining Partners and Routes with their dependents: ContactExportRequestDto EnvironmentsExportRequestDto MessageFormatExportRequestDto MessageTypeExportRequestDto StandardsExportRequestDto TpaMetaDataExportRequestDto TpmDataTypesExportRequestDto TpmDomainSubTypesExportRequestDto Objects whose ids can be null if another selection criteria field takes precedence: BusinessRoutesExportRequestDto { // ignored if fromToPartners is populated; if fromToPartners is null, then ids has to be populated "ids": [3,6,23], // takes precedence over ids if populated; can be null in which case ids should be populated "fromToPartners": [ { // PartnerFrom name, or * if all partners are considered "from": "[partnerName|*]", // PartnerTo name, or * if all partners are considered "to": "[partnerName|*]", } // , // ... ] } PartnerSystemsExportRequestDto { // ignored if partnerIDs is populated; if partnerIDs is null, then ids has to be populated "ids": [3,6,23], // list of partner IDs for which PartnerSystems are exported // takes precedence over ids if populated; can be null in which case ids should be populated "partnerIDs": [300, 765,1023] } Export File Structure All exported files, regardless to the endpoint that generated them, have the following structure: { // 1. Who generated the export "@type": "PartnersImpExEnvelopeDto", // 2. Specifies if the TpaValue nodes are generated in the structure of the entities they extends "tpaValuesAreAttached": false, // 3. If the export specified "childrenAreIncluded": true, // 4. exported entities or dependencies "partners": { // 5. serializer type (this one describes in-memory serialization) "@type": "EntityImpExInMemStreamDto", // 6. entity domain "domainType": "PARTNER", // 7. entities exported according to the selection critria "entities": [ { // 8. source entity ID "idPartner": 3, // 9. source entity name "name": "Partner 1", "domainSubTypeList": "1", "idParent": 0, "siteCode": "Partner1 Site Code", "buName": "Partner1 BU", "orgAddress1": "Partner1 Address1", "city": "city", "postCode": "22222", "state": "state", "country": "country" }, { "idPartner": 6, "name": "Partner 2", "domainSubTypeList": "1", "idParent": 0, "siteCode": "Partner2 Site Code", "buName": "Partner2 BU", "orgAddress1": "Partner2 Address1", "city": "city", "postCode": "22222", "state": "state", "country": "country" }, ... ] }, "partnerSystems": { // 5. serializer type "@type": "EntityImpExInMemStreamDto", // 6. entity domain "domainType": "PARTNERSYSTEM", // 7. entities exported according to the selection critria "entities": [ { // 8. source entity ID "idPartnerSystem": 213, // 9. source entity name "name": "CB US1-Generic", "partnerIdPartner": 6, "partnerSystemIdSystem": 19, "domain": "US Internal", "domainSubTypeList": "3" }, ... ] }, // ... // 10. TpaValues to extend imported entities "tpaValues": { // 11. DomainType - contains all TpaValues associated with this domain type "PARTNER": { // 12. DomainCode - all TpaValues for this DomainType-DomainCode combination "SapSalesOrg": { // 5. serializer type "@type": "TpaValueImpExInMemStreamDto", "domainType": "PARTNER", "domainCode": "SapSalesOrg", // 13. All TpaValues under DomainType-DomainCode "tpaValues": [ { // 14. Entity ID this is associated with "domainId": 3, // 15. property value "domainValue": "C1CA", "domainSubtypeName": "default", "inheritanceControl": "CanInheritAndEdit" }, { "domainId": 6, "domainValue": "C1US", "domainSubtypeName": "default", "inheritanceControl": "CanInheritAndEdit" }, ... ] }, // 12. DomainCode - all TpaValues for this DomainType-DomainCode combination "Description": { // 5. serializer type "@type": "TpaValueImpExInMemStreamDto", "domainType": "PARTNER", "domainCode": "Description", // 13. All TpaValues under DomainType-DomainCode "tpaValues": [ { // 14. Entity ID this is associated with "domainId": 3, // 15. property value "domainValue": "Sears Canada Smart Non, or other", "domainSubtypeName": "default", "inheritanceControl": "CanInheritAndEdit" }, { "domainId": 6, "domainValue": "Ulta - Adidas brand house 10001; CVS 360", "domainSubtypeName": "default", "inheritanceControl": "CanInheritAndEdit" } ] }, ... }, // 11. DomainType - contains all TpaValues associated with this domain type "BUSINESSCHANNEL": { // 12. DomainCode - all TpaValues for this DomainType-DomainCode combination "ISA12_VersionID": { // 5. serializer type "@type": "TpaValueImpExInMemStreamDto", "domainType": "BUSINESSCHANNEL", "domainCode": "ISA12_VersionID", "tpaValues": [ { // 14. Entity ID this is associated with "domainId": 809, // 15. property value "domainValue": "y", "domainSubtypeName": "default", "inheritanceControl": "CanInheritAndEdit" } ] }, ... }, ... }, // 16. Mappings used to pre-validate the import "mappings": { // 17. standrads required by the import "standardNames": [ "ansi/asc/x12", "generic", "sap" ], // 18. metadata codes grouped by domain types used by the import "metaDataCodes": { "PARTNER": [ "Description", "Division", ... ], "BUSINESSCHANNEL": [ "ISA11_StandardsID", "ISA12_VersionID", ... ], "PARTNERSYSTEM": [ "tests", ... ] }, // 19. domain subtypes grouped by domain types used by the import "domainSubtypeNames": { "SYSTEM": { "6": "default" }, "PARTNER": { "1": "default" }, "BUSINESSCHANNEL": { "4": "default" }, "PARTNERSYSTEM": { "3": "default" }, "CONTACT": { "2": "default" } } } } In the above JSON each relevant entry is preceded by a commented line which is prefixed by a number referenced in the following table to provide more context. Entry/Attribute Explanation Used for Validation Notes (1) Who generated the export Marker of the identity of who generated this export. This is used to validate the import and issue an error if this file is used for an import which does not match the export. For instance, you cannot call import for a business route passing for a partner export. yes DO NOT CHANGE. Changing can lead to disastrous consequences. (2) tpaValuesAreAttached Specifies if the TpaValue nodes are generated as part of the structure of the entities they extend (true), or as a flat list with a backpointer to the entity they are associated to yes DO NOT CHANGE. Changing can lead to disastrous consequences (3) childrenAreIncluded true if during the export children were requested yes DO NOT CHANGE. Changing can lead to disastrous consequences. If changed from false to true and child synchronization is required, then all children can be removed from parents. (4) exported entities/dependencies data structures containing the exported entities and the dependencies to support the import. Can be only one entry of a kind (cannot be two partner entries). The kind of entry is determined by the type of entity being exported no (5) serializer type Type of serializer used for the export. This dictates the farther structure of this object. Will have a different value for file serialization no DO NOT CHANGE (6) entity domain Domain type describing the entity. Used by the deserializer. no DO NOT CHANGE (7) list of exported entities List of exported entities for the specified domain type. This entry will be different for file serialization (not yet implemented) no (8) source entity ID ID of the exported source entity can be used for validation/resolution if this is a dependency DO NOT CHANGE (9) source entity name Name of exported entity. Used to establish the identity of the target entity. In some instances the identity of the target entity cannot be solely established based on the name. no (10) associated TpaValues TpaValues associated with the exported entities. Note that no TpaValue is exported for the dependencies. This entry can miss when TpaValues flatten mode is off. no (11) DomainType On export, if the flatten mode is on, TpaValues are grouped by domain types and by domain code. This helps for faster imports. DomainType is the highest level of grouping for TpaValues. no (12) DomainCode Describes subgroups of TpaValues under the parent DomainType. no (13) TpaValues for DomainType-DomainCode All TpaValues for a DomainType-DomainCode combination no (14) Source entity ID TpaValue associates with Backpointer to the imported entity ID this TpaValue node is associated with no Import fails if this ID is not amongst the imported entities of the specified type (15) Property value Value of the property set by this TpaValue no (16) Export Mappings Generated during the export and used to pre-validate the import and also to resolve dependencies. These values can be missing in which case they are picked up while the import is running. If this option is off it will make the import slower. yes - pre-validation (17) Required Standards Standards required by the imported objects yes - pre-validation (18) Metadata codes Used to pre-validate the imported TpaValues. Values are grouped by domain types. yes - pre-validation (19) Domain subtypes Referenced by extensible type entities. yes - pre-validation NOTE: when exporting Standards, MetaData and DomainSubtypes entries (17), (18) and/or (19) are not generated since they will show up in the exported objects section (4). Export EndPoints A typical export endpoint for parent entities will receive the following parameters: a request object specifying what objects to export. Most of the request objects as of this writing contain a list of IDs. In some instances, the selection criteria can be narrower. includeDependents - default true. This will cause all children to be exported. flattenMetadata - default true. Enables/disables the TpaValue flatten mode. If the endpoint addresses a leaf domain (no dependents/children) then the includeDependents is not part of the list of arguments. Every export endpoint outputs an ImpExEnvelopeDto object of a type unique to the export, which extends the basic type EntitiesImpExEnvelopeDto having the structure described at Export File Structure. Entity URL Request Object Type supports includeDependents Return Object Type Partner /api/v1/partners/export/ids PartnersExportRequestDto yes PartnersImpExEnvelopeDto PartnerSystem /api/v1/partners/system/export PartnerSystemsExportRequestDto yes PartnerSystemsImpExEnvelopeDto PartnerSystemMessages /api/v1/partners/system/message/export PartnerSystemMessagesExportRequestDto no PartnerSystemMessagesImpExEnvelopeDto Contacts /api/v1/partners/contact/export ContactExportRequestDto no ContactsImpExEnvelopeDto BusinessRoute /api/v1/routes/export BusinessRoutesExportRequestDto yes BusinessRoutesImpExEnvelopeDto BusinessRouteContact /api/v1/routes/contacts/export BusinessRouteContactsExportRequestDto no BusinessRouteContactsImpExEnvelopeDto BusinessRouteCodeSubs /api/v1/routes/codesub/export BusinessRouteCodeSubsExportRequestDto no BusinessRouteCodeSubsImpExEnvelopeDto Standards /api/v1/standard/export StandardsExportRequestDto yes StandardsImpExEnvelopeDto CodeSubstitution /api/v1/standard/codesub/export CodeSubstitutionsExportRequestDto no CodeSubstitutionsImpExEnvelopeDto MessageFormat /api/v1/messages/export MessageFormatExportRequestDto yes MessageFormatImpExEnvelopeDto MessageType /api/v1/messages/type/export MessageTypeExportRequestDto no MessageTypeImpExEnvelopeDto Environment /api/v1/environment/export EnvironmentsExportRequestDto yes EnvironmentsImpExEnvelopeDto System /api/v1/systems/export SystemsExportRequestDto no SystemsImpExEnvelopeDto TpmDomainSubType /api/v1/admin/metadata/subtype/export TpmDomainSubTypesExportRequestDto yes TpmDomainSubTypesImpExEnvelopeDto TpaMetaData /api/v1/tpa/export TpaMetaDataExportRequestDto no TpaMetaDataImpExEnvelopeDto TpmDataType /api/v1/metadata/datatype/export TpmDataTypesExportRequestDto no TpmDataTypesImpExEnvelopeDto NOTE: some of the request objects mentioned above accept a null or empty ID list in which case all objects of that type in the system will be returned. This usually applies to quasi-static objects which can be considered part of the infrastructure: environment, system, metadata definition, standards and standard code substitutions. Export Synopsis Export synopsis is a feature meant to bring a quick overview of the objects being exported. In all synopsis endpoints will return a SynopsisImpExEnvelopeDto instance which is a map whose key is the name of the entity type and the mapping is a list of exported entity names. In instances where the exported entity does not have a name, or the name is ambiguous without a context, a concatenation of attributes that might consitute a (synthetic) identity is returned instead. For instance for: TpaMetaData a name could be [Subtype:ABC]-[Type:PARTNER]-[Value:ABC1] TpmDomainSubtype the name is ExternalPartner [Type: PARTNER] - subtype name followed by the domain type it belongs to. CodeSubstitution the name is invoicetype [Code:380]->[Subst:PIE]-[StdFrom:UN/CEFACT]->[StdTo:OIOXML] - name followed by the substitution codes and standrads it subsitutes. etc. Export synopsis works similarly to regular exports with the following exceptions: does not support flattenMetadata parameter. does not output the names of the dependency entity object the exported entities depend on always produce the same output object NOTE synopsis objects are produced and entirely stored in memory therefore, like memory export mode, you have to be aware that it can create out-of-memory issues for large datasets. Entity URL Request Object Type supports includeDependents Partner /api/v1/partners/export/ids/synopsis PartnersExportRequestDto yes PartnerSystem /api/v1/partners/system/export/synopsis PartnerSystemsExportRequestDto yes PartnerSystemMessages /api/v1/partners/system/message/export/synopsis PartnerSystemMessagesExportRequestDto no Contacts /api/v1/partners/contact/export/synopsis ContactExportRequestDto no BusinessRoute /api/v1/routes/export/synopsis BusinessRoutesExportRequestDto yes BusinessRouteContact /api/v1/routes/contacts/export/synopsis BusinessRouteContactsExportRequestDto no BusinessRouteCodeSubs /api/v1/routes/codesub/export/synopsis BusinessRouteCodeSubsExportRequestDto no Standards /api/v1/standard/export/synopsis StandardsExportRequestDto yes CodeSubstitution /api/v1/standard/codesub/export/synopsis CodeSubstitutionsExportRequestDto no MessageFormat /api/v1/messages/export/synopsis MessageFormatExportRequestDto yes MessageType /api/v1/messages/type/export/synopsis MessageTypeExportRequestDto no Environment /api/v1/environment/export/synopsis EnvironmentsExportRequestDto yes System /api/v1/systems/export/synopsis SystemsExportRequestDto no TpmDomainSubType /api/v1/admin/metadata/subtype/synopsis TpmDomainSubTypesExportRequestDto yes TpaMetaData /api/v1/tpa/export/synopsis TpaMetaDataExportRequestDto no TpmDataType /api/v1/metadata/datatype/export/synopsis TpmDataTypesExportRequestDto no Import Endpoints Currently for each entity there are two type of import endpoints: one that accepts a JSON object. Usually the URL ends in import one that accepts a multipart file content. Usually the URL ends in import/file Each import endpoint accepts on input: the type of objects that has been created by its counterpart export. For import/file this object is replaced by a multipart file whose content is validated against the expected objects type. excludeDependents (defaults to false) that determines if children will be excluded from import (import only the parent entity). If in the export file the attribute childrenAreIncluded is false and excludeDependents is also false (we attempt to import children, but children were not exported), the import will fail. NOTE: this param is present only when importing entities with dependents/children synchronizeChildren (default is false) will synchronize the children (for more see Parent Child Synchronization Mode). This option is ignored if excludeDependents is true. NOTE: this param is present only when importing entities with dependents/children Entity URL Input Object Can Exclude Dependents? Partner /api/v1/partners/import/ids PartnersImpExEnvelopeDto yes Partner /api/v1/partners/import/ids/file MultipartFile yes PartnerSystem /api/v1/partners/system/import PartnerSystemsImpExEnvelopeDto yes PartnerSystem /api/v1/partners/system/import/file MultipartFile yes PartnerSystemMessages /api/v1/partners/system/message/import PartnerSystemMessagesImpExEnvelopeDto no PartnerSystemMessages /api/v1/partners/system/message/import/file MultipartFile no Contacts /api/v1/partners/contact/import ContactsImpExEnvelopeDto no Contacts /api/v1/partners/contact/import/file MultipartFile no BusinessRoute /api/v1/routes/import BusinessRoutesImpExEnvelopeDto yes BusinessRoute /api/v1/routes/import/file MultipartFile yes BusinessRouteContact /api/v1/routes/contacts/import BusinessRouteContactsImpExEnvelopeDto no BusinessRouteContact /api/v1/routes/contacts/import/file MultipartFile no BusinessRouteCodeSubs /api/v1/routes/codesub/import BusinessRouteCodeSubsImpExEnvelopeDto no BusinessRouteCodeSubs /api/v1/routes/codesub/import/file MultipartFile no Standards /api/v1/standard/import StandardsImpExEnvelopeDto yes Standards /api/v1/standard/import/file MultipartFile yes CodeSubstitution /api/v1/standard/codesub/import CodeSubstitutionsImpExEnvelopeDto no CodeSubstitution /api/v1/standard/codesub/import/file MultipartFile no MessageFormat /api/v1/messages/import MessageFormatImpExEnvelopeDto yes MessageFormat /api/v1/messages/import/file MultipartFile yes MessageType /api/v1/messages/type/import MessageTypeImpExEnvelopeDto no MessageType /api/v1/messages/type/import/file MultipartFile no Environment /api/v1/environment/import EnvironmentsImpExEnvelopeDto yes Environment /api/v1/environment/import/file MultipartFile yes System /api/v1/systems/import SystemsImpExEnvelopeDto no System /api/v1/systems/import/file MultipartFile no TpmDomainSubType /api/v1/admin/metadata/subtype/import TpmDomainSubTypesImpExEnvelopeDto yes TpmDomainSubType /api/v1/admin/metadata/subtype/import/file MultipartFile yes TpaMetaData /api/v1/tpa/import TpaMetaDataImpExEnvelopeDto no TpaMetaData /api/v1/tpa/import/file MultipartFile no -------- TpmDataType /api/v1/metadata/datatype/import TpmDataTypesImpExEnvelopeDto no TpmDataType /api/v1/metadata/datatype/import/file MultipartFile no More Info More info on the structure of the request objects used in by the endpoints can be found on the swagger page at the URL path (relative to the TPM server URL): swagger-ui/index.html?urls.primaryName=default#/ Export/Import in the TPM Web UI For exports the web interface currently supports only one entity exports (with or without the dependents). In this sense two menu items are added for each entry of a domain that has dependents in any of the domain tables: - Export - Export with Dependents Selecting Export with Dependents will bring up the synopsis dialog for a quick preview of what is going to be exported: Depending on the situation, the save will generate a file with a single top item. In the above example only one partner is going to be generated in the saved file. The file will also contain all direct and indirect children as well as dependencies that have to be resolved during the import. Imports, form the UI perspective, are contextual operations. In this sense, import invocation form the Partner screen will require for import a file which was produced by exporting a partner. If the file is of a different type, then the import will fail with a message pointing out to this requirement. Clicking import button for a domain that can have dependents will bring up the following dialog which allows to specify if the whole item including dependent should be imported, or just the item without any children: At this point, in order to take advantage of all the features provided by export/import you may want to use the REST API.

TPM 9 comes up with a new set of cool features and improvements that target usability, performance, data coherence, and data transfer flexibility. TPM 9 is substantially improved based on our customers' needs and experience, on our observation and understanding of TPM use patterns, as well as through adopting newer industry standards. TPM continues as we speak to go through an evolutive process whose goal is to provide a better experience on all areas of concern. Modernized Architecture and UI for Better Performance and Usability Dramatically improved the ease of accessing data. Now data can be entered following a more natural flow providing relevant contextual information. There are several concepts we borrowed from the programming world leading to a better conceptualization of the stored data. Great strides were made in improving the server performance and eliminate data incoherence. Improved Support for More Complex Scenarios The new TPM is geared to fulfill complex customer use scenarios like data loads, data transfers between different system etc. This is an area where we made significant improvements and continue to do so with a number of cool features implemented or being implemented as we speak. REST APIs for Automation Management a good number of APIs was introduced, and this number continues to grow to support different scenarios like automation of more complex scenarios, the REST APIs are meant to be used by any TPM REST client, and perhaps the most important so far is iSM, which until recently had direct access to the TPM database iSM being the de facto TPM server for most of the uses cases. with the advent of the new TPM server and the TPM REST API exposure, many operations are, and will be possible without the need to intimately know the internal structure of the TPM database, thus forcing the business rules check on the server side rather than on the client/caller side. The end effect is a huge increase in the usability, shifting of concerns on the business purpose TPM would solve in a ecosystem rather than data migrations. REST APIs also makes TPM a perfect candidate as a microservice with easy integration in cloud ecosystems. Improved Export/Import The core of import/export became one of the cornerstone features of the new improved TPM: It is used to accomplish a number of features that increase the ease of use from both UI and API Automation perspective. Imports, besides the fact that are checked, can now update existing data domains and, if desired, synchronize them. For instance, transport of data from one TPM system to another is greatly simplified. New import/export can be done in a very discrete way, allowing insert/update/delete operations not only on individual elements, but whole hierarchical data sets. New import is also optimized for performance allowing import of very large data sets in a greatly reduced time and memory requirements. This feature continues to grow as we speak based on our understanding of the product utilization and practical use cases which are triggered by actual customer requirements and practices. In this sense we are open to any suggestion that can lead to substantial product improvements. Initial Database Load. Custom Database Changes TPM provides a new mechanism to run SQL scripts each time the TPM server starts. Here are some scenarios that can be covered using this mechanism: initial custom database load for situations where the database needs to be primed with a custom set of predefined data tailored for specific setups. For instance, when you plan to distribute TPM on multiple machines, or to multiple customers, where you have a predefined set of metadata types, standards, formats, message type, systems etc. This will preclude the need to manually enter the data (an thus human error) for every single distribution. Of course, such scenario could also be covered using the import feature with a predetermined set of import data packages. any custom database changes can be applied as needed. For instance, you want to apply a database wide change (could look at this as a data patch), like renaming certain objects, enhancing the set of existing metadata types, removing some properties or objects etc. This mechanism would help to automate such processes in situations where your routing engine along with the TPM server is distributed to multiple clients. Metadata Extensibility While previous versions of the product provide a domain/type extension mechanism, the new TPM improved on this concept by allowing definition of named custom sets of properties to extend the default domain property set. The new approach can cover new scenarios where almost every entity in the system can have an identity that attaches specialized meaning thus helping in a complex system with the separation of concerns. For instance, now a Partner may not just be "any partner", but through type extension it can be seen as a partner whose function is suggested and supported by the extension type assigned to it. Metadata Quick Preview The new TPM brings an improved way of visualizing the relevant custom metadata properties by eliminating the need to drill down into the entity itself. This mechanism can offer a synoptic view of multiple entities custom metadata greatly improving the user experience.