Adaptors

Adaptor Intro: An introduction to YOUnite and Adaptors can be found on the Introduction to YOUnite page

Adaptor Configuration: How to configure and manage adaptors is covered on the Managing Adaptors page.

Adaptor Development: Developing adaptors can be found on the YOUnite Adaptor Guide for Java Developers page.

Adaptors and Federated MDRs

In order to provide data management, the YOUnite Data Hub needs to know where the data domain and properties live. Federated Master Data Records (MDRs) are stored in the source system (federated domain) instead of centrally in the Data Hub. The Data Hub contains a reference to the federated domain so it can retrieve the data. In order for this to happen the federated domain elements must be mapped to the YOUnite Data Hub. This mapping can happen organically or by creating a master list of all source system entities that need to be mapped to YOUnite. 

The mapping process includes creating YOUnite data records for each domain entity using the POST /drs request. YOUnite will then retain the uniqueness rules for that data record. Once the data records exists in YOUnite, the source entities connected to YOUnite can be mapped through adaptors . The adaptor for the domain must be paused while you map the entities in the systems and services attached to the adaptors to the YOUnite data records, linking them together.

See Mapping for Federated Domains for details. 

Adaptors Can Be Either Uni-Directional or Bi-Directional

Uni-Directional - Inbound Only

An inbound uni-directional adaptor connects the YOUnite datahub to a source system in the following way:

• Accept data events from the YOUnite datahub
• Transform data events from the data domain format to the source system's native format
• Is capable of performing one or more CRUD event on source entities in the source system

Uni-Directional - Outbound Only

An outbound uni-directional adaptor connects the YOUnite data to a source system in the following way:

• Detects change events to source entities in the source system
• Transform data change events from the source system's native format to the data domain format 
• Sends the change data event to the YOUnite datahub for routing to other adaptors based on applicable governance ACLs.

Bi-Directional - Inbound and Outbound

A bi-directional adaptor performs all the functions of both an uni-directional inbound and outbound adaptor.

When developing a bi-directional adaptor, its imperative that the developer understand the "Infinite Update Issue" discussed below. A well written equals() method may be required to avoid this issue.

The Infinite Update Issue

The Importance of  Strong Eventing in Source Systems and the equals() Method With Bi-Directional Adaptors

Infinite updates can occur when bi-directional adaptors are required for source systems that have less than optimal change event notification APIs (eventing).

A bi-directional adaptor can be thought of as having a right and left brain where each side isn't always aware of all the goings-on in the other side. 

With a bi-directional adaptor, one side of the adaptor accepts data events from YOUnite and applies them to the source system (MDM side)  while the other side waits to be notified of change events from the source system and sends the data events to YOUnite (source system side). Optimally, the source system has robust APIs that allow the adaptor developer to listen and be notified of specific change events inside of the source system (i.e. robust eventing).

However, if the source system has weak eventing then the adaptor developer needs to poll for changes instead of being notified. When the MDM side of the adaptor receives a data event from YOUnite and writes it to the source system, the source-system side detects the change but has no way of knowing if this was a change initiated by YOUnite or by the source system itself (e.g. a new user added to the customer DB via its normal business logic).

In the case of a source system with week eventing, as changes are received from YOUnite, they need to be stored in a short-term Change Detection Cache in the SDK (TBD). When the change is detected, the adaptor can look in the cache to see if it is equals(), and if it is, it can then be ignored.  The entry is then removed from the short-term cache. 

When an adaptor propagates changes made by itself then an infinite update loop in the MDM ecosystem can occur and create "governance chaos" as shown in the following scenarios:

• All adaptors are bi-directional
• Governance is set so that:
  • adaptorA shares with adaptorB, but not adaptorC
  • adaptorB shares with both adaptorA and adaptorC
• A source entity attached to adaptorA has a change to its data record, adaptorA detects it and sends it to the datahub
• According to adaptorA's governance rules, the change is routed to adaptorB, but not adaptorC 
• AdaptorB applies the change to the appropriate source entity in the source system attached to adaptorB
• Because AdaptorB does not have an equals() method and the change detection logic has no way of knowing that the adaptor itself wrote the change; consequently,  adaptorB will push the change it just wrote up to YOUnite as a new data event
• AdaptorB's governance allows it to send changes to adaptorA and adaptorC
• So even though the change originated at adaptorA, adaptorC received the change – a violation of the intended governance.
• Worse, adaptorA receives its own change and if it too doesn't have an equals() method the changes will continue infinitely.

Adaptor Design Pitfalls

Having two uni-directional adaptors (one inbound only and the other outbound only) should be avoided unless the source system has strong eventing that allows the adaptor to filter out changed initiated by itself.  Otherwise it will be difficult to avoid the "Infinite Update Issue" without adding special caches to implement the equals() method described above. 

Selecting Enterprise Applications That Can Meet Your Organization's Governance Requirements

Developing adaptors can be the most difficult task in the MDM development and implementation phases. While it is unlikely an organization will swap out an existing enterprise applications to meet its governance needs (although, it may be required in some circumstances), an organization can require new applications and vendors meet certain MDM and data governance requirements so that their solutions  can be made MDM/Governance aware.

To meet the needs of MDM and governance, new applications and solutions must have robust APIs that allow:

• Notification for source entity updates that map to the organization's data domain definitions. (Read carefully "The Infinite Update Issues" above especially regarding the need for robust eventing.)  Adaptors should not have to poll a service for changes but should be able to register for change notifications for entities that map to the organization's data domain definitions
• Eventing and source entity CRUD operations should be at the data domain property level. This is a critical distinction because if the API only allows change detection and source entity updates at the source record level then governance can only be applied to the source entities in the system at the data record level. Ideally, an organization will want to detect changes and perform CRUD operations at the domain property level. This is the difference between allowing a customer record from being shared or restricted (record level) versus being able to restrict individual properties (property level) such as tax ID numbers, phone numbers, etc.
• Source entity CRUD operations:
  • Create
  • Update
  • Delete
  • Get