YOUnite groups an organization'sresources by the organization's structure (e.g. divisions, departments, districts, etc) and uses these groupings to create relationships within the organization. With YOUnite these groupings are called zones.
It's important to understand the distinction that:
- Access to resources is granted through permissions
- Access to data is manged through scopes (covered in the Scopes page).
Zones
As mentioned above, YOUnite provides zones so an organization can group master data resources along its organization structure.
Zones are associated with each other in a hierarchical structure with parent, child and sibling zones e.g. the following illustrates a college district as the parent zone with three child college zones (siblings of each other):
Zone characteristics:
- Zones generally have two types of users associated with them:
- Zone admin is responsible for general zone management. A zone admin is defined when the zone is created.
- Zone data steward is responsible for the domains, data and data governance.
User types are defined by polices. Polices are covered below.
- Zones receive notifications of master data changes and operational events.
- Zones can have zero, one or more adaptors tied to services containing federated master data.
- The Zone Admin controls access to zone resources with the exception of the master data and domains
- The Zone Data Steward controls access to the master data associated with a zone
- The Zone Data Steward can restrict either out-bound or in-bound data shared with or from other zones.
- Centralized YOUnite logs are indexed on a per-zone basis
- A Zone Data Steward can define and share data domains (domains) but generally a single top-level domain creates domains for the entire YOUnite deployment.
The Ultimate Root Zone
Upon initial deployment, YOUnite creates a root zone called root with a zone admin user mdmadmin. All zones created are subordinate to it. The UUID of the root zone is always 6c5a754b-6ce0-4871-8dec-d39e255eccc3. The root zone's UUID was necessary when creating the "College District" zone below:
Zone Users
A zone is created by a user with an SSO ID (TODO See YOUnite and SSO Providers). The zone creation requires an SSO ID. The first zone user is tied to this zone and becomes the zone's Zone Admin (more on user types and permissions to follow). The Zone Admin has full administrative privileges for the zone.
If this is the first zone associated to a SSO ID, a YOUnite User (User) is created that is tied to the associated SSO ID.
For example, if the IT admin created above for the college district zone (senor_jefe@college_district.edu) creates a zone called "Central College" and assigns Cece Jones SSO ID to it as the IT admin for the zone, then a new YOUnite User (cece@college_district.edu) is created and she will be associated with the "Central College" zone:
If two more zone's are created and Cece is associated with them, the same YOUnite User name is used for all associated zones. So now the one YOUnite User (with SSO ID cece@college_district.edu) is associated with three zones:
The User's permissions are specific to the zone they are logged into so the YOUnite User's permissions can be different from one zone to another. This is accomplished using permissions, roles and groups.
Permissions
YOUnite users can be granted or denied access to resources by setting appropriate permissions. Permissions do not stand on their own but are grouped into Polices (which are explained below). Generally permissions are managed by the Zone Admin or other users that have been given control over a zone's permissions.
Permissions are broken out into two properties:
- Resource URI: The YOUnite resource that is part of the user's zone. If the zone user has the appropriate permissions, they can allow other users to access a resource such as a domain, MDRs, logs, adaptors, etc.
- Actions: These are the actions the user can perform on the resource. They include GET, PUT, POST, DELETE, PATCH (and ALL).
Permission Example
In this example a user is granted full access to all domains in the zone except for:
- staff: The user has no access to the staff domain
- students: The user has full access except DELETE to the students zone
Any user with this permission can create, modify, delete and view all other user's for a zone.
Rolies
A role is a group of permissions that can be used to manage user access to resources. YOUnite has two default managed roles. These two managed roles are visible to all zones and can not be deleted.
Typically there are two types of users associated with a zone that leverage these roles:
- Zone Admin: These users have zone admin privileges. As mentioned above, the first Zone Admin has full administrative privileges for the zone but additional roles and permissions can be configured that restrict access to other Zone Admins. The Zone Admin has general zone management responsibilities such as creating subordinate zones, adding adaptors, creating/managing groups and Users and attaching Roles to Groups and Users.
- Zone Data Steward (ZDS): Zone Admins can create additional Users with data steward permissions. These Users have access and manage control (scope) to the master data (covered in the Scopes & Metadata and Metadata pages).
See the YOUnite API documentation for more specific on roles.
Groups
A Group is a collection of Users in a zone that has roles associated with it. A Group can have multiple roles associated with it and Users can belong to more than one group.
Effective Permissions
In particular zone, the User's effective permissions are a union of all the permissions associated with all of the groups they are in and any roles directly associated with them.
The following diagram pulls all of the above topics together and shows how the user's effective permissions are calculated
See the YOUnite API documentation for more specifics on zones, users, groups and roles.
Notifications
TODO
Logging
TODO
API Users
An API consumer for the zone can register a callback URL to receive notifications of events such as completed transactions. TODO
Adaptor Intro: An introduction to YOUnite and Adaptors can be found on the Introduction to YOUnite page
Adaptor Configuration: How to configure and manage adaptors is covered on the Managing Adaptors page.
Adaptor Development: Developing adaptors can be found on the YOUnite Adaptor Guide for Java Developers page.
An adaptor is software that is located within a system that is participating in the sharing of data through the YOUnite Data Hub (via the Message Broker) and acts as the connection point between that system and Data Hub. It focuses on ETL (Extract, Transform, and Load) functions, ensuring the outbound data from that system meets the format requirements of the Data Hub and transforming the inbound data from the Data Hub into what any other system requires. An adaptor may have additional business logic such as filtering for specific data from the Data Hub.
Adaptors perform data operations (CRUD) on data records that conform to one or more data domains (for more information on data domains see the Data Domains page). The data operations an adaptor can perform are referred to as its "Capabilities" and are defined in an Adaptor Capabilities List.
The white ellipse in the diagram below represents a JSON data object being passed between the adaptors and the Data Hub. The data object could have originated in one of several ways. The most common are:
- By an API POST /drs/assembler request made by an API consumer to the Data Hub
- An adaptor detecting a change and sending it to the Data Hub
- The Data Hub receiving a change from an adaptor and routing it out to other adaptors and API consumers that have subscribed to receive notifications for that specific change
- The Data Hub requiring data from an adaptor
Adaptors and Federated MDRsAnchor adaptorsFederatedMdr adaptorsFederatedMdr
| adaptorsFederatedMdr | |
| adaptorsFederatedMdr |
In order to provide data management, the YOUnite Data Hub needs to know where the data domain and properties live. Federated Master Data Records (MDRs) are stored in the source system (federated domain) instead of centrally in the Data Hub. The Data Hub contains a reference to the federated domain so it can retrieve the data. In order for this to happen the federated domain elements must be mapped to the YOUnite Data Hub. This mapping can happen organically or by creating a master list of all source system entities that need to be mapped to YOUnite.
The mapping process includes creating YOUnite data records for each domain entity using the POST /drs request. YOUnite will then retain the uniqueness rules for that data record. Once the data records exists in YOUnite, the source entities connected to YOUnite can be mapped through adaptors . The adaptor for the domain must be paused while you map the entities in the systems and services attached to the adaptors to the YOUnite data records, linking them together.
See Mapping for Federated Domains for details.
Adaptors Can Be Either Uni-Directional or Bi-Directional
Uni-Directional - Inbound Only
An inbound uni-directional adaptor connects the YOUnite datahub to a source system in the following way:
- Accept data events from the YOUnite datahub
- Transform data events from the data domain format to the source system's native format
- Is capable of performing one or more CRUD event on source entities in the source system
Uni-Directional - Outbound Only
An outbound uni-directional adaptor connects the YOUnite data to a source system in the following way:
- Detects change events to source entities in the source system
- Transform data change events from the source system's native format to the data domain format
- Sends the change data event to the YOUnite datahub for routing to other adaptors based on applicable governance ACLs.
Bi-Directional - Inbound and Outbound
A bi-directional adaptor performs all the functions of both an uni-directional inbound and outbound adaptor.
When developing a bi-directional adaptor, its imperative that the developer understand the "Infinite Update Issue" discussed below. A well written equals() method may be required to avoid this issue.
The Infinite Update Issue
The Importance of Strong Eventing in Source Systems and the equals() Method With Bi-Directional Adaptors
Infinite updates can occur when bi-directional adaptors are required for source systems that have less than optimal change event notification APIs (eventing).
A bi-directional adaptor can be thought of as having a right and left brain where each side isn't always aware of all the goings-on in the other side.
With a bi-directional adaptor, one side of the adaptor accepts data events from YOUnite and applies them to the source system (MDM side) while the other side waits to be notified of change events from the source system and sends the data events to YOUnite (source system side). Optimally, the source system has robust APIs that allow the adaptor developer to listen and be notified of specific change events inside of the source system (i.e. robust eventing).
However, if the source system has weak eventing then the adaptor developer needs to poll for changes instead of being notified. When the MDM side of the adaptor receives a data event from YOUnite and writes it to the source system, the source-system side detects the change but has no way of knowing if this was a change initiated by YOUnite or by the source system itself (e.g. a new user added to the customer DB via its normal business logic).
In the case of a source system with week eventing, as changes are received from YOUnite, they need to be stored in a short-term Change Detection Cache in the SDK (TBD). When the change is detected, the adaptor can look in the cache to see if it is equals(), and if it is, it can then be ignored. The entry is then removed from the short-term cache.
When an adaptor propagates changes made by itself then an infinite update loop in the MDM ecosystem can occur and create "governance chaos" as shown in the following scenarios:
- All adaptors are bi-directional
- Governance is set so that:
- adaptorA shares with adaptorB, but not adaptorC
- adaptorB shares with both adaptorA and adaptorC
- A source entity attached to adaptorA has a change to its data record, adaptorA detects it and sends it to the datahub
- According to adaptorA's governance rules, the change is routed to adaptorB, but not adaptorC
- AdaptorB applies the change to the appropriate source entity in the source system attached to adaptorB
- Because AdaptorB does not have an equals() method and the change detection logic has no way of knowing that the adaptor itself wrote the change; consequently, adaptorB will push the change it just wrote up to YOUnite as a new data event
- AdaptorB's governance allows it to send changes to adaptorA and adaptorC
- So even though the change originated at adaptorA, adaptorC received the change – a violation of the intended governance.
- Worse, adaptorA receives its own change and if it too doesn't have an equals() method the changes will continue infinitely.
Adaptor Design Pitfalls
Having two uni-directional adaptors (one inbound only and the other outbound only) should be avoided unless the source system has strong eventing that allows the adaptor to filter out changed initiated by itself. Otherwise it will be difficult to avoid the "Infinite Update Issue" without adding special caches to implement the equals() method described above.
Selecting Enterprise Applications That Can Meet Your Organization's Governance Requirements
Developing adaptors can be the most difficult task in the MDM development and implementation phases. While it is unlikely an organization will swap out an existing enterprise applications to meet its governance needs (although, it may be required in some circumstances), an organization can require new applications and vendors meet certain MDM and data governance requirements so that their solutions can be made MDM/Governance aware.
To meet the needs of MDM and governance, new applications and solutions must have robust APIs that allow:
- Notification for source entity updates that map to the organization's data domain definitions. (Read carefully "The Infinite Update Issues" above especially regarding the need for robust eventing.) Adaptors should not have to poll a service for changes but should be able to register for change notifications for entities that map to the organization's data domain definitions
- Eventing and source entity CRUD operations should be at the data domain property level. This is a critical distinction because if the API only allows change detection and source entity updates at the source record level then governance can only be applied to the source entities in the system at the data record level. Ideally, an organization will want to detect changes and perform CRUD operations at the domain property level. This is the difference between allowing a customer record from being shared or restricted (record level) versus being able to restrict individual properties (property level) such as tax ID numbers, phone numbers, etc.
- Source entity CRUD operations:
- Create
- Update
- Delete
- Get