A data domain (domain) refers to a data model, such as student or course, and is defined by the parties responsible for data governance. The data domain defines the model schema and other attributes for that focus. Common examples of data domains are customers, students, employees, parts, and product orders. Data domains are typically created by the Data Governance Steward (DGS).
...
Federated: Federated domains do not store their data in YOUnite but retrieve and update the data on the systems in which it resides. For example, MIS, ERP, or CRM systems. Federated domains require adaptors, metadata, and governance configurations and are covered in detail *TODO: TUTORIAL ON FEDERATED*. Accessing federated data is covered on Accessing Data Records,
Domain Model Schemas
A domain Model Schema refers to the attributes (properties), format, and other metadata that defines how a specific domain should expect to store the data (either in the YOUnite Data Store or Federated), for the purposes of standardizing how data is exchanged between systems. The Data Governance Steward is responsible for configuring and maintaining domain model schemas. A domain model schema is a JSON object describing/defining the properties for the domain's schema. The root node of the model schema is the properties element. See Valid Property Names and Valid Types 949725525 and 949725525 for ModelSchema Properties below.
...
property | required | valid values | description |
---|---|---|---|
name | yes | Must be between 2 to 128 characters long and must start with an alpha character. The | The domain name. Must be unique to the entire YOUnite deployment since domains are typically shared. |
description | no | 0 to 255 characters long. If longer it will be truncated. | A human readable description of the domain. |
zoneUuid | noyes | Owning zone's domain UUID | The zone that the domain, the domain's versions, and its data records will be tied to. If this is omitted the caller's current zone will be used. Note that the caller must have permissions to create a domain. |
domainType | no | MDM_DATA_STORE or FEDERATED | The domain type can be either: MDM_DATA_STORE, which is when the domain's data records are stored in YOUnite. This is used when the entire organization is comfortable or mandated to migrate a domain to a single store. MDM_DATA_STORE is optimal for reference data such as a list of states, countries, zip-codes, etc (default). FEDERATED domains do not store their data in YOUnite, but reference and update data on the systems in which it resides. Federated domains require adaptors, metadata, and governance configurations that are covered in detail. |
...
The root node of the model schema is the properties
element. See Valid Property Names and Valid Types for Model Schema Properties below for details.
A domain version is defined with a domain JSON Object as described below:
...
Code Block | ||
---|---|---|
| ||
{
"modelSchema": {
"properties": {
"<property-name>": {
"type": "<property-type>",
...item1 properties....
},
"<property-name>": {
...
}
}
},
"description": "<description>",
"drLabel": "<property-name>",
"fastDuplicateDetectionPropertiess": "property-name, [, <property-name2>, ...]"
} |
...
property | required | valid values | description |
---|---|---|---|
modelSchema | yes | See Model Schema Properties and Post a Domain 949725525 and 949725525 below for details. | A JSON model describing the schema for the data domain; it defines the properties that make up the domains schema. The root node of the model schema is the properties element. |
description | no | 0 to 255 characters long. If longer it will be truncated. | A human readable description of the domain version. |
fastDuplicateDetectionProperties | yes | A list of of one or more valid properties for the given domain version. Each property must be of type either String. Number, Int or Boolean. | A list of properties that will insure that a given data record is unique for the domain version:
|
Once the domain/version has been created you can POST data records to, and retrieve data records from, the domain.
...
- fastDuplicateDetectionProperties
- Federated Domains: Uniqueness Rules & Required Properties
- Valid property names
- Valid property types
Fast Duplicate Detection Properties (FDDPS)
Anchor | ||||
---|---|---|---|---|
|
Each domain must have a duplicate detection property (fastDuplicateDetectionProperties
). Fast Duplicate Detection Properties (FDDPS) are attributes of a domain and identify those fields that, when combined, MDM should use to detect whether the data record is unique. Refer to Domain Uniqueness, Deterministic Uniqueness, and Probabilistic Uniqueness for more information.
Each domain must have a data record label (drLabel
). The data record label acts as a primary key for the domain. For example, the "states" domain above uses the abbreviation property as the display property, Use the /drs endpoint and the appropriate domain and display property to GET a data record:
GET /drs?filters=name:states,displayProperty:CA
If there are multiple versions of a domain, and a domain other than the default
GET /drs?filters=name:states,displayProperty:CA
If there are multiple versions of a domain, and a domain other than the default is needed, the version number can be included in the URI. For example, assume there are three versions of the "states" domain and the current version is version 3. The consumer can retrieve the California version 1 data record by using the following:
GET /drs?filters=name:states,version:1,displayProperty:CA
11/17/17: Per Robbie, drLabel replaced displayProperty, yet is optionaloptiona - TO DIANA FROM MARK - We are going to ditch drLabel and just use FDDPsl.
drLabelfastDuplicateDetectionProperites": {
"description": "The optional property that serves as a label for data records defined by the domain version. It also can be used as a filter in the API's GET resource.",
...
GET /drs?filters=name:states,version:1,displayProperty:CA
Note: See Posting a Data Record and Retrieving a Data Record sections for further details on posting/retrieving data records.
Fast Duplicate Dectection Properites Rules
Fast Duplicate Detection Properties are attributes of a Domain and identify those fields that, when combined, MDM should use to detect whether the DR is unique. Refer to Domain Uniqueness, Deterministic Uniqueness, and De-Duplication for more information.
- The value provided for the
displayProperty
must be unique between all domain entries of a given domain type (e.g. each entry in the "state"
domain must have a uniquestateName
). - Only one property in a domain can be the
displayProperty
. If more than one property is required to ensure uniqueness see the Uniqueness Rules Property below. - Display properties are limited to type STRING.
- Properties designated as the
displayProperty
are required; i.e. null values are not allowed. - Display properties are case sensitive e.g. "California" is NOT equal to "california".
...
Property | Description | ||||||
---|---|---|---|---|---|---|---|
enumType | Types of enums:
| ||||||
xrefLocation | If For example, if a version of a domain "countries"contains a list of data record values that represent all countries then a domain can reference it by specifying an enumeration type of
A subset of the data record values in the domain can be specified using the | ||||||
data | An array containing a comma-separated list of enum values. e.g. Is a three-item string enumeration. If the
Assuming the display values for Canada, the United States, and Mexico are those shown above in the data elements. | ||||||
default | If the item is not provided or is null, the default value is used. | ||||||
required | A non-null value for this item must be provided (false by default). | ||||||
description | A human-readable description of the property. |
URI
TODO: "xref" type and "xrefLocation" has been moved to a new type called "xref". Documentation needs to be updated.
A properly formed Uniform Resource Locator (URL). A URL is a type of Uniform Resource Identifier (URI). By default, a property of type uri
can be any valid URI or it can be limited by a regex pattern or to a domain cross-reference (xrefLocation:"xref").
...
Property | Description |
---|---|
minItems | Minimum items allowed in the array. |
| Maximum items allowed in the array |
default | If the item is not provided or is null, the default value is used. |
items | Contains the list of sub-properties in the node. |
required | A non-null value for this item must be provided (false by default). |
description | A human-readable description of the property. |
Note: Updating arrays can be handled differently when the domain is federated and an "Array Advisory Practice" should be developed by the ZDS/DGS and communicated to developers.
For example, this can be illustrated using an array of phone numbers… e.g. mobile, home, work, home-fax, etc.
When an adaptor detects a change in a source entities' array of phone numbers, it can send up the entire array or just the modified phone number in the array. In either case, the router will forward it based on the ACLs. The recipient can take these changes and add or update existing entries based on the phone number type property.
Since adaptor developers can handle arrays in one of these two different ways, it is recommended that the ZDS/DGSs make a decision about the preferred method, an "Array Advisory Practice," and communicate this to the adaptor developers.
Data Governance (DG) may say:
Never send a partial array of phone numbers but only the entire set so the entire set at the receiving adaptor can do a complete delete and replace on all phone numbers.
— OR —
Just send updates to an array only and the receiving adaptors can process them as they see fit and handle them using an index (which would be an agreed-upon domain version property; in this example it would be “type” (phone number type)).
An example of a good way for DG to articulate this would be in the “description” field for the phone number array in the domain version modelSchema.
Code Block | ||
---|---|---|
| ||
{ "properties": { "name": { "type": "string", "min": 2, "max": 80 }, "phone": { "type": "array", "minItems": 1, "maxItems": 50, "description": "An array of phone numbers - ADAPTOR DEV ADISORY - Sending/receiving individual phone numbers is OK - sending/receiving the entire list is not necessary ", "items": { "type": { "type": "string", "min": 2, "max": 80, "regex": "^[A-Za-z]+" }, "number": { "type": "number", "min": 2, "max": 10 } } } } } |
...
- POST the Domain
- POST a Domain Version
POST the Domain Domain
A domain can be either FEDERATED or MDM_DATA_STORE (stored in YOUnite's DB). For example, to create a simple MDM_DATA_STORE "states" domain :
POST /domains
Code Block language js { "name": "states", "description": "A reference domain of states that should be referenced by all applications in the local YOUnite ecosystem", "zoneUuid": "a1aca070-846f-44e5-9471-c73b46c35f4a", "domainType": "MDM_DATA_STORE" }
The location header returned provides the URI for POSTing a domain version below.
e.g. Location /domains/7f28180b-7d9f-42b5-b5ed-d4a0e7ec09fc
POST a Domain Version
...
The response code on success is: 201 CREATED
Posting Data Records to the MDM_DATA_STORE Domain
Anchor | ||||
---|---|---|---|---|
|
Once the domain/version has been created, data records can be POSTed to it using the /dr endpoint:
...
Code Block | ||
---|---|---|
| ||
{ "name": "states", "version": 1, "json": "{ \"name\": \"California\", \"abbreviation\" : \"CA\"} } |
Retrieving Data Records from
...
MDM_DATA_STORE Domains
Anchor | ||||
---|---|---|---|---|
|
The data records for the default version of a domain domain (and their respective UUIDs) can be retrieved with the following:
POST GET /drs?filters=name:<domain-name>
Retrieving the data records (and their respective UUIDs) for a given domain and version:
POST GET /drs?filters=name:<domain-name>,version:<version>
Retrieving a specific data record for a MDM_DATA_STORE domain
GET /drs/<dr-uuid>
Posting Data Records with a FEDERATED Domain
Anchor | ||||
---|---|---|---|---|
|
Adaptors capable of a given domain version detect changes in the source system they are connected to then, send them to YOUnite. YOUnite considers the domain and adaptor where the change was detected and, applies the appropriate governance and routes the data event to the other adaptors in the YOUnite ecosystem.
Retrieving Data Records with a FEDERATED Domain: Data Record Assembly
Anchor | ||||
---|---|---|---|---|
|
Since YOUnite does not store the data records for federated data domains, it cannot return the entire data record in lists as it does with MDM_DATA_STORE domains. The first two requests below will return only the FDDPs and UUIDs for the data records in a domain. The third request shows to retrieve an assembled data record using a POST request. See the descriptions below for more details:
The FDDPs for data records or the default version of a domain (and their respective UUIDs) can be retrieved with the following:
GET /drs?filters=name:<domain-name>
Retrieving the FDDPs for the data records (and their respective UUIDs) for a given domain and version:
GET /drs?filters=name:<domain-name>,version:<version>
Make a request for a specific federated data record. Since YOUnite must make requests to the adaptors where the data resides and assemble the data, the request payload must include a "callbackUrl" so YOUnite can deliver the payload (see http://youite.us/api). The POST request will return with a location that can be followed to check on the status of the request:
POST /drs/<dr-uuid>/assembler