A data domain (domain) is a defined grouping of properties to allow the users to view the data with a perspective that highlights the particular properties of the data. The domain defines the model schema (model) and other properties for that specific focus. Common examples of data domains are customers, students, employees, parts and product orders.
YOUnite doesn't restrict the data architect to a single data domain such as customers (single domain MDM) but allows the architect to model for multiple types (customers, students, employees, parts, product orders, etc) as part of a single solution.
Once a version of a domain is created and its data records (DRs) are loaded (or mapped in a federated model), it can be referenced by other data domains, adaptors and by API consumers as a source of truth. Domains have version numbers so that other domains, adaptors and applications can bind to a specific domain and version (e.g. students:v3
). Data governance can designate data at specific adaptors as the master data and can control which zone has the appropriate ACLs to access and update it.
Domain Types
The data records for a domain can be stored either as:
MDM Data Store: When the domain data records are stored in YOUnite. This is used when the entire organization is comfortable or mandated to migrate a domain to a single store. The YOUnite data store is optimal for reference data such as a list of states countries, zip-codes, etc.
Federated domains do not store their data in YOUnite but retrieve and update the data on the systems in which it resides. For example, MIS, ERP or CRM systems. Federated domains require adaptors, metadata, and governance configurations and are covered in detail *TODO: TUTORIAL ON FEDERATED*
Domain Model Schemas
A domain model schema (model) is a JSON object describing the schema for a data domain; it defines the properties that make up the domain's schema. The root node of the model schema is the properties element. See Valid Property Names and Valid Types for ModelSchema Properties below.
POST the Domain
The first step in creating a domain is to define the domain name. You will have to also define the new domain's type and the zone it is attached to or its "owning zone". If you do not define the type and owning zone, then YOUnite will be do it for you by default):
POST /domains
{ "name": "<domain name>", "description": "<human readable description of the data domain>", "zoneUuid": "<owning zone uuid>", "domainType": "<domain type>" }
property | required | valid values | description |
---|---|---|---|
name | yes | Must be at least 3 characters long but no longer than 128. | The domain name. Must be unique to the entire YOUnite deployment since domains are typically shared. TODO NEED SOME DESCRIPTION OF VALID CHARACTER PARAMETERS. |
description | no | 0 to 255 characters long. If longer it will be truncated. | A human readable description of the domain. |
zoneUuid | no | Owning zone's domain UUID | The zone that the domain, the domain's versions and its data records will be tied to. If this is omitted the caller's current zone will be used. Note that the caller must have permissions to create a domain. |
domainType | no | MDM_DATA_STORE or FEDERATED | The domain type can be either: MDM_DATA_STORE which is when the domain's data records are stored in YOUnite. This is used when the entire organization is comfortable or mandated to migrate a domain to a single store. MDM_DATA_STORE is optimal for reference data such as a list of states, countries, zip-codes, etc. FEDERATED domains do not store their data in YOUnite, but reference and update data on the systems in which it resides. Federated domains require adaptors, metadata and governance configurations which are covered in detail *TODO: TUTORIAL ON FEDERATED* |
The model is not created in this step. Models are tied to specific versions covered below.
POST a Domain Version
With the domain in place, its first version can be created. The domain version defines the properties that make up the domain version's model. You may want to create a new version if you want to add more properties to the model, for instance. The domain version numbers are automatically generated and start with 1 and continue in ascending order. The root node of the model schema is the properties
element. See Valid Property Names and Valid Types for ModelSchema Properties below.
A domain version is defined with a domain JSON Object as described below:
POST /domains/versions/<zone-uuid>
{ "modelSchema": { "properties": { "<property-name>": { "type": "<property-type>", ...item1 properties.... }, "<property-name>": { ... } } }, "descriptin": "<description>", "displayProperty": "<property-name>", "uniquenssRuless": "<property-name1> [, <property-name2>, .....]" }
Descriptions of the Domain Version Properties
property | required | valid values | description |
---|---|---|---|
modelSchema | yes | See Model Schema Properties and Post a Domain below. | A JSON model describing the schema for the data domain; it defines the properties that make up the domains schema. The root node of the model schema is the properties element. See Valid Property Names and Valid Types for ModelSchema Properties below. |
description | no | 0 to 255 characters long. If longer it will be truncated. | A human readable description of the domain version. |
displayProperty | yes | A valid model schema property name | A property defined in the modelSchema that acts as an index for the domain. The data values for the displayProperty must always be unique. See Retrieving Data Records; the Display Property below. |
uniquenessRules | no | One or more valid model schema property names | Required data and their associated rules that are cache'd to make sure duplicate data entries are not POSTed to the domain. This defaults to the displayProperty if it is not provided. See UniquenessRules Property below. |
The Default Domain Version
The first version of a domain is the default version and will remain the default version if more versions of a given domain are created.The PATCH method for the /domains/<domain-uuid> endpoint can be used to change a domain's default version. See the YOUnite API for implementation details.
Creating a Domain is Three-Step Process
For example, we'll create a MDM_DATA_STORE data domain and load one data record:
- POST the Domain
- POST a Domain Version
- POST (or Map if it is a federated data domain) the Data record
POST the Domain
For example, to create a simple "states" domain :
POST /domains
{ "name": "states", "description": "A reference domain of states that should be referenced by all applications in the local YOUnite ecosystem", "zoneUuid": "a1aca070-846f-44e5-9471-c73b46c35f4a", "domainType": "MDM_DATA_STORE" }
The location header returned provides the URI for POSTing a domain version below.
e.g. Location /domains/7f28180b-7d9f-42b5-b5ed-d4a0e7ec09fc
POST a Domain Version
POST /domains/versions/
7f28180b-7d9f-42b5-b5ed-d4a0e7ec09fc
{ "displayProperty": "abbreviation", "uniquenessRules": "abbreviation", "description": "A reference list of states in the North American States: USA, Mexico and Canada", "modelSchema" { "properties": { "name": { "type": "string", "description": "The state's official name", "min": 2, "max": 80, "required": true }, "abbreviation": { "type": "string", "description": "The state's official abbreviation", "min": 2, "max": 2, "required": true } } } }
POST Data Records to the Domain
Once the domain/version has been created, data records can be POSTed to it using the /dr endpoint:POST /drs
{ "name": "states", "version": 1, "json": { "name" : "California", "abbreviation" : "CA" } }
Retrieving Data Records
The data records for a the default version of a domain can be retrieved with the following:
POST /drs?filters=name:<domain-name>
Retrieving the data records for a given domain and version:
POST /drs?filters=name:<domain-name>,version:<version>
The Display Property
Each domain must have a display property (displayProperty
). The display property acts as a primary key for the domain. For example, the "states"
domain above uses the abbreviation
property as the display property, Use the /drs
endpoint and the appropriate domain and display property to GET a data record:
GET /drs?filters=name:states,displayProperty:CA
If there are multiple versions of a domain, and a domain other than the default is needed, the version number can be included in the URI. For example, assume there are three versions of the "states"
domain and the current version is version 3. The consumer can retrieve the California
version 1 data record by using the following:
GET /drs?filters=name:states,version:1,displayProperty:CA
Display Property Rules
- The value provided for the
displayProperty
must be unique between all domain entries of a given domain type (e.g. each entry in the "state"
domain must have a uniquestateName
). - Only one property in a domain can be the
displayProperty. I
f more than one property is required to ensure uniqueness see the Uniqueness Rules Property below. - Display properties are limited to type STRING.
- Properties designated as the
displayProperty
are required; i.e. null values are not allowed. - Display properties are case sensitive e.g. "California" is NOT equal to "california".
Uniqueness Rules Property
Uniqueness Rules
- MDM provides uniqueness rules to prevent data record duplication in the case where a simple
displayProperty
won't suffice. - Optionally, specify a comma-separated list of domain properties whose values, in aggregate, must be unique across all the data records for the domain.
- If not provided, MDM will use the
displayProperty
as theuniquenessRules
. - Uniqueness rule properties are limited to type STRING
- Values in
uniquenessRules
properties are case sensitive e.g. "California" is NOT equal to "california"
Federated Domains, Uniqueness Rules and Required Properties
The goal for federated domains is to keep the combined list of:
- Properties in the uniqueness rules to a minimum
- Properties defined with the "required" option (see "required" option for properties) to a minimum
If the list is long, some adaptors that are associated with the domain might not contain all of the properties and will be unable to add new records to a data domain.
It is important that for a service associated with a domain that it should have governance through ACLs to create data records for a domain. And that it stores or has access to all of the properties that are defined with the "required" option and that are part of the uniqueness rules list.
Valid Property Names
- Must start with a letter or "\_" (underscore).
- Can only contain digits, "-" (dash) and "\_" (underscore).
- Can be up to 64 characters in length.
- Are case in-sensitive.
- If two properties have the same name at the same level only one will be used. In Example 1 below, only one name "property" will be used. In Example 2, both will be used because "
name"
occurs at different levels in the JSON structure.
{ "properties": { ... "name": {...}, "city": {...}, "state": {...}, "name": {...}, ... } }
{ "properties": { ... "owner": { "name": {...}, "phone": {...}, .... }, "pet": { "name": {...}, "classification": {...}, .... }, .... } }
Valid Types
Each property with the exception of a node, requires a type
property e.g.:
"type": "string"
Node
A container-node item is a node that contains sub-properties. For example, "address"
is a container node with the sub-items "city"
and "state
."
{ "properties": { ... "address": { "city": {...}, "state": {...} } } }
Container-nodes can be nested.
The property "type": "node"
isn't required (or recommended) but can be used for clarity. If the "type" and/or "required" properties are used, the sub-properties must be contained inside of the "items" property:
{ "properties": { ... "address": { "type": "node", "items": { "city": {...}, "state": {...} } } } }
Property | Description |
---|---|
required | A non-null value for this item must be provided (false by default). Items inside the container-node canoverride the parent container node's required setting. |
items | Contains the list of sub-properties in the node. This is required only if the "type", "required" and/or "description" properties are used. |
String
A string of characters variable. The following properties are applied when data is posted for this item:
Property | Description |
---|---|
min | Minimum string length. |
| Maximum string length. |
regex | String must match the regex pattern. |
default | If the item is not provided or is null, the default value is used. |
required | A non-null value for this item must be provided (false by default). |
description | A human-readable description of the property. |
Int
A numeric, whole number.
Property | Description |
---|---|
min | Minimum value allowed. |
| Maximum value allowed. |
default | If the item is not provided or is null, the default value is used. |
required | A non-null value for this item must be provided (false by default). |
description | A human-readable description of the property. |
Number
A numeric, decimal number with up to 15 bits of precision.
Property | Description |
---|---|
min | Minimum value allowed. |
| Maximum value allowed. |
default | If the item is not provided or is null, the default value is used. |
required | A non-null value for this item must be provided (false by default). |
description | A human-readable description of the property. |
Boolean
A Boolean, allowing only the two values of true
or false
Property | Description |
---|---|
default | If the item is not provided or is null, the default value is used. |
required | A non-null value for this item must be provided (false by default). |
description | A human-readable description of the property. |
Enum
Enumerations (enums) can be either a primitive type (string
, int,
or a number)
or, a cross-reference to an entire set or subset of data records in another domain.
Property | Description |
---|---|
enumType | Types of enums:
|
xrefLocation | If For example, if a version of a domain "countries"contains a list of data record values that represent all countries then a domain can reference it by specifying an enumeration type of
A subset of the data record values in the domain can be specified using the |
data | An array containing a comma separated list of enum values. e.g. Is a three-item string enumeration. If the
Assuming the display values for Canada, the United States and Mexico are those shown above in the data elements. |
default | If the item is not provided or is null, the default value is used. |
required | A non-null value for this item must be provided (false by default). |
description | A human-readable description of the property. |
URI
A properly formed Uniform Resource Locator (URI). By default, a property of type uri
can be any valid URI or it can be limited by a regex pattern or to a domain cross-reference (xrefLocation:"xref").
A cross-reference (xref
) points to another domain or domain data element. For more on "uriType": "xref"
and "xrefLocation"
see Cross Domains.
See type enum
and the data
property to limit the cross-reference to a subset of the domain's data records (e.g. if a domain "Company"
exists with all Global 1000 companies, an enumeration of EU_Companies
could be created referencing only the Global 1000 companies in the European Union).
Property | Description |
---|---|
uriType | The uri type (e.g. url, blob, xref). |
| The location of the domain item or domain item data element. Used in conjunction with "uritype": "xref" . |
regex | String must match the regex pattern. |
default | If the item is not provided or is null, the default value is used. |
required | A non null value for this item must be provided (false by default). |
description | A human-readable description of the property. |
Array
An array can contain any type of value including nodes and nested arrays.
Property | Description |
---|---|
minItems | Minimum items allowed in the array. |
| Maximum items allowed in the array |
default | If the item is not provided or is null, the default value is used. |
items | Contains the list of sub-properties in the node. |
required | A non-null value for this item must be provided (false by default). |
description | A human-readable description of the property. |
Rules About Required and Default Values
- If a domain property has a
default
value defined in itsmodelSchema
, then any domain data records posted will use the default value if the property either does not include a value or is sent in with a null value.
For example, if a domain named "College"
contains a model schema
with a property StateAbbreviation
with a default
value of "CA"
,
then any college posted to the College
domain without the StateAbbreviation
property or with StateAbbreviation
set with a null value, will use the default value "CA"
.
- If a domain property has
required
set totrue
, then a valid non-null value must be posted for it (by defaultrequired
is set tofalse
).
For example, if a college
domain model schema
has a property CollegeName
with required
set to true
, then any
domain resource data posted must include the CollegeName
property or a BAD_REQUEST(400)
will be returned.
- If both
required
anddefault
are used to define a given property, thendefault
is ignored and POSTing data must include the required property.
POST a Domain Version
Following is a simple example of a model for a domain version.
POST /domains/versions/2a556060-e694-4b61-a3bb-67beda37da13
{ "displayProperty": "countrycode", "modelSchema" { "properties": { "name": { "type": "string", "min": 2, "max": 80, "required": true }, "countrycode": { "type": "string", "min": 3, "max": 3, "required": true, "description": "ISO Standard 3-character Country Code" }, "population": { "type": "int", "required": false }, "capital": { "city": { "type": "string", "required": true }, "districtOrState": { "type": "string", "required": true } } } } }
Note that inside the value portion of the modelSchema
node, all of the double quotes must be escaped with a backslash (\). Newlines have been added above for readability but they should not be included in the request body. The request should look like this:
{ "name": "countries", "displayProperty": "countrycode" "modelSchema": "{ \"properties\": { \"name\": { \"type\": \"string\", \"min\": 2, \"max\": 80, \"required\": true }, \"countrycode\": { \"type\": \"string\", \"min\": 3, \"max\": 3, \"required\": true, \"description\": \"ISO Standard 3-character Country Code\" }, \"population\": { \"type\": \"int\", \"required\": false }, \"capital\": { \"city\": { \"type\": \"string\", \"required\": true }, \"districtOrState\": { \"type\": \"string\", \"required\": true } } } }" }
The response code on success is 201 CREATED