A data domain (domain) refers to a data model, such as student or course, and is defined by the parties responsible for data governance. The data domain defines the model schema and other attributes for that focus. Common examples of data domains are customers, students, employees, parts, and product orders.  Data domains are typically created by the Data Governance Steward (DGS).

YOUnite allows the DGS to architect model for multiple domains (customers, students, employees, parts, product orders, etc) as part of a single solution.

Once a version of a domain is created and its data records are loaded (or mapped, in a federated model), it can be referenced by other data domains, adaptors, and API consumers as a source of truth. Domains have version numbers so that other domains, adaptors, and applications can bind to a specific domain and version (e.g. students:v3). Data governance can designate data at specific adaptors as the master data and can control which zone has the appropriate ACLs to access and update it.

Domain Types

The data records for a domain can be stored either as:

MDM Data Store: When the domain data records are stored in YOUnite. This is used when the entire organization is comfortable or mandated to migrate a domain to a single store. The YOUnite data store is optimal for reference data such as a list of states, countries, zip-codes, etc. 

Federated: Federated domains do not store their data in YOUnite but retrieve and update the data on the systems in which it resides. For example, MIS, ERP, or CRM systems. Federated domains require adaptors, metadata, and governance configurations. Accessing federated data is covered on Accessing Data Records,

Domain Model Schemas

A domain Model Schema refers to the attributes (properties), format, and other metadata that defines how a specific domain should expect to store the data (either in the YOUnite Data Store or Federated), for the purposes of standardizing how data is exchanged between systems. The Data Governance Steward is responsible for configuring and maintaining domain model schemas. A  domain model schema is a JSON object describing/defining the properties for the domain's schema. The root node of the model schema is the properties element. See Data Domains#Valid Property Names and Data Domains#Valid Types for ModelSchema Properties below.

Domain Creation Overview

An overview of the domain creation process is described below, and is followed by an example of the domain creation process and then posting records to and retrieving records from the domain. 

POST the Domain

The first step in creating a domain is to define the domain name. You can also define the new domain's type and the zone it is attached to (its "owning zone"). If you do not define the type and zone, then YOUnite will use defaults as described beow:

POST /domains

{
	"name": "<domain name>",
    "description": "<human readable description of the data domain>",
    "zoneUuid": "<owning zone uuid>",
    "domainType": "<domain type>" 
}

Domain Properties Descriptions

The model is created in the next step. Models are tied to specific versions, which is covered below.

POST a Domain Version

With the domain in place, its first version can be created. The domain version defines the properties that make up its model schema. You may want to create a new domain version if you want to add more properties to the model, for instance.

Domain version numbers are automatically generated and start at 1 and continue in ascending order. The first version of a domain is the default version and will remain the default version if more versions of a given domain are created.The PATCH method for the /domains/<domain-uuid> endpoint can be used to change a domain's default version. See the YOUnite API for implementation details.

The root node of the model schema is the properties element. See Valid Property Names and Valid Types for Model Schema Properties below for details.

A domain version is defined with a domain JSON Object as described below:

POST /domains/versions/<zone-uuid>

&&&Remove displayProperty and uniquenessRules from example below and replace with fastDuplicateDetectionProperties&&&

{
	"modelSchema": {
		"properties": {
			"<property-name>": {
				"type": "<property-type>",
				...item1 properties....
			},
			"<property-name>": {
				 ... 
			}
		}
	},
	"description": "<description>",
	"fastDuplicateDetectionPropertiess": "property-name, [, <property-name2>, ...]"
}

11/16/17 Per Mark: THINGS HAVE CHANGED…. in the “Domain Version Properties Descriptions” and throughout the page.

• No more displayProperty, uniquenessRules: they got replaced with “fastDuplicateDetectionProperties” (FDDP). See the glossary. It has a good description of “Deterministic Uniqueness” , "De-Duplication” and "Probabilistic Uniqueness” —   it also has a  pretty good description of "Fast Duplicate Detection Properties”.
• Basically, we can’t have dupes so FDDP are the way we prevent them.

Domain Version Properties Descriptions

Once the domain/version has been created you can POST data records to, and retrieve data records from, the domain.

Domain Properties Details

The information below provides further domain property details:

• fastDuplicateDetectionProperties
• Federated Domains: Uniqueness Rules & Required Properties
• Valid property names
• Valid property types

Fast Duplicate Detection Properties (FDDPS)

Each domain must have a duplicate detection property (fastDuplicateDetectionProperties). Fast Duplicate Detection Properties (FDDPS) are attributes of a domain and identify those fields that, when combined, MDM should use to detect whether the data record is unique. Refer to Domain Uniqueness, Deterministic Uniqueness, and Probabilistic Uniqueness for more information.

GET /drs?filters=name:states,displayProperty:CA

If there are multiple versions of a domain, and a domain other than the default is needed, the version number can be included in the URI. For example, assume there are three versions of the "states" domain and the current version is version 3. The consumer can retrieve the California version 1 data record by using the following:

GET /drs?filters=name:states,version:1,displayProperty:CA

11/17/17: Per Robbie, drLabel replaced displayProperty, yet is optiona - TO DIANA FROM MARK - We are going to ditch drLabel and just use FDDPsl. 

fastDuplicateDetectionProperites": {

      "description": "The optional property that serves as a label for data records defined by the domain version.  It also can be used as a filter in the API's GET resource.",

      "type": "string"

    },

&&&Update code block and other content in this section??&&&

{
	"name": "states",
	"version": 1,
	"json": {
		"name" : "California", 
		"abbreviation" : "CA"
	}
 }

Use the /drs endpoint and the appropriate domain and display property to GET a data record:

GET /drs?filters=name:states,displayProperty:CA

If there are multiple versions of a domain, and a domain other than the default is needed, the version number can be included in the URI. For example, assume there are three versions of the "states" domain and the current version is version 3. The consumer can retrieve the California version 1 data record by using the following:

GET /drs?filters=name:states,version:1,displayProperty:CA

Note: See Posting a Data Record and Retrieving a Data Record sections for further details on posting/retrieving data records.

Fast Duplicate Dectection Properites Rules

Fast Duplicate Detection Properties are attributes of a Domain and identify those fields that, when combined, MDM should use to detect whether the DR is unique. Refer to Domain Uniqueness, Deterministic Uniqueness, and De-Duplication for more information.

• The value provided for the displayProperty must be unique between all domain entries of a given domain type (e.g. each entry in the "state" domain must have a unique stateName).
• Only one property in a domain can be the displayProperty. If more than one property is required to ensure uniqueness see the Uniqueness Rules Property below.
• Display properties are limited to type STRING.
• Properties designated as the displayProperty are required; i.e. null values are not allowed.
• Display properties are case sensitive e.g. "California" is NOT equal to "california".

Valid Property Names

• Must start with a letter or "\_" (underscore).
• Can only contain digits, "-" (dash) and "\_" (underscore).
• Can be up to 64 characters in length.
• Are case in-sensitive.
• If two properties have the same name at the same level only one will be used. In Example 1 below, only one name "property" will be used. In Example 2, both will be used  because "name" occurs at different levels in the JSON structure.

{
	"properties": {
		...
		"name": {...},
		"city": {...},
		"state": {...},
		"name": {...},
		...
	}
}

{
	"properties": {
		...
		"owner": {
			"name": {...},
			"phone": {...},
			....
		},
		"pet": {
			"name": {...},
			"classification": {...},
			....
		},
		....
	}
}

Valid Model Schema Types

Each property with the exception of a node, requires a type property e.g.:

"type": "string"

Rules About Required and Default Values

• If a domain property has a default value defined in its modelSchema, then any domain data records posted will use the default value if the property either does not include a value or is sent in with a null value.

For example, if a domain named "College" contains a model schema with a property StateAbbreviation with a default value of "CA",
then any college posted to the College domain without the StateAbbreviation property, or with StateAbbreviation
set with a null value, will use the default value "CA".

• If a domain property has required set to true, then a valid non-null value must be posted for it  (by default required is set to false).

For example, if a college domain model schema has a property CollegeName with required set to true, then any
domain resource data posted must include the CollegeName property or a BAD_REQUEST(400) will be returned.

• If both required and default are used to define a given property, then default is ignored and POSTing data must include the required property.

Node

A container-node item is a node that contains sub-properties. For example, "address" is a container node with the sub-items "city" and "state."

{
	"properties": {
		...
		"address": {
			"city": {...},
			"state": {...}
		}
	}
}

Container-nodes can be nested.

The property "type": "node" isn't required (or recommended) but can be used for clarity. If the "type" and/or "required" properties are used, the sub-properties must be  contained inside of  the "items" property:

{
	"properties": {
		...
		"address": {
			"type": "node",
			"items": {
				"city": {...},
				"state": {...}
			}
		}
	}
}

String

A string of characters variable. The following properties are applied when data is posted for this item:

Int

A numeric, whole number.

Number

A numeric, decimal number with up to 15 bits of precision.

Boolean

A Boolean, allowing only the two values of true or false

Enum

Enumerations (enums) can be either a primitive type (string, int, or a number) or, a cross-reference to an entire set or subset of data records in another domain.

URI

TODO: "xref" type and "xrefLocation" has been moved to a new type called "xref". Documentation needs to be updated.

A properly formed Uniform Resource Locator (URL). A URL is a type of Uniform Resource Identifier (URI). By default, a property of type uri can be any valid URI or it can be limited by a regex pattern or to a domain cross-reference (xrefLocation:"xref").

A cross-reference (xref) points to another domain or domain data element. For more on "uriType": "xref" and "xrefLocation" see Cross Domains.

See type enum and the data property to limit the cross-reference to a subset of the domain's data records (e.g. if a domain "Company" exists with all Global 1000 companies, an enumeration of EU_Companies could be created referencing only the Global 1000 companies in the European Union).

Array

An array can contain any type of value including nodes and nested arrays.

Note: Updating arrays can be handled differently when the domain is federated and an "Array Advisory Practice" should be developed by the ZDS/DGS and communicated to developers.

For example, this can be illustrated using an  array of phone numbers… e.g. mobile, home, work, home-fax, etc.

When an adaptor detects a change in a source entities' array of phone numbers, it can send up the entire array or just the modified phone number in the array. In either case, the router will forward it based on the ACLs. The recipient can take these changes and add or update existing entries based on the phone number type property.

Since adaptor developers can handle arrays in one of these two different ways, it is recommended that the ZDS/DGSs make a decision about the preferred method, an "Array Advisory Practice," and communicate this to the adaptor developers.

Data Governance (DG) may say:

Never send a partial array of phone numbers but only the entire set so the entire set at the receiving adaptor can do a complete delete and replace on all phone numbers.

— OR —

Just send updates to an array only and the receiving adaptors can process them as they see fit and handle them using an index (which would be an agreed-upon domain version property; in this example it would be “type” (phone number type)).

An example of a good way for DG to articulate this would be in the “description” field for the phone number array in the domain version modelSchema.

{
        "properties": {
                "name": {
                        "type": "string",
                        "min": 2,
                        "max": 80
                },
                "phone": {
                        "type": "array",
                        "minItems": 1,
                        "maxItems": 50,
                        "description": "An array of phone numbers - ADAPTOR DEV ADISORY - Sending/receiving individual phone numbers is OK - sending/receiving the entire list is not necessary ",
                        "items": {
                                "type": {
                                        "type": "string",
                                        "min": 2,
                                        "max": 80,
                                        "regex": "^[A-Za-z]+"
                                },
                                "number": {
                                        "type": "number",
                                        "min": 2,
                                        "max": 10
                                }
                        }
                }
        }
}

&&&11/16/17 From Mark: double back and check the section titled "An Example of Creating a Domain in Two Steps” is accurate since some of the API options parameters may change.&&&

An Example of Creating a Domain in Two Steps

For this domain creation example, we'll create an MDM_DATA_STORE data domain. After you create the domain, the section below this covers posting and retrieving data records.

1. POST the Domain
2. POST a Domain Version

1. POST the Domain

    

   A domain can be either FEDERATED or MDM_DATA_STORE (stored in YOUnite's DB). For example, to create a MDM_DATA_STORE  "states" domain :

   POST /domains

   {
   	"name": "states",
   	"description": "A reference domain of states that should be referenced by all applications in the local YOUnite ecosystem",
   	"zoneUuid": "a1aca070-846f-44e5-9471-c73b46c35f4a",
   	"domainType": "MDM_DATA_STORE"
    }

   The location header returned provides the URI for POSTing a domain version below.

   e.g.   Location /domains/7f28180b-7d9f-42b5-b5ed-d4a0e7ec09fc

2. POST a Domain Version

POST /domains/versions/7f28180b-7d9f-42b5-b5ed-d4a0e7ec09fc

{
	"fastDuplicateDetectionProperties": "abbreviation",
	"description": "A reference list of states in the North American States: USA, Mexico and Canada",
	"modelSchema" {
		"properties": {
			"name": {
				"type": "string",
				"description": "The state's official name",
				"min": 2,
				"max": 80,
				"required": true
			},
			"abbreviation": {
				"type": "string",
				"description": "The state's official abbreviation",
				"min": 2,
				"max": 2,
				"required": true
			}
		}
	}
}

The following JSON is another simple example of a model domain version.

POST /domains/versions/2a556060-e694-4b61-a3bb-67beda37da13

{
	"fastDuplicateDetectionProperties": "countrycode",
	"modelSchema" {
		"properties": {
			"name": {
				"type": "string",
				"min": 2,
				"max": 80,
				"required": true
			},
			"countrycode": {
				"type": "string",
				"min": 3,
				"max": 3,
				"required": true,
				"description": "ISO Standard 3-character Country Code"
			},
			"population": {
				"type": "int",
				"required": false
			},
			"capital": {
				"city": {
					"type": "string",
					"required": true
				},
				"districtOrState": {
					"type": "string",
					"required": true
				}
			}
		}
	}
}

Note: Inside the value portion of the modelSchema node, all of the double quotes must be escaped with a backslash (\). Newlines have been added above for readability but they should not be included in the request body. The request should look like this:

{
    "name": "countries",
	"fastDuplicateDetectionProperties": "countrycode"
	"modelSchema": "{ \"properties\": { \"name\": { \"type\": \"string\", \"min\": 2, \"max\": 80, \"required\": true }, \"countrycode\": { \"type\": \"string\", \"min\": 3, \"max\": 3, \"required\": true, \"description\": \"ISO Standard 3-character Country Code\" }, \"population\": { \"type\": \"int\", \"required\": false }, \"capital\": { \"city\": { \"type\": \"string\", \"required\": true }, \"districtOrState\": { \"type\": \"string\", \"required\": true } } } }"
}

The response code on success is: 201 CREATED

Posting Data Records to the MDM_DATA_STORE Domain

Once the domain/version has been created, data records can be POSTed to it using the /dr endpoint: 

POST /drs

{
	"name": "states",
	"version": 1,
	"json": {
		"name" : "California", 
		"abbreviation" : "CA"
	}
 }

Notes: 

• In a federated domain you will need to Map the data record.
• Inside the value portion of the json node, all of the double quotes must be escaped with a backslash (\). Newlines have been added above for readability but they should not be included in the request body. The request should look like this:

{
	"name": "states",
	"version": 1,
	"json": "{ \"name\": \"California\", \"abbreviation\" : \"CA\"}
 }

Retrieving Data Records from MDM_DATA_STORE Domains

The data records for the default version of a domain (and their respective UUIDs) can be retrieved with the following:

GET /drs?filters=name:<domain-name>

Retrieving the data records (and their respective UUIDs) for a given domain and version:

GET /drs?filters=name:<domain-name>,version:<version>

Retrieving a specific data record for a MDM_DATA_STORE domain

GET /drs/<dr-uuid>

Posting Data Records with a FEDERATED Domain

Adaptors capable of a given domain version detect changes in the source system they are connected to then, send them to YOUnite. YOUnite considers the domain and adaptor where the change was detected and, applies the appropriate governance and routes the data event to the other adaptors in the YOUnite ecosystem.

Retrieving Data Records with a FEDERATED Domain: Data Record Assembly

Since YOUnite does not store the data records for federated data domains, it cannot return the entire data record in lists as it does with MDM_DATA_STORE domains.  The first two requests below will return only the FDDPs and UUIDs for the data records in a domain. The third request shows to retrieve an assembled data record using a POST request.  See the descriptions below for more details:

The FDDPs for data records or the default version of a domain (and their respective UUIDs) can be retrieved with the following:

GET /drs?filters=name:<domain-name>

Retrieving the FDDPs for the data records (and their respective UUIDs) for a given domain and version:

GET /drs?filters=name:<domain-name>,version:<version>

Make a request for a specific federated data record. Since YOUnite must make requests to the adaptors where the data resides and assemble the data, the request payload must include a "callbackUrl" so YOUnite can deliver the payload (see http://youite.us/api). The POST request will return with a location that can be followed to check on the status of the request:

POST /drs/<dr-uuid>/assembler