Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 84 Next »

A data domain (domain) refers to a data model, such as student or course, and is defined by the parties responsible for data governance. The data domain defines the model schema and other attributes for that focus. Common examples of data domains are customers, students, employees, parts, and product orders.

YOUnite allows the data architect to model for multiple domains (customers, students, employees, parts, product orders, etc) as part of a single solution.

Once a version of a domain is created and its data records are loaded (or mapped, in a federated model), it can be referenced by other data domains, adaptors, and API consumers as a source of truth. Domains have version numbers so that other domains, adaptors, and applications can bind to a specific domain and version (e.g. students:v3). Data governance can designate data at specific adaptors as the master data and can control which zone has the appropriate ACLs to access and update it.

Domain Types

The data records for a domain can be stored either as:

MDM Data Store: When the domain data records are stored in YOUnite. This is used when the entire organization is comfortable or mandated to migrate a domain to a single store. The YOUnite data store is optimal for reference data such as a list of states, countries, zip-codes, etc. 

Federated domains do not store their data in YOUnite but retrieve and update the data on the systems in which it resides. For example, MIS, ERP, or CRM systems. Federated domains require adaptors, metadata, and governance configurations and are covered in detail *TODO: TUTORIAL ON FEDERATED*

Domain Model Schemas

A domain Model Schema refers to the attributes (properties), format, and other metadata that defines how a specific domain should expect to store the data (either in the YOUnite Data Store or Federated), for the purposes of standardizing how data is exchanged between systems. The Data Governance Steward is responsible for configuring and maintaining domain model schemas. A  domain model schema is a JSON object describing/defining the properties for the domain's schema. The root node of the model schema is the properties element. See Valid Property Names and Valid Types for ModelSchema Properties below.

Domain Creation Overview

An overview of the domain creation process is described below, and is followed by an example of the domain creation process and then posting records to and retrieving records from the domain. 

POST the Domain

The first step in creating a domain is to define the domain name. You will have to also define the new domain's type and the zone it is attached to (its "owning zone"). If you do not define the type and zone, then YOUnite will do it for you by default):

POST /domains

{
	"name": "<domain name>",
    "description": "<human readable description of the data domain>",
    "zoneUuid": "<owning zone uuid>",
    "domainType": "<domain type>" 
}

Domain Properties Descriptions

propertyrequiredvalid valuesdescription
nameyesMust be at least 3 characters long but no longer than 128.The domain name. Must be unique to the entire YOUnite deployment since domains are typically shared. TODO NEED SOME DESCRIPTION OF VALID CHARACTER PARAMETERS.
descriptionno0 to 255 characters long. If longer it will be truncated.A human readable description of the domain.
zoneUuidnoOwning zone's domain UUIDThe zone that the domain, the domain's versions, and its data records will be tied to. If this is omitted the caller's current zone will be used. Note that the caller must have permissions to create a domain.
domainTypenoMDM_DATA_STORE or FEDERATED

The domain type can be either:

MDM_DATA_STORE,  which is when the domain's data records are stored in YOUnite. This is used when the entire organization is comfortable or mandated to migrate a domain to a single store. MDM_DATA_STORE is optimal for reference data such as a list of states, countries, zip-codes, etc. 

FEDERATED domains do not store their data in YOUnite, but reference and update data on the systems in which it resides. Federated domains require adaptors, metadata, and governance configurations that are covered in detail *TODO: TUTORIAL ON FEDERATED*

The model is created in the next step. Models are tied to specific versions, which is covered below.

POST a Domain Version

With the domain in place, its first version can be created. The domain version defines the properties that make up its model schema. You may want to create a new domain version if you want to add more properties to the model, for instance.

Domain version numbers are automatically generated and start at 1 and continue in ascending order. The first version of a domain is the default version and will remain the default version if more versions of a given domain are created.The PATCH method for the /domains/<domain-uuid> endpoint can be used to change a domain's default version. See the YOUnite API for implementation details.

The root node of the model schema is the properties element. See Valid Property Names and Valid Types for ModelSchema Properties below.

A domain version is defined with a domain JSON Object as described below:

POST /domains/versions/<zone-uuid>

{
	"modelSchema": {
		"properties": {
			"<property-name>": {
				"type": "<property-type>",
				...item1 properties....
			},
			"<property-name>": {
				 ... 
			}
		}
	},
	"description": "<description>",
	"displayProperty": "<property-name>",
	"uniquenssRules": "<property-name1> [, <property-name2>, .....]"
}

Domain Version Properties Descriptions

propertyrequiredvalid valuesdescription
modelSchemayesSee Model Schema Properties and Post a Domain below.A JSON model describing the schema for the data domain; it defines the properties that make up the domains schema. The root node of the model schema is the properties element. See Valid Property Names and Valid Types for ModelSchema Properties below.
descriptionno0 to 255 characters long. If longer it will be truncated.A human readable description of the domain version.
displayPropertyyesA valid model schema property nameA property defined in the modelSchema that acts as an index for the domain. The data values for the displayProperty must always be unique. See Retrieving Data Records; the Display Property below.
uniquenessRulesnoOne or more valid model schema property namesRequired data and their associated rules that are cached to ensure duplicate data entries are not POSTed to the domain. This defaults to the displayProperty if it is not provided. See UniquenessRules Property below.

Once the domain/version has been created you can POST data records to, and retrieve data records from, the domain.

Domain Properties Details

The information below provides further important details on:

The Display Property

Each domain must have a display property (displayProperty). The display property acts as a primary key for the domain. For example, the "states" domain above uses the abbreviation property as the display property. Use the /drs endpoint and the appropriate domain and display property to GET a data record:

GET /drs?filters=name:states,displayProperty:CA

If there are multiple versions of a domain, and a domain other than the default is needed, the version number can be included in the URI. For example, assume there are three versions of the "states" domain and the current version is version 3. The consumer can retrieve the California version 1 data record by using the following:

GET /drs?filters=name:states,version:1,displayProperty:CA

Display Property Rules
  • The value provided for the displayProperty must be unique between all domain entries of a given domain type (e.g. each entry in the "state" domain must have a unique stateName).
  • Only one property in a domain can be the displayProperty. If more than one property is required to ensure uniqueness see the Uniqueness Rules Property below.
  • Display properties are limited to type STRING.
  • Properties designated as the displayProperty are required; i.e. null values are not allowed.
  • Display properties are case sensitive e.g. "California" is NOT equal to "california".

The Uniqueness Rules Property

The optional uniquenessRules property is used as an edded data record identifier to prevent data record duplication.

Uniqueness Rules
  • MDM provides uniqueness rules to prevent data record duplication in the case where a simple displayProperty won't suffice.
  • Optionally, specify a comma-separated list of domain properties whose values, in aggregate, must be unique across all the data records for the domain.
  • If not provided, MDM will use the displayProperty as the uniquenessRules.
  • Uniqueness rule properties are limited to type STRING.
  • Values in uniquenessRules properties are case sensitive e.g. "California" is NOT equal to "california"

Federated Domains, Uniqueness Rules and Required Properties

The goal for federated domains is to keep the combined list of:

  1. Properties in the uniqueness rules to a minimum
  2. Properties defined with the "required" option (see "required" option for properties)  to a minimum

If the properties list is long, some adaptors associated with the domain might not contain all of the properties and will be unable to add new records to a data domain.

It is important that a service associated with a domain:

  • have governance through ACLs to create data records for that domain
  • stores or has access to all of the properties that are defined with the "required" option and that are part of the uniqueness rules list

Valid Property Names

  • Must start with a letter or "\_" (underscore).
  • Can only contain digits, "-" (dash) and "\_" (underscore).
  • Can be up to 64 characters in length.
  • Are case in-sensitive.
  • If two properties have the same name at the same level only one will be used. In Example 1 below, only one name "property" will be used. In Example 2, both will be used  because "name" occurs at different levels in the JSON structure.


Example 1
{
	"properties": {
		...
		"name": {...},
		"city": {...},
		"state": {...},
		"name": {...},
		...
	}
}
Example 2
{
	"properties": {
		...
		"owner": {
			"name": {...},
			"phone": {...},
			....
		},
		"pet": {
			"name": {...},
			"classification": {...},
			....
		},
		....
	}
}

Valid Types

Each property with the exception of a node, requires a type property e.g.:

"type": "string"
Rules About Required and Default Values
  • If a domain property has a default value defined in its modelSchema, then any domain data records posted will use the default value if the property either does not include a value or is sent in with a null value.

For example, if a domain named "College" contains a model schema with a property StateAbbreviation with a default value of "CA",
then any college posted to the College domain without the StateAbbreviation property or with StateAbbreviation
set with a null value, will use the default value "CA".

  • If a domain property has required set to true, then a valid non-null value must be posted for it  (by default required is set to false).

For example, if a college domain model schema has a property CollegeName with required set to true, then any
domain resource data posted must include the CollegeName property or a BAD_REQUEST(400) will be returned.

  • If both required and default are used to define a given property, then default is ignored and POSTing data must include the required property.
Node

A container-node item is a node that contains sub-properties. For example, "address" is a container node with the sub-items "city" and "state."

{
	"properties": {
		...
		"address": {
			"city": {...},
			"state": {...}
		}
	}
}


Container-nodes can be nested.


The property "type": "node" isn't required (or recommended) but can be used for clarity.  If the  "type" and/or "required"  properties are used, the sub-properties must be  contained inside of  the "items" property:

{
	"properties": {
		...
		"address": {
			"type": "node",
			"items": {
				"city": {...},
				"state": {...}
			}
		}
	}
}
PropertyDescription
requiredA non-null value for this item must be provided (false by default). Items inside the container-node can
override the parent container node's required setting.
itemsContains the list of sub-properties in the node. This is required only if the "type", "required" and/or "description" properties are used.

String

A string of characters variable. The following properties are applied when data is posted for this item:

PropertyDescription
minMinimum string length.

max

Maximum string length.
regexString must match the regex pattern.
defaultIf the item is not provided or is null, the default value is used.
requiredA non-null value for this item must be provided (false by default).
descriptionA human-readable description of the property.
Int

A numeric, whole number.

PropertyDescription
minMinimum value allowed.

max

Maximum value allowed.
defaultIf the item is not provided or is null, the default value is used.
requiredA non-null value for this item must be provided (false by default).
descriptionA human-readable description of the property.
Number

A numeric, decimal number with up to 15 bits of precision.

PropertyDescription
minMinimum value allowed.

max

Maximum value allowed.
defaultIf the item is not provided or is null, the default value is used.
requiredA non-null value for this item must be provided (false by default).
descriptionA human-readable description of the property.
Boolean

A Boolean, allowing only the two values of true or false

PropertyDescription
defaultIf the item is not provided or is null, the default value is used.
requiredA non-null value for this item must be provided (false by default).
descriptionA human-readable description of the property.
Enum

Enumerations (enums) can be either a primitive type (stringint, or a number) or, a cross-reference to an entire set or subset of data records in another domain.

PropertyDescription
enumType

Types of enums:

  • int: A numeric whole number.
  • number: A numeric, decimal number with up to 15 bits of precision.
  • string: A string of character variables. 
  • uri: A cross-reference to another domain; use in conjunction with xrefLocation.
xrefLocation

If enumType is set to uri then xrefLocation points to the domain that makes up the elements of the enumeration.

For example, if a version of a domain "countries"contains a list of data record values that represent all countries then a domain can reference it by specifying an enumeration type of uri and the desired domain and version number:

enumType: uri

xrefLocation: /domains/countries:v1

A subset of the data record values in the domain can be specified using the data property.

data

An array containing a comma separated list of enum values.

e.g.    "data":["Mays", "McCovey", "Marichal"]

Is a three-item string enumeration.

If the enumType is uri the list of values in the xrefLocation can be limited to the subset of data records whose display values contained in the data array. For example, if /domains/countries:v1 contains a list of all countries but the domain should only reference countries in North America, then the data entries would be:

  "data":["CANADA", "USA", "MEXICO"]

Assuming the display values for Canada, the United States and Mexico are those shown above in the data elements.

defaultIf the item is not provided or is null, the default value is used.
requiredA non-null value for this item must be provided (false by default).
descriptionA human-readable description of the property.
URI

A properly formed Uniform Resource Locator (URI). By default, a property of type uri can be any valid URI or it can be limited by a regex pattern or to a domain cross-reference (xrefLocation:"xref").

A cross-reference (xref) points to another domain or domain data element. For more on "uriType": "xref" and "xrefLocation" see Cross Domains.

See type enum and the data property to limit the cross-reference to a subset of the domain's data records (e.g. if a domain "Company" exists with all Global 1000 companies, an enumeration of EU_Companies could be created referencing only the Global 1000 companies in the European Union).

PropertyDescription
uriTypeThe uri type (e.g. url, blob, xref).

xrefLocation

The location of the domain item or domain item data element.  Used in conjunction with "uritype": "xref".
regexString must match the regex pattern.
defaultIf the item is not provided or is null, the default value is used.
requiredA non null value for this item must be provided (false by default).
descriptionA human-readable description of the property.

Array

An array can contain any type of value including nodes and nested arrays.

PropertyDescription
minItemsMinimum items allowed in the array.

maxItems

Maximum items allowed in the array
defaultIf the item is not provided or is null, the default value is used.
itemsContains the list of sub-properties in the node.
requiredA non-null value for this item must be provided (false by default).
descriptionA human-readable description of the property.


An Example of Creating a Domain in Two Steps

For this domain creation example, we'll create an MDM_DATA_STORE data domain. After you create the domain, the section below this covers posting to and retrieving data records.

  1. POST the Domain
  2. POST a Domain Version


  • POST the Domain 

    For example, to create a simple "states" domain :

    POST /domains

    {
    	"name": "states",
    	"description": "A reference domain of states that should be referenced by all applications in the local YOUnite ecosystem",
    	"zoneUuid": "a1aca070-846f-44e5-9471-c73b46c35f4a",
    	"domainType": "MDM_DATA_STORE"
     }

    The location header returned provides the URI for POSTing a domain version below.

    e.g.   Location /domains/7f28180b-7d9f-42b5-b5ed-d4a0e7ec09fc

  • POST a Domain Version

    POST /domains/versions/7f28180b-7d9f-42b5-b5ed-d4a0e7ec09fc

    {
    	"displayProperty": "abbreviation",
    	"uniquenessRules": "abbreviation",
    	"description": "A reference list of states in the North American States: USA, Mexico and Canada",
    	"modelSchema" {
    		"properties": {
    			"name": {
    				"type": "string",
    				"description": "The state's official name",
    				"min": 2,
    				"max": 80,
    				"required": true
    			},
    			"abbreviation": {
    				"type": "string",
    				"description": "The state's official abbreviation",
    				"min": 2,
    				"max": 2,
    				"required": true
    			}
    		}
    	}
    }


  • The following JSON is another simple example of a model for a domain version.

    POST /domains/versions/2a556060-e694-4b61-a3bb-67beda37da13

    {
    	"displayProperty": "countrycode",
    	"modelSchema" {
    		"properties": {
    			"name": {
    				"type": "string",
    				"min": 2,
    				"max": 80,
    				"required": true
    			},
    			"countrycode": {
    				"type": "string",
    				"min": 3,
    				"max": 3,
    				"required": true,
    				"description": "ISO Standard 3-character Country Code"
    			},
    			"population": {
    				"type": "int",
    				"required": false
    			},
    			"capital": {
    				"city": {
    					"type": "string",
    					"required": true
    				},
    				"districtOrState": {
    					"type": "string",
    					"required": true
    				}
    			}
    		}
    	}
    }

    Note: Inside the value portion of the modelSchema node, all of the double quotes must be escaped with a backslash (\). Newlines have been added above for readability but they should not be included in the request body. The request should look like this:

    {
        "name": "countries",
    	"displayProperty": "countrycode"
    	"modelSchema": "{ \"properties\": { \"name\": { \"type\": \"string\", \"min\": 2, \"max\": 80, \"required\": true }, \"countrycode\": { \"type\": \"string\", \"min\": 3, \"max\": 3, \"required\": true, \"description\": \"ISO Standard 3-character Country Code\" }, \"population\": { \"type\": \"int\", \"required\": false }, \"capital\": { \"city\": { \"type\": \"string\", \"required\": true }, \"districtOrState\": { \"type\": \"string\", \"required\": true } } } }"
    }


    The response code on success is 201 CREATED

Posting Data Records to the Domain

Once the domain/version has been created, data records can be POSTed to it using the /dr endpoint: 


POST /drs

{
	"name": "states",
	"version": 1,
	"json": {
		"name" : "California", 
		"abbreviation" : "CA"
	}
 }

Note: In a federated domain you will need to Map the data record.

Retrieving Data Records from the Domain

The data records for a the default version of a domain can be retrieved with the following:

POST /drs?filters=name:<domain-name>

Retrieving the data records for a given domain and version:

POST /drs?filters=name:<domain-name>,version:<version>

  • No labels