...
Introductions to Adaptors can be found on the Introduction to YOUnite Introductions to Adaptors can be found on the Introduction to YOUnite page and the Adaptors page.
...
The YOUnite API Documentation can be found at https://younite.us/api
...
Adaptor Workflow
The workflow required in getting a fully functioning adaptor are as followsFollow this workflow to get an adaptor fully operational:
- Add an Adaptor using Type using the YOUnite API, which will generate the necessary configuration details the Adaptor will need at startup time.
- Develop an adaptor in your development environment using a YOUnite Adaptor SDK (see YOUnite Adaptor Guide for Java Developers).
- Deploy the adaptor on a server (e.g. on a physical server, virtual server, instance, container, etc.) that has access to the YOUnite message bus.
- Connect the adaptor to the native data store or service that it is connect to YOUnite (e.g. CRM, MIS, SIS, DB, etc.).
- Configure the adaptor with the details from step 1 "Register an Adaptor" above.
- Start the adaptor so it can connect to the YOUnite DataHub over the YOUnite Message Bus.
- Change an adaptor's state.
- Pause
- Play
- Play Read Only
- Manage the adaptor.
- Delete an Adaptor
- Get new Credentials for an adaptor
Step 1 ("Adding an Adaptor") and steps 5 through 8 are covered on this page. Steps 2 through 4 are implementation-specific.
1. Adding an Adaptor
Adding an adaptor merely informs the Data Hub that there is an intention to add an adaptor to a zone. Only users with POST /zones/zone-uuid/adaptors permissions can add in adaptor to a zone. By default only the Zone Data Steward (ZDS) has this permission.
An adaptor name is all that is required to POST an adaptor. If the request succeeds, an adaptor UUID and security credentials are returned.
POST /zones/3c9000a9-3eb6-41fe-a11b-5a5859020c65/adaptors
Code Block | ||
---|---|---|
| ||
{
"name": "My First Adaptor"
} |
Adaptor configuration (metadata) and capabilities can be added to the POST request body. The metadata must be in JSON format and the capabilities JSON schema can be seen below (TODO - More on capabilities):
Code Block | ||
---|---|---|
| ||
{
"name": "My First Adaptor",
"description": "Our very first adaptor",
"metadata": {
"jdbcDriver": "com.mysql.jdbc.Driver",
"dbUrl": "jdbc: mysql: //localhost/EMP"
},
"capabilities": [{
"action": "GET",
"direction": "IN",
"domainVersionUuid": "cf7a0ad4-efe5-4618-8590-ae9c670da9c6"
}, {
"action": "PUT",
"direction": "IN",
"domainVersionUuid": "cf7a0ad4-efe5-4618-8590-ae9c670da9c6"
}, {
"action": "PUT",
"direction": "IN",
"domainVersionUuid": "cf7a0ad4-efe5-4618-8590-ae9c670da9c6"
}, {
"action": "GET",
"direction": "OUT",
"domainVersionUuid": "cf7a0ad4-efe5-4618-8590-ae9c670da9c6"
}, {
"action": "PUT",
"direction": "OUT",
"domainVersionUuid": "cf7a0ad4-efe5-4618-8590-ae9c670da9c6"
}, {
"action": "POST",
"direction": "OUT",
"domainVersionUuid": "cf7a0ad4-efe5-4618-8590-ae9c670da9c6"
}, {
"action": "DELETE",
"direction": "OUT",
"domainVersionUuid": "cf7a0ad4-efe5-4618-8590-ae9c670da9c6"
}]
}
|
The response would look similar to:
Code Block | ||
---|---|---|
| ||
{
"uuid": "3dfcc03d-e5d4-4d57-9e9b-5c5d2db32f9a",
"zoneUuid": "3c9000a9-3eb6-41fe-a11b-5a5859020c65",
"clientId": "8c9167a6-bb83-4f77-bdfc-1947a946f77b",
"clientSecret": "de02e3fa-4b23-46cb-aed6-5665a16e73d3"
} |
The metadata and adaptor capabilities can be updated with a PUT request.
WARNING: HTTP PUT stipulates that all properties defined in the PUT request body must be provided. Any properties omitted or left blank will overwrite the resource with a null value.
Note: Steps 2 through 4 are implementation specific. After they are completed, continue on to step 5, below.
5. Configuring an Adaptor
On startup an adaptor consults either a properties file (e.g. adaptor.yml) or configuration object (TODO Kevin) for its configuration information. The properties file resides on the same system as the adaptor. The list below explains the minimum properties needed by an adaptor. Consult the adaptors documentation for a definitive list and property names since they can change from adaptor to adaptor???:
...
Example:
Code Block | ||
---|---|---|
| ||
# Configuration
# Transport implementation class
className: com.younite.adaptor.sdk.transport.amq.AMQConnect
# UUID of the zone this adaptor belongs to
zoneUuid: 6ab9380f-d7f2-477c-b93c-3a762e70095e
# Adaptor UUID
adaptorUuid: 3dfcc03d-e5d4-4d57-9e9b-5c5d2db32f9a
# ClientID and Secret to be used by JMS to verify adaptor has valid access to message bus (and API)
clientId: 8c9167a6-bb83-4f77-bdfc-1947a946f77b
clientSecret: de02e3fa-4b23-46cb-aed6-5665a16e73d3
# Message Broker URL
brokerUrl: nio+ssl://192.2.200.25:61617
# OAUTH Server to validate adaptor access credentials
oauthServerUrl: http://192.2.200.15:8080 |
6. Start the Adaptor - Connect to the Hub
Once the adaptor has been added (posted) and configured, it can be launched. It should successfully connect to the hub and move briefly into the "Configured" state and then into the "Play" state unless otherwise configured.
TODO KEVIN: Debugging
...
Once an adaptor is added (step 1, above) it is in the "Posted" state. The states normally set by the YOUnite Data Hub are: POSTED, CONFIGURED, and DOWN.
The adaptor moves between states as follows:
Adaptor States
...
The adaptor has:
- Connected successfully to the YOUnite Datahub via the message bus/broker
- Subscribed to the appropriate message broker topics
- Created its message broker queue
...
Play Read-Only
...
Code Block | ||
---|---|---|
| ||
PUT /zones/zone-uuid/adaptors/adaptor-uuid/ops |
Before making a request, the adaptor's changeVersion needs to be retrieved by peforming a GET on the adaptor:
Code Block | ||
---|---|---|
| ||
GET /zones/3c9000a9-3eb6-41fe-a11b-5a5859020c65/adaptors/3dfcc03d-e5d4-4d57-9e9b-5c5d2db32f9a
|
A response that includes the adaptor's changeVersion is returned:
Code Block | ||
---|---|---|
| ||
{
"uuid": "3dfcc03d-e5d4-4d57-9e9b-5c5d2db32f9a",
"name": "Test Adaptor1",
"zoneUuid": "661f5d76-6bc7-4fc0-97fd-f331ab683379",
"state": "ADAPTOR_POSTED",
"dateCreated": 1503634367407,
"lastUpdated": 1503635641288,
"changeVersion": 4117664412
} |
...
The following request and request body will change an adaptor's state to "pause":
Code Block | ||
---|---|---|
| ||
PUT /zones/661f5d76-6bc7-4fc0-97fd-f331ab683379/adaptors/3dfcc03d-e5d4-4d57-9e9b-5c5d2db32f9a
|
Code Block | ||
---|---|---|
| ||
{
"state": "ADAPTOR_PAUSE",
"changeVersion": 4117664412
} |
...
The following request and request body will change an adaptor's state to "play":
Code Block | ||
---|---|---|
| ||
PUT /zones/661f5d76-6bc7-4fc0-97fd-f331ab683379/adaptors/3dfcc03d-e5d4-4d57-9e9b-5c5d2db32f9a |
Code Block | ||
---|---|---|
| ||
{
"state": "ADAPTOR_PLAY",
"changeVersion": 4117664412
} |
...
The following request and request body will change an adaptor's state to "play read only":
Code Block | ||
---|---|---|
| ||
PUT /zones/661f5d76-6bc7-4fc0-97fd-f331ab683379/adaptors/3dfcc03d-e5d4-4d57-9e9b-5c5d2db32f9a |
Code Block | ||
---|---|---|
| ||
{
"state": "ADAPTOR_PLAY_RO",
"changeVersion": 4117664412
} |
Changing an Adaptor's Operational State
...
- UI's Adaptor Types page (or YOUnite API endpoint /adaptorTypes). Adaptor types are not required since all adaptor will default to the "default" adaptor type. Adaptor types are designed to make configuring an adaptor easier especially for deployments that will be deploying many adaptors of the same type.
- Follow the adaptor's README on where to load the adaptor software on the instance (e.g. docker image, VM, cloud instance, local system). It must be on an instance that can connect to the YOUnite message bus.
- In the YOUnite UI Adaptor's page (or YOUnite API endpoint /zones/<target-zone-uuid>/adaptors ) add the adaptor to the target zone specifying the adaptor's type (or just specify the "default" type).
- The prior step will generates then necessary configuration (Adaptor UUID, clientId and clientSecret for connecting to the YOUnite message bus), the Adaptor will need these at startup. Typically, these can be added to the adaptor's adaptor.yml file, the adaptor's property file or set in the adaptor's environment variables if using a staging tools such as Kubernetes.
- Set the adaptor capabilities and/or configuration (metadata) in the YOUnite UI's adaptor page or the API (PUT /zones/<target-zone-uuid>/adaptors/<adaptor-uuid>). See the adaptor README for capability and metadata settings.
- Deploy the adaptor. Once it is running you can:
- Change an adaptor's state.
- Pause
- Play
- Manage the adaptor.
- Delete an Adaptor
- Get new Credentials for an adaptor
- Change an adaptor's state.
A. Develop an adaptor in your development environment using a YOUnite Adaptor SDK (see YOUnite Adaptor Guide for Java Developers).
B. Create an AdaptorType in the Adaptor Type page in the YOUnite UI or using the API (see the /adaptorTypes endpoint in the YOUnite API)
1. Adding an Adaptor
Adding an adaptor merely informs the Data Hub that there is an intention to add an adaptor to a zone. Only users with POST /zones/zone-uuid/adaptors permissions can add in adaptor to a zone. By default only the Zone Data Steward (ZDS) has this permission.
An adaptor name is all that is required to POST an adaptor. If the request succeeds, an adaptor UUID and security credentials are returned.
POST /zones/3c9000a9-3eb6-41fe-a11b-5a5859020c65/adaptors
Code Block | ||
---|---|---|
| ||
{
"name": "My First Adaptor"
} |
4. Copy the Adaptor Credentials
The response would look similar to:
Code Block | ||
---|---|---|
| ||
{
"uuid": "3dfcc03d-e5d4-4d57-9e9b-5c5d2db32f9a",
"zoneUuid": "3c9000a9-3eb6-41fe-a11b-5a5859020c65",
"clientId": "8c9167a6-bb83-4f77-bdfc-1947a946f77b",
"clientSecret": "de02e3fa-4b23-46cb-aed6-5665a16e73d3"
} |
The metadata and adaptor capabilities can be updated with a PUT request.
WARNING: HTTP PUT stipulates that all properties defined in the PUT request body must be provided. Any properties omitted or left blank will overwrite the resource with a null value.
5. Modify the Adaptor Configuration
Adaptor configuration (metadata) and capabilities can be added to the POST request body. The metadata must be in JSON format and the capabilities JSON schema can be seen below (TODO - More on capabilities):
Code Block | ||
---|---|---|
| ||
{
"name": "My First Adaptor",
"description": "Our very first adaptor",
"metadata": {
"jdbcDriver": "com.mysql.jdbc.Driver",
"dbUrl": "jdbc: mysql: //localhost/EMP"
},
"capabilities": [{
"action": "GET",
"direction": "IN",
"domainVersionUuid": "cf7a0ad4-efe5-4618-8590-ae9c670da9c6"
}, {
"action": "PUT",
"direction": "IN",
"domainVersionUuid": "cf7a0ad4-efe5-4618-8590-ae9c670da9c6"
}, {
"action": "PUT",
"direction": "IN",
"domainVersionUuid": "cf7a0ad4-efe5-4618-8590-ae9c670da9c6"
}, {
"action": "GET",
"direction": "OUT",
"domainVersionUuid": "cf7a0ad4-efe5-4618-8590-ae9c670da9c6"
}, {
"action": "PUT",
"direction": "OUT",
"domainVersionUuid": "cf7a0ad4-efe5-4618-8590-ae9c670da9c6"
}, {
"action": "POST",
"direction": "OUT",
"domainVersionUuid": "cf7a0ad4-efe5-4618-8590-ae9c670da9c6"
}, {
"action": "DELETE",
"direction": "OUT",
"domainVersionUuid": "cf7a0ad4-efe5-4618-8590-ae9c670da9c6"
}]
}
|
On startup an adaptor consults either a properties file (e.g. adaptor.yml) or configuration object (TODO Kevin) for its configuration information. The properties file resides on the same system as the adaptor. The list below explains the minimum properties needed by an adaptor. Consult the adaptors documentation for a definitive list and property names since they can change from adaptor to adaptor???:
Property | Description | Example Value |
---|---|---|
className | Transport implementation class (should be a constant value if using the YOUnite Java SDK) | com.younite.adaptor.sdk.transport.amq.AMQConnect |
zoneUuid | UUID of the zone this adaptor belongs to | 3c9000a9-3eb6-41fe-a11b-5a5859020c65 |
adaptorUuid | UUID of the adaptor | 3dfcc03d-e5d4-4d57-9e9b-5c5d2db32f9a |
clientId | ClientID returned from POST /adaptors used to connect to message broker | 8c9167a6-bb83-4f77-bdfc-1947a946f77b |
clientSecret | Secret returned from POST /adaptors used to connect to message broker | de02e3fa-4b23-46cb-aed6-5665a16e73d3 |
brokerUrl | Message Broker URL | nio+ssl://message-broker-uri:61617 |
oauthServerUrl | OAuth Server to validate adaptor access credentials. YOUnite runs an embedded OAuth server that your implementation may be using. By default it runs on port 8080 so, in this case the value would be http://ip-address-of-the-YOUnite-datahub:8080 | http://oauth-server-uri |
Example:
Code Block | ||
---|---|---|
| ||
# Configuration
# Transport implementation class
className: com.younite.adaptor.sdk.transport.amq.AMQConnect
# UUID of the zone this adaptor belongs to
zoneUuid: 6ab9380f-d7f2-477c-b93c-3a762e70095e
# Adaptor UUID
adaptorUuid: 3dfcc03d-e5d4-4d57-9e9b-5c5d2db32f9a
# ClientID and Secret to be used by JMS to verify adaptor has valid access to message bus (and API)
clientId: 8c9167a6-bb83-4f77-bdfc-1947a946f77b
clientSecret: de02e3fa-4b23-46cb-aed6-5665a16e73d3
# Message Broker URL
brokerUrl: nio+ssl://192.2.200.25:61617
# OAUTH Server to validate adaptor access credentials
oauthServerUrl: http://192.2.200.15:8080 |
6. Start the Adaptor - Connect to the Hub
Once the adaptor has been added (posted) and configured, it can be launched. It should successfully connect to the hub and move briefly into the "Configured" state and then into the "Play" state unless otherwise configured.
Once an adaptor is in the PLAY state, messages can be passed an adaptor to the datahub, and vice versa. Two situations may occur that cause the adaptor to no longer send/receive messages. The first is if the API is used to change the state to PAUSE. The second is in the rare case something goes awry between the datahub and adaptor. This could be the message bus service itself is down/restarting, the queue used for communication has an issue, or possibly something else. In the latter cases, usually the adaptor will auto restart and that will start the Datahub startup sequence again and communications should be re-established. If that does not occur, it may be required to dig further in to the YOUnite logs and possibly the adaptor logs to determine what might have happened.
6a. Change an Adaptor's State
Anchor | ||||
---|---|---|---|---|
|
Once an adaptor is added (step 1, above) it is in the "Posted" state. The states normally set by the YOUnite Data Hub are: POSTED, CONFIGURED, and DOWN. The ../controls endpoint changes an adaptor's state to operational states and those normally set only by the datahub. However, there may be situations where you want to use this endpoint to set these states, such as if an adaptor is an unknown state and restarting the adaptor does not provide a remedy.
Note: Under normal circumstances this resource should never be used and it can render an adaptor and the source entity read/write requests in an unknown state.
This resource overwrites the adaptor to a state normally set by the YOUnite Data Hub. Valid states for this request are the normal adaptor operational states:
- ADAPTOR_PAUSE
- ADAPTOR_PLAY
- ADAPTOR_PLAY_RO
And the states normally set only by the datahub:
- ADAPTOR_POSTED
- ADAPTOR_CONFIGURED
- ADAPTOR_DOWN
Code Block | ||
---|---|---|
| ||
PUT /zones/zone-uuid/adaptors/adaptor-uuid/controls |
...
The adaptor moves between states as follows:
Adaptor States
State | Description |
---|---|
Posted | Adaptor is successfully POSTed. An API consumer can make this request or it can be done through the YOUnite UI. |
Configured | The adaptor has:
|
Pause | The adaptor is running but not accepting adaptor (read/write) requests. |
Play | The adaptor is accepting read and write requests. |
An adaptor state can be changed using the request including JSON request body:
Anchor | ||||
---|---|---|---|---|
|
Code Block | ||
---|---|---|
| ||
PATCH /zones/zone-uuid/adaptors/adaptor-uuid
Request body to pause:
{
"state": "ADAPTOR_PAUSE"
}
Request body to resume:
{
"state": "ADAPTOR_PLAY"
} |
6b. Manage an Adaptor
See the YOUnite API documentation (https://younite.us/api) for more requests that can be made to the adaptors endpoint.
...
Code Block | ||
---|---|---|
| ||
DELETE /zones/661f5d76-6bc7-4fc0-97fd-f331ab683379/adaptors/3dfcc03d-e5d4-4d57-9e9b-5c5d2db32f9a |
Note: When you delete an adaptor it is a soft delete (meaning the adaptor's "active" setting will be set to false). A soft-delete for entities avoids purging important information, including that which can be used to associate past activity in the system, such as logged events, zone, and adaptor type.
However, the linking (mapping) between the (YOUnite) Data Record and the source entities in the source system attached to the deleted adaptor will be hard deletes. Linking for deleted adaptors is hard-deleted as this particular data carries no useful additional data or metadata (the link entities are just an adaptorUuid and a drUuid). Post-delete, an adaptor's linking history can still be viewed in system logs that should include all the relevant details about that link, as well as a log timestamp.
Get Credentials for an Adaptor
...