...
With small organizations, there usually isn't an issue keeping just a few systems up to date. The process of keeping data up to date between disparate systems is called Data Integration or (DI).
As organizations bring more and more systems online, their business data is typically gets increasingly duplicated even further and the problem of keeping all of the systems up to date becomes problematic. For example, imagine a delivery service that stores information about its customers in several systems:
As information about their customers change, the difficulty of customer information changes over time (i.e. customer address or phone number), keeping all the systems up to date becomes exponentially more difficult as the number of systems increases.
Considering the diagram above:
...
A customer's email address would be stored on several systems and there may be conflicting values. If this is the case, which system holds the truthtrue/correct value?
- Which system gets notified of a change and what data elements do other systems need?
If all systems need to be updated, then each system needs to know how to transform the data to be consumable consumed by the each of the other systems, which . This requires programming many transformations. In an extreme case, the example above could take five transformations for each of the six systems, totaling 30 30 transformation modules or adaptors.
Note: An adaptor is software located within a system that connects and shares data through the YOUnite Data Hub. The adaptor focuses on Extract, Transform, and Load (ETL): ensuring any system "outbound data" meets defined format requirements and then gets transformed into "inbound data" format that another system requires.
- Are changes handled in real time or in batch?
The easiest and most common change handling is via batch updates. However, the latency between batch updates usually causes issues with business processesmay cause business process issues.
- How is DI handled?
The onus of transferring data between systems becomes can become a daunting task. While some applications have built-in adaptors to handle transformations they , these generally handle only a subset of what is the required data. Where the built-in adaptors fall short or don't exist, the organization has to spend may end up spending resources developing adaptors as "one-off" applications adaptors to meet the ongoing transformation need.
- How does the organization manage access to the data?
Perhaps Using the example above, the Warehouse Management system should get access to only a subset of the customer data or for security reasons. And it shouldn't get any level of access to the Credit Card Processing system. The term Data Governance is used to describe managing data access to the data.
| Anchor | ||||
|---|---|---|---|---|
|
...