Parker's Thoughts on Project Glue and Master Data Managment

These are my thoughts based on observations and may not reflect the actual state of project Glue, Course Exchange or MDM


The idea for project glue came out of the Course Exchange project.  The key requirement of Course Exchange is that a student at one district/college be allowed to enroll in a class offered at another district/college from a single user interface, the Course Exchange application. 

Each district/college uses different Student Information Systems, Student Directories and Master Data sets to support their own student population. Project Glue was born out of the necessity for colleges to share course and student information from disparate systems using a common set of API interfaces. "College Adaptors" are SIS and/or college specific implementations of the standard API interfaces.

In reality, there x major Student information systems used across the CCC system including

  • Banner
  • PeopleSoft
  • Colleague

Banner, PeopleSoft and Colleague adaptors have been implemented  and colleges using these systems can used the pre-created adaptors with only configuration changes.  

Colleges that have implemented custom SIS will need to develop custom College Adaptors.


Master Data

As previously mentioned, each college SIS requires a source of Master data within the SIS or an auxiliary college system.

Master Data can be described as the key business information that supports the transactions.
Master Data describes the customers, products, parts, employees, materials, suppliers, sites, etc involved in the transactions. It is commonly referred to
as Places (locations, geography, sites, etc.), Parties (persons, customers, suppliers, employees, etc.) and Things (products, items, material, vehicles, etc.).

Master data in the context of Course Exchange.

The following diagram is the DB schema for course exchange.




Drilling into the schema, there are data sets that are not only specific to Course Exchange but are also useful in other applications such as CCCApply, MyPath and Common Assessment.

Some of these data sets include

  • Districts
  • Schools (Colleges)
  • Persons (Students)
  • Terms
  • Courses

Apply currently requires colleges to maintain an Apply specific tables for Districts, Colleges and Terms.  College Staff typically enter this data in their own College SIS, then re-enter the exact same data in the Apply Administrator Application.  What this means is that we now maintain multiple sets of duplicate master data at the colleges and in the Apply database.  To eliminate duplicates, or to keep the duplicate sets of data in sync, we introduce the concept of Master Data Management.

Master Data Management

As mentioned in the previous section, using Terms as an example, the college enters Terms in their own SIS, then re-enters the same Term data in the Apply administrator.

Because we now have duplicate copies of Term master data, we need to consider the following:

  • What is the system of record? (college or apply)
  • What are the risks/benefits of duplicating data?
  • What are the risks of out of sync data?
  • What systems need the data?
  • How quickly do the systems need to access the data?
  • How often does the data change?
  • How quickly do systems need access to recently updated data?


What does Master Data Management and Project Glue have in common?

Because Project Glue exchanges master data such as Courses and terms between multiple systems via a common interface, it has aspects or a master data management (MDM) system.

Though project glue was originally designed to meet the data requirements of course exchange, its ability to share master data between colleges makes Project Glue well suited to support the master data requirements of other systems such as Apply, Assess, MyPath, CID and others as well.


When does it make sense to have multiple copies of master data

Many different applications need fast access Master data, for instance, most applications need access to a full set of colleges and districts, and others need information about Terms and Majors.  Because this type of  data has a slow rate of change, the data is usually cached in the application and is refreshed many times a day.   For data like terms of majors, the system of record will always be the SIS of the hosting school or district, but if an application such as CCCApply can not rely on the availability of the SIS.  Therefore, it makes sense for the Master data to be duplicated in a data store that has high availability. That datastore could be Elasticache, where data is persisted for short amounts of time and when the system of record is not available, or could be stored in something more durable like RDS or DynamoDB.