Versions Compared

Old Version 9

changes.mady.by.user Former user

Saved on

compared with

New Version 10

changes.mady.by.user Former user

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Item

Presenter

Notes

1

CCC Data update

Mark Cohen

Data Warehouse Report Server

  • 95 colleges live with Data Warehouse report server

  • Outreach continues to bring remaining colleges onboard

CCC Data 2.0.0

  • Provides direct (ODBC/JDBC) access to the Data Warehouse

  • Deployed to Pilot 10/2/20

  • 6 Pilot Participants

    • Team/Enabling Services working w Phase 1 Pilot Colleges (2) to setup VPN  site to site connection so that the pilot testing can commence

      • 1 college configured with VPN

      • 1 college expected to be configured this week.

    • Engagement starting w Phase 2 Pilot Colleges (4)

  • To be deployed to production in December

CCC Data 2.1.0

  • New data sources in DW segmented by MIS code & made available to CCC’s through DW Report Server (DWRS) and direct  DW access, including:

    • Canvas (as opted-in to by individual colleges)

    • MyPath

    • COCI

    • C-ID

  • New data sources in DL & made available to CCCCO:

    • Nova

    • Launchboard

  • DWRS CVC-OEI enrollment report 

Development work has begun

  • Pilot Q2

  • Production Q3

2

Update on status of vision for success and student equity measurements metrics

Manos Stefanakos

Student Equity and Achievement/Vision for Success Metrics

  • Addresses data change management issues and the connections to the metrics and that aspect of the data change management

View file
nameStudent Equity and Achievement Metrics.pptx

Student Equity and Achievement

  1. Background: Student Equity and Achievement (SEA)

  2. SEA Reporting Requirements

    1. Executive Summary (Includes goals identified, activities to achieve those goals)

    2. Budget Information

  3. SEA Sample Report from NOVA

    1. Set of demographic groups

    2. Four categories of metrics they can select from (Access, Readiness, Retention,

    3. Can select what is the goal, what is the gap seeking to close

    4. Report is in table format, comes out of NOVA where reporting is done

    5. Accompanied by planning information that is free form text fields, to support report table

  4. What are the SEA Metrics

    1. Colleges have to select at least 5 different groups of traditionally underrepresented students and select a metric for each (from Student Success Metrics list) in the following categories: Access, Readiness, Retention,
      Completion

  5. What are the SEA Metics - Examples

    1. Percentage of students who enroll after applying (Access, SEA)

    2. In parenthesis you can see SEA metrics

    3. Analogous component that comes from the other two sets of metrics being evaluated

  6. Data Lake/Data Warehouse Inclusion

    1. The NOVA annual reporting includes both metrics as well as planning and activity information that is provided in the form of text

    2. The metrics are generally student counts/ratios, often disaggregated by subgroups of students, e.g., by race, gender, LGBTQ, veteran, and economically disadvantaged status

    3. The numbers behind these already exist in MIS data, as student counts and demographic flags on individual student records

    4. The text fields could be brought in to provide additional insight

  7. Suggestions for SEA Metrics

    1. While, ideally, one would be able to recreate the NOVA report using MIS data, because of timing, and other reasons, it is suggested that the NOVA SEA report be brought in, as it might be impossible to recreate the numbers from existing MIS data.

  8. Questions/Concerns

  • Manos Stefanakos: If it is based on completely on MIS data may not need to sort of re-import it as it comes out of Nova. The only suggestion that I would have is that it's if possible that a

    • Suggestion: similar view as to

    that

    • the reporting table be available maybe as part of the reporting or in a table

    that has

    • saved

    somewhere,

    • either in Data Lake or Data Warehouse

    .

  • “Mark, Steve. Any thoughts on maybe saving sort of the, the analogous data that would sort of populate this report other than Jasper reports or someplace else?”

    1. Mark Cohen: “I mean it sounds. It sounds like it would it would make you know it would make sense to add this as a data source that will be available, you know, trough the Jasper reporting and direct access. You know I know when we talked about it before it was, you know, there was a question as to whether this was a one off or or ongoing it how frequently updated are the data sources that make up the report?”

      1. Virginia Moran: “I believe its annual because there's most of the indicators are lagging indicators. The only thing that we do locally, at least for VVC is term the persistence one. But we are required report annually on this thing and I don't think they updated any more frequently than that, honestly, they can't even the term data. Yeah.”

      2. This is something that should be on Data Source prioritization list

    • Dulce Delgadillo: “Are we saying that this data set would only have desegregation for metrics that are associated with student equity? Versus all of the other initiatives so SSM and we look at adult ed because some of those are also presented in a desegregated method. So I'm just a little curious on you know if the overlap. And when we say looking at SEA metrics are we saying that we are going to produce that the database would be able to produce a report that would list all of the SEA metrics with the desegregation that is intended that the state has told us to look at, because that also shifts right?”

  • Manos Stefanakos: You'll see a little bit in the next section, we're talking about the success metrics, it absolutely captures the point you're making that similar metrics are looked at in a variety of different ways for different reasons. So, most of the students success metrics are represented in the list of the SEA metrics, you're just looking at them in a slightly different cut. I think we're trying to figure out, since the data we're finding out the most of it anyway, except the groups that might not be available within the selection already exist in MIS, do we need to bring in sort of the extract of the Nova, as reported by districts colleges back into the data warehouse. I think we're leaving that way based on this conversation so it would for this purpose, this would serve just the SEA metrics and then the generic version of a MIS data. I don't know what else to call it, you would still be able to use to do different desegregations on similar metrics. So completion and who received what type of degree and what happened to them after they left by different subgroups for different groupings, even you could do with that data, but this will be specific, so that the way I looked at it was if the schools want to sort of look over time progress and how things are going. By having sort of this data set as a unique source would facilitate make the reporting a little easier.”

  • Virginia Moran: “So I'm then going back to Dulce’s question. It sounds like they'll say, as long as local colleges have identified specific groups that we're targeting, in addition to the mandated ones or whatever that are standard, we should be able to continue to track them if the this new system is going to take all of that that we've put into SEA when we're planning. That's what you're saying, right, because any original data that we're keying in to the SCA to Nova will be picked up then into this new system.”

  • Manos Stefanakos: “That is the recommendation, yes.”

    1. “It sounds like the first part of the question is yes, we should import this output from NOVA, specifically SEA metrics. The second question that I think Mark and Steve are interested in, is this a priority data set, and that I am unable to answer on behalf of the group or even make a suggestion. I think that's a conversation that we probably need to have. “

    • Mark Cohen: “I believe what we are saying is that these data

    • This is something that will be added to the Data Source prioritization list

    • These data sources should be added to the list of potential data sources though nota high priority of this time. Next time we go over the privatization exercise we could, throw these into the mix and see where it ranks. But for now we should assume that it's not on the top

    .”

  • Dustin Tamashiro: “I haven't been keeping on top of like the SEA discussions recently, but, um, is there any thought of like expanding, I thought they were supposed to do like cohort tracking, which would probably be a little bit more challenging for us to be replicated locally. So, I mean, I think I might personally find it useful to have like the like SEA measures like readily available so that I can try to reproduce it if that's upcoming in the future.”

    • Z Reisz: “Thinking about metrics in two flavors. One is like an institutional health metric, which is usually a cross section at any given term, it's how many graduates, did you have in this term. This lets you know how your institutions health is doing. So we got to keep doing those but then another one and this is kind of what I was hoping to see some of the student success metrics go was really more of this student journey cohort based tracking

    • .

    Where we're really looking okay, how did this student fair across our system.

    • ”

    1. Craig Hayward: “Many of the current SEA metrics are unusual in the way they are calculated. As a field, we are waiting for cohort-based SEA metrics to drop.”

    2. Manos Stefanakos: “So then I think I'll sum up with Mark suggestion that this is a data set, we would like to include and it will be evaluated as to what level priority deserves that the next time the prioritization discussion.”

    Vision for Success Metrics

    1. Vision for Success Background

    2. What are the Vision for Success Metrics?

    3. Vision for Success Commitments

    4. VFS Reporting

    5. Recommendations VFS Metrics

      1. Harmonization/improved definitions so that people know if the metric they are looking at represents what they think it does

      2. Changes to Launchboard to show the appropriate target as part of reports/charts that track VfS metrics (have not seen latest builds that are coming out soon)

    6. Questions/Concerns

    1. Z Reisz: “Yes to harmonizing the data sets so that there's fewer small details to confuse the board members when we present this to them.”

    3

    Change Management; to include both an update on change management efforts at the CO and discussion on how we can best support changes to metrics and data elements. 

    David Kendall

    Address Change Management as process/artifact

    1. Need for Change Management

    2. Best Practices

    3. How we’re going to handle change management

      1. We want to known state of anything that is under change management

      2. We want a clear documented impact analysis, impact to Stakeholder community

      3. We want solid representation from known group of reviewers

      4. Needs to be presented to the appropriate decision making body for deciding on what to do with that change

      5. Ensures that we have the appropriate voice the appropriate identified and approved authority for making decisions around it

      6. Currently creating this concept and presenting it to leadership for their consideration.

    Questions/Concerns

  • Z Reisz: “I think I just have kind of a broad one. So when we're talking about change management. So if you were to take, for example, the student success metrics, we're kind of talking about how we document and make modifications to any of those elements?”

    1. David Kendall: “The decisions have to be made about the level of granularity, for sure, but that absolutely would seem logical to me.”

    • Virginia Moran: “David can you please speak to maybe starting assumptions/core principles as well?”

    David Kendall: “One of the things that makes any kind of change management or governance discussion most effective as you decide on a few things before you get to the decision like,I really like to see a decision authority matrix that says this level within a governance structure has this authority to make decisions and if it's if it's intended to be a decision made it a higher level it escalates, or if it's something that we've empowered work group level to make decisions on and we you know we support that. That's just, you know, some good fundamental ways to create a decision support system. On top of that, we want to make sure that we're all clear on the principles that we're making decisions towards right. The vision for success is intended to provide some of those principles, but I, I still think you review them and you document them and then when a decision is made those principles are called out so that it's clear what's driving the recommendation and ultimately the decision, and thank you, Virginia for pointing that out.

  • Virginia Moran: “So Mark, what role does this group have in terms of advising those things as they get sort of firmed up what what do you want us to do?”

    1. Mark Cohen: “Alex, this is an activity you're working on now, that I'm working on directly.”

    2. Alexander Jackl: “Yeah, I think, here's what I think about that. And I know you know Barney and Valerie are on the call. Right. And so I can speak to. But I think what's going to happen is we're going to start the process of actually making these decisions and looking at how to build out these structures at the chancellor's office level and then when Barney and Valerie and others have sort of decided on okay, this is the path that makes sense. We'll probably come back to you. All right, with that. Um, so I think the next step is just if anyone has any comments on anything they they saw today or thoughts about what they saw today, they can get that to Mar, Mark and then make sure that that we get it. And then I think once the, once the chances offices had a chance to sort of review and think through what probably will be rolling. I can't speak to it. Right. But I think we'll be rolling something out, right, that will then will then feed back out into the field. Once we have it. Um, Any, any comments on that or thoughts on that.”

    3. Barney Gomez: “I think Alex and Steve, what you guys have summed up and that's about this, that's a pretty accurate. I mean, our goal really is to just not hinder the process but create a better process to basically track and manage what we're doing to make sure everybody's on the same page about the changes that we're going forward, how do we calculate these metrics. How do we establish metrics, etc. The research group has to play an important role on this. So our goal is to make sure that we had just have a process, nothing more than a process in place to make sure it's being vetted and then obviously once from a Chancellor's Office perspective, we want to share that information with the field with a system to make sure everybody, you know, to see what kind of input you may or may not have. And if we need to adjust. We can adjust if we need to talk about it. We talked about it. It's really about broadening and promulgating the information about what we're doing, where we're going and how we're going about doing so. So we're just trying to be more collaborative.”

    		• 4

    Discussion on concept of enabling districts to share data

    Intro: Mark Cohen

  • Steve Klein: “And Mark and I just add is the question is also really around a design perspective because the use case that was presented as I understand, was about colleges or districts that had inter district agreements or inter college agreements for data sharing, as part of the consortium or other efforts. So I think where those may be in place. It's a question around the architecture, I think, around, around the design of the access to data across those colleges are districts, is that work is that.”

  • Mark Cohen: “Furthermore, it would be around specific data set. So wouldn't I don't think anyone suggesting that it would be an open, you know, accessing any data associate with the institution. But the question of whether a college district could so whether they're certain data sets that should be designed to enable sharing with select colleges or districts as an obviously that you know there's ways outside of the data warehouse for, you know, for data sharing to be facilitated but but the question was, as we're moving more data into the data warehouse, whether that should be a design consideration, or at least to start a dialogue to explore that, as it has come up several times.”

  • Elaine Kuo: “Foothill De-Anza is location is unusual. We have a lot of community colleges just within our 50 mile radius. So there is a lot of folks, you know, back and forth. And when we, you know, meet regionally in terms of the workforce in the strong workforce efforts we recognize that's happening, but it is extremely labor intensive to figure out where the students are going and the fluidity between the back and forth.”

  • Barney Gomez: “Mark Let's table this for right now. Mark, because this is a much bigger and broader topic for us to basically address at this point but it's certainly something that I'm willing to consider, but it's it's way brought in much bigger. So let's just table this for now. Further discussion down the road.”

    • Valerie Lundy-Wagner: “Is there a priorities that you have that you can share? Is for the data warehouse and then how this group connects to them.”

    Mark Cohen: “Yes. We can we can discuss the work that we're that we have from our work plan, you know, in our roadmap for this year and number of those items. Can you know will connect to the to the group in terms of providing you know feedback and input that will go into requirements as we continued development.”

    4

    Discussion on concept of enabling districts to share data

    Intro: Mark Cohen

    • Discussion around a design perspective and specific data set.

    • Question of whether a college district could use certain data sets and enable sharing with select colleges or districts

    • Will table topic for right now. Will discuss further at a later time

    Issues/Questions Resolved

    Issue/Question

    Resolution/Answer

    Date Resolved/Answered

    1

    Virginia Moran: In the past SCA was pre populated with MIS data and then we would just respond. So, but in the future. You're saying enable us to change what was reported in state MIS?”

    *Related to discussion topic #2

    1. Elaine Kuo: We just input the data we receive the data file. We then and then we enter it into Nova. There was no additional manipulation, there will be some drop downs that would occur, but I don't remember us doing any manipulation, there was no manipulation and then depending on what boxes you collect check that would also create other drop downs was my recollection, but there was no additional sort of like analysis of the data or inputting of the data.

    2. Craig Hayward: That I can think of that may go to this, which is, I think it was possible to add a group that was not included in the derived in the data that was sent to the colleges so that may be the piece that you know can't be replicated or picked up kind of automatically from MIS.

    2

    Dulce Delgadillo: Are we saying that this data set would only have desegregation for metrics that are associated with student equity? Versus all of the other initiatives so SSM and we look at adult ed because some of those are also presented in a desegregated method. So I'm just a little curious on you know if the overlap. And when we say looking at SEA metrics are we saying that we are going to produce that the database would be able to produce a report that would list all of the SEA metrics with the desegregation that is intended that the state has told us to look at, because that also shifts right?

    *Related to discussion topic #2

    Manos Stefanakos: You'll see a little bit in the next section, we're talking about the success metrics, it absolutely captures the point you're making that similar metrics are looked at in a variety of different ways for different reasons. So, most of the students success metrics are represented in the list of the SEA metrics, you're just looking at them in a slightly different cut.

    3

    Z Reisz: “I think I just have kind of a broad one. So when we're talking about change management. So if you were to take, for example, the student success metrics, we're kind of talking about how we document and make modifications to any of those elements?”

    Related to discussion topic #3

    David Kendall: The decisions have to be made about the level of granularity, for sure, but that absolutely would seem logical to me.

    4

    Valerie Lundy-Wagner: Isn't this a legal issue?

    *Related to discussion topic #4

    Bridget (she/her/hers) Herrin: Do you mean under FERPA? If it's being used for legitimate academic purposes it should be covered.

    5

    Elaine Kuo: How much coordination is there between the data warehouse work and the launchboard efforts? I recognize that is part of our continuing conversation. Should there be representation from the launchboard side on this task force (especially when these data elements are discussed)?

    *Related to discussion topic #4

    Valerie Lundy-Wagner: I think that sharing data should be done only when needed/necessary. The "legitimate" academic purposes has not been articulated to me just yet, but I understand it is possible.

  • Valerie Lundy-Wagner: The question was posed as 'of interest' and it should be focused on the problem we're trying to solve.

  • Valerie Lundy-Wagner: Thanks Elaine, yes, there does need to be a bit more conversation between these meetings and the Launchboard conversation. I don't know if John Hetts was invited or just couldn't make this time, but it seems appropriate for he and I to spend a bit more time with Mark and team to make sure the Research and Data Analytics conversations aligns with the MIS-type conversation.

  • Alexander Jackl: @Elaine It is part of the data harmonization efforts to coordinate Launchboard and MIS and DW so we will be working on that and continuing to work on that

    		• 5

    Issues/Questions Resolved

    Issue/Question

    Resolution/Answer

    Date Resolved/Answered

    Owner

    		1

    1. Virginia Moran: Concern. “In the past SCA was pre populated with MIS data and then we would just respond. So, but in the future. You're saying enable us to change what was reported in state MIS?”

  • Elaine Kuo: “we just input the data we receive the data file. We then and then we enter it into Nova. There was no additional manipulation, there will be some drop downs that would occur, but I don't remember us doing any manipulation, there was no manipulation and then depending on what boxes you collect check that would also create other drop downs was my recollection, but there was no additional sort of like analysis of the data or inputting of the data.”

    • Craig Hayward: “That I can think of that may go to this, which is, I think it was possible to add a group that was not included in the derived in the data that was sent to the colleges so that may be the piece that you know can't be replicated or picked up kind of automatically from MIS”

    Issues/Questions Needing Resolution

    ...