Student Success, Deeper Insights Enabled by Data

On this episode, Tom interviews four UC Irvine leaders who have played major roles in the institution’s Student Success Initiative. Together they discuss the journey over several years to understand student success, the different aspects that impact it - from policies to what happens in the classroom - and how the role of technology and data are transforming the way we can personalize and improve the student experience. 

Podcast Guests:

• Michael Dennin, Vice Provost and Dean, Office of the Vice Provost for Teaching and Learning
• Patty Morales, Vice Provost for Enrollment Management
• Diane O’Dowd, Vice Provost for Academic Personnel
• Richard Arum, Professor of Sociology and Education

Transcript

Copy Transcript Copied

SUMMARY KEYWORDS

students, faculty, patty, data, teaching, uci, tom, diane, research, pandemic, diane, university, richard, people, campus, class, piece, year, peri, support, michael, education, space

SPEAKERS

Tom Andriola, Richard Arum, Diane O'Dowd, Patty Morales, Michael Dennin

Beginning:

Tom’s Intro: 00:33

At the end of this past year, I sat down with four UC Irvine leaders who have played major roles in the institution’s Student Success Initiative. Together we discussed the journey over several years to understand student success, the different aspects that impact it - from policies to what happens in the classroom - and how the role of technology and data are transforming the way we can personalize and improve the student experience. My guests today are Michael Dennin, vice Provost for teaching and learning and dean of undergraduate education, Patty Morales, vice provost for enrollment management, Diane O’Dowd, vice provost for academic personnel, and Richard Arum, professor of education and former dean for the school of education here at UC Irvine.

Tom Andriola  01:22

Welcome. So we are here to talk about UC Irvine and our journey around data driven student success. And we've put together a panel of experts and contributors to this effort, over the last several years, and we're here to actually hear the story of where we've gone. And so Michael, I'm going to start with you. Where did this effort begin?

Michael Dennin  01:41

What you have is, I think, a story of leadership engaging intentionally with data evidence around student success, and people choosing leadership positions with that intention in mind. So I'm going to speak for my colleagues briefly, Diane, you know, chose to become a vice provost about a year before I did, very much around the space of leveraging second pieces of evidence in teaching, and she can talk about what that means. I chose to become a vice provost in this space very much to impact sort of the structures of teaching, the data faculty use to think about their teaching, and the data used in the direct academic support of students success. Richard claims to have come here, in part because my role existed as a Dean of Education, because then the research and the implementation can be tied together. I'm going to give Patty credit for convincing the administration that we needed actual real enrollment management that was data driven. And so an entire new office in the sense that it was structurally new, that people had been here before was created. So you have these transformational leadership decisions being made all intentionally in this space to basically be coherent across the campus. So that's kind of the beginning and the structure of this whole effort. 

Tom Andriola  02:58

Absolutely. And these pieces are now coalescing together in a way that, you know, we only kind of initially imagined right now we're actually starting to make it happen. So Diane, I want to start with you. Can you talk a little bit about this second piece of evidence in the work that you did with HHMI? 

Diane O'Dowd  03:13

Sure. So I had gotten into doing some evidence based teaching and actually developing new strategies that teach in a student centered way, in very large classrooms of like 400 students. And it had been super exciting, because it was transformational to me to have a dialogue with my students, rather than just talking to them and having them listen. And so in thinking about taking the job as Vice Provost of academic personnel, who is really involved in overseeing all of the merit and promotion system, for our faculty at UCI, that, I felt like this would be a place where we could think about how do we both value and evaluate our teaching differently in a process, which is looking at both our contributions to research, to teaching and to service. And oftentimes research would out trump the other two, and I felt it was really important that we have a balance. And so in our review process, we had always included student evaluations, right? Students come into your class, they take your class, and then they fill out Oh, Diane O'Dowd, well, she's kind of boring. She just reads off her slides, or whatever. And that's how we use how we evaluated somebody's contribution to teaching. And that just didn't seem consistent with how we evaluated or valued our research contributions, where we looked at multiple pieces of evidence. So we decided to put in place and when I say we, when I became vice provost of academic personnel in collaboration with the Senate, because we're shared governance, we decided that we would do an experiment where we would require a second piece of evidence to evaluate teaching. So you had to submit your student evaluations and one additional thing. And the first year, we just said, here's a bucket, just stick whatever you have in that bucket. And we said, you could put your syllabus because everybody has a syllabus. So the faculty couldn't say, I don't have time to do that. You could put a teaching statement, you could put a teaching award that you won, whatever. And then I evaluated all of those things the first year to see how, which ones actually looked like were they were impactful for different levels of review, because I see the Department says, Oh, and this person's teaching is great because they got a six from the students out of x and they didn't say anything else. Whereas another one would say, well, they got good ratings from students, but they described a really interesting new strategy that they use in their class that increased student learning and this is how it did it. And so that was super excited to me. Only about 20% of the submissions fell in that category, actually, probably less, 10% fell in that category. But the next year, we gave guidelines that said that the information that was most helpful and effective in evaluating teaching was a reflective teaching statement, where you talked about what you were doing in the classroom, and the impact that that had. So that was really a way to not, now about 80% of our merit and promotion files have these and every file is discussed at the department level, where this is an organic discussion of new ways of both evaluating teaching, and I get to learn from Richard what he did in his classroom. And maybe I could apply that to my classroom, because he said his students learned more. And this is the data that showed how that convinced me that they learned more, so it was really an organic way to spread both excellent teaching practices, as well as consideration of additional pieces of evidence in evaluation of teaching. 

Tom Andriola  07:13

We throw around the term “teaching transformation” as part of this initiative. Michael, can you tell us what we really mean when we say that, and what we're doing to forward what happens here at a university?

Michael Dennin  07:25

For me, I think the realization I had when I when I was finally you know, an administrator and realized I had to worry about money, like that was the biggest difference between being a faculty member and an administrator, was the only thing that costs the university the same in the Student Success space, no matter what you do, and is scalable, are the faculty and faculty salaries, right? Everything else in student success and student support, costs more if you're trying to impact more students, but all students take classes, they take basically the same number, all faculty teach, no matter how they're teaching, whether it's towards, you know, using evidence based practices, and really enhancing student success, or, as Diane said, just kind of boring, and going on. So the place, in my mind, to most transform the effectiveness of the university, if we're going to serve the diverse student body we want to serve, is to transform how we think about teaching. And some of those can be very surprisingly small policy changes, some of it can be very little more involved in terms of adding active learning. There's a whole range. And I think that's where maybe Richard, I'll let you make a few comments about the things we're attempting to measure now, to kind of better understand, because faculty have a limited amount of time, if it is everybody has to totally radically change their teaching, that's a lot of effort and time and a big lift. But if it's piece by piece, and we can find for you, targeted things that you as a faculty can tweak, that's a much easier path forward. 

Richard Arum  09:02

This also goes back to your comment, Michael, about our frustrations with off the shelf products that used predictive analytical reporting from existing measures. The existing measures in the field were inadequate. They were simplistic, they came almost solely from administrative records around students course taking, credit hours and grades, and their descriptive characteristics when they entered. To really do something transformative in this space, you have to develop a different set of measures. And so one of the things we did around the teaching space is the learning management systems that all exist now on all campuses. Students are interfacing with their course materials and their instructors. It's all mediated by these learning management systems. At UCI, it's Canvas. And so every time a student engages in that site, clicks on the space, we have a record that they're doing something around their learning. And yet, before we got this program off the ground, that data was being wasted, nothing was being done with it. And that's not just true here at UCI, it's true, really across the field as a whole. And so what we've been able to develop over the last few years here is for measures for every student in every course at UCI, measures of academic engagement, measures of the extent to which they're posting things online for their peers to observe, responding to peer discussion posts, engaging on a regular basis, weekly and daily. And so those sorts of fine grained measures of student’s experiences can be extraordinarily informative to efforts to transform teaching. And then, on top of that, we also started surveying students, some on a weekly basis, about their experiences inside the classroom and outside of the classroom, again, to really be able to understand their experiences, their trajectories, and their outcomes here at UCI. And again, unless you have these kinds of new measures to make use of, and you're just using kind of the old 20th century measures from the administrative data systems, you're not going to be able to kind of transform and move the institution forward. 

Tom Andriola  11:34

And is this just research? Or is this also about changing practice? And how do we get to changing practice?

Diane O'Dowd  11:42

I think it's really important to know how students are reacting to it, but it's really important to know how faculty are thinking about these practices and using them. So in some of the courses that we taught, there are always things that the students don't like to do, but if you find out that this increases their performance on a summative or formative exam, then they're much more willing to do it. So, for example, I introduced class cards into my class 20 years ago where they had to write a response with a group and turn it in. And the first year they wrote, we hate these class cards, they're just a pain. And so the next year, I did class cards for every other assignment that was associated with a set of assessments that we would do, and the ones where they did class cards, they did much better. And so I showed them the data. And the next year they said, well, we hate these class cards, but we know it improves our learning, so we like having them. And then showing faculty that data with the same students, my students in freshman year are their students in sophomore year. So it's not like seeing a paper that's written at Claremont McKenna, which is a small, elite, you know, school where there are 40 students in the classroom. It's where we have 400 students in the classroom. So providing that kind of information to faculty, as well as to students can really help change practice, but also, it's part of our research mission is to look at things in an evaluative and critical way.

Richard Arum  13:21

Yeah, I could jump in. So not on campus here, not just an openness to these measures, but you know, active support and buy in, I can give you a couple of examples. One is the measurement project. We went to the student senate, the student, undergraduate Senate and said, this is the measurement Project we'd like to do on campus. What do you think? And they formally endorsed the project as essential for supporting their own success. All the changes that Diane implemented through her office went through the Academic Senate. So it's, it's the faculty and the students, they're not resisting this move. They're embracing it and looking for ways it can be used to improve how we work together. 

Tom Andriola  14:08

Patty, so when you got here, you started enrollment management analytics, what unmet need, did you see when you got here, and what led then to developing the capability your team has given us?

Patty Morales  14:16

Sure. So when I started at UCI, I was the Director of Undergraduate Admissions. And if I just think about the admission application as a starting point, what is the purpose of it? Well, students are telling us a lot about themselves. And those of us on the admission and evaluation side are presumably using that to make a decision of whether or not that student might be a good fit in our community. That's all well and good. But then we still have all that information. And similarly, I think Diane was saying something to this point earlier, it was maybe Richard, probably both, it was just sitting there, we were collecting all of this wonderful information about students, using it for that one purpose, and then not doing anything else with it. Moreover, data and financial aid again, using it for a specific purpose of determining a student's need, so we can award financial aid, but then it's just sitting there in a database, registrar data, all the students course taking patterns, all the faculties course scheduling patterns, degree audit, progress, etc. Not using it for any purpose. So the unmet need was that I saw around campus, faculty administrators needing to make a lot of decisions about policies or practices, sometimes systems were being replaced, a lot of that is, of course, taking place right now. And when you're replacing a system, you have to update a business process typically. And they were making these decisions or approaching those activities without a lot of data behind it, using their own experience and professionalism, which is, of course, essential. But there was a lot of data that they could be turning to as well to see around corners, or make discoveries about things they just hadn't even had on their mind. Moreover, I felt that especially for example, with admissions data, students are telling us about themselves. I felt an obligation of stewardship for that to say, okay, this wasn't just for a moment of making a decision of whether or not you were admissible to the university, but I felt we have an obligation then to to make our university receptive to them, to ensure that we were keeping pace with who our students are, what they're telling us about themselves, we have a responsibility to understand what their learning pathways have been, especially changing demographics, especially as we broaden our reach. We're reaching students in every part of the state in every part of the country, in the world. That's a lot of information, a lot of difference. And if the university isn't ready to make those pivots, and have those responses to all of those changes, and evolutions we'll become stale pretty quickly. And that fit, that magic sort of fit between student and institution will start to get more and more strained. So I feel that that data, for example, is essential to ensuring that we're providing. Mike and I, our offices work in very close collaboration, because he oversees a lot of the students support services. In order for those to be optimized, his folks and his programming need to reflect the needs of the students who are coming in. And the only way that he can do that effectively is if he understands, well, who are they? How have they been changing in terms of their preparation, demographics, expectations, the way that they learn modality, this these days is especially important. All of that information is contained within the student's application for admission, but then also, they start to emerge within financial aid data, even registrar data where you have, like I mentioned, all the course taking patterns, where there's just a wealth of information that when it's pieced together, helps develop a picture that then can be shared with other colleagues, with faculty and deans and other campus planners to ensure that we're keeping ahead of what the students will need.

Tom Andriola  18:50

So we're really building this holistic picture of the individual who's coming here to join our committee, not not an individual slice, but really seeing the full 360 view.

Richard Arum  19:00

It's very important; the students are not the problem. The institution needs to adjust its practices to meet the needs of the students. And that shift right to kind of like a more asset-based approach that recognizes that the students that come into UCI are extraordinary. I mean, I’m sitting next to the woman who's in charge of that project, what 115...?

Patty Morales  19:24

142,000 applications last year.

Richard Arum  19:29

I can't keep with the number of applications. Yeah, so the selection of these students, they're so special, they're so gifted, highly motivated. Large numbers are low income. Large numbers are first generation. 85% are non white, non white Hispanic, that's incredible for an R1 institution, but they have incredible assets and gifts. And it's our obligation to use this data to figure out what's not working, and how to fix it and address it.

Diane O'Dowd  20:01

I have one thing I'd like to say about scheduling classes, which is I think Michael made the brilliant decision of requiring people, faculty, to schedule their classes in the brand new Anteater Learning Pavilion that the Chancellor supported building a totally a building totally associated with classrooms that have high technology and active learning spaces. And Michael says, wait, can't schedule this until you demonstrate to us that you have clear training and facility with using active learning in the classroom. And in when I used to schedule my classes, it was did I have it last year, then I get preference this year, whereas this way, nobody had had any of those classrooms, and they couldn't get in, and everybody wants to teach in a new fancy space. They couldn't get in unless they took that course or demonstrated their facility with this student centered approach to learning that has been so important for the transformation on our campus. So that was Michael's decision to say, nope, can't teach in there unless you do that.

Richard Arum  21:13

I was going to add one other piece of information, so, about the pride the faculty took in this work. So part of it was the incentives to get access to the new classroom. But remember, at the same point in time, the New York Times came out with this new ranking, and ranked UCI as the number one in the country for doing the most to support the American dream of a large research university, taking students from low income backgrounds and moving them into the top fifth of the the income distribution. And so that that cultural message that was widely shared, nationally, faculty took pride in and so I think a lot of faculty leaned into this work of serving students, because they saw that they were getting not they and the institution was getting nationally recognized for that, that excellence.

Tom Andriola  22:12

Okay, so here we are 2022. We've done a lot of amazing things to get us here. We've built off of successes, we've built a momentum to cultural change and putting students in the center of what we do. In this kind of initiative of bringing the pieces together, right, we've identified a couple of units, one is called PERI Squared. Someone tell me what is PERI Squared, what is it going to do?

Michael Dennin  22:37

So it's hopefully a transformative way of looking at a research institute. The acronym is post secondary educational research and implementation Institute. That's the squared at the end, the PERI Squared. And the key part of it is the research and implementation piece. It came out of, I mean, Richard said it to me, and I've said it to other people and to Richard, when he was dean of the School of Ed, and it's still a collaboration that's continuing, you'd be hard pressed to find a school of ed, and an Office of Teaching and Learning that collaborated as much as we do. I challenge people to find that. That research, and implementation collaboration was informal. And this entity formalizes that and says, you know, it's sort of like translational medicine, right? You want, if you're doing research, and that could lead to a better drug, you want to get the better drug, you don't want to just keep doing the research. And Diane, you said it a number of times, as a research university, our obligation is to do the research, but then to actually translate it into action and success. And PERI Squared is kind of the institute that at least makes that formal and visible. That's kind of the high level piece.

Diane O'Dowd  23:53

Right. So traditionally, research universities have their tenured research faculty who do lots of research and some teaching. And then we had contingent faculty, like lecturers or adjunct professors that would do lots of teaching and didn't do research. And then in the UC system, we had a really interesting title that wasn't used very much that were faculty who had the majority of their work, or effort associated with teaching, yet they still were required to do scholarly work research in either pedagogy or in a discipline like education, or they could do it in biology. And these are the people that are both the practitioners and the people that are developing and assessing and using new technology that are embedded within the departments. So a department will have maybe 20 research professors, and two professors of teaching. In that mix, you end up with experts in your own department where I don't have to go to the Division of Teaching and Learning, I really don't want to take the time to do that. I can go to Richard who is right next door to me, and I can say, you know, I am really interested in understanding how you can increase student's understanding of photosynthesis, I cannot get it across and you can help me with that. And then when you come to me, you can say, well, I really need to teach our students about gene editing. Can you give me help with that? So it's a interpolation, that is really important, that having departments of Chemistry, Education, in my view, is way less effective than having a Department of Chemistry, who has some people who are focused on the scholarship of chemistry education, but are chemists themselves, and that, that have people that are focused on generation of new knowledge and chemistry. And so we went from 2% of our faculty being in this position in 2012, to 9%. And they have been for, from my point of view, super transformative, not just in educating our own students here on campus using evidence based and student centered practices, but influencing lots of other people on campus faculty on campus to do that, but influencing the entire nation and world because they're publishing and presenting this new way of engaging students in a research university. And I think that that has really been transformative. And I do feel like we're leaders in this space. And that's because of our professors of teaching who have been enormously effective.

Tom Andriola  26:48

That's great. Okay, we'll talk a little bit about COMPASS, right? If PERI Squared is the research and translation arm, we sometimes say COMPASS is the toolkit that is available to our community. Tell me about the future for COMPASS.

Michael Dennin  27:03

Oh, well, Patty and I are gonna make it incredibly successful. Do you need to know anything else? COMPASS is one of the few acronyms that I think worked out really well. It's comprehensive analytics for students success. It was a reverse engineered acronym. People wanted COMPASS, and they figured out what words went to it. But it was one of these very, very luckily, forward looking things because it started with what we now call Teaching and Learning Analytics reports, mainly for faculty, some reports for the academic advisors, it was first emerging, as we talked earlier about needing some analytics for our advisors, we did learn that from the Student Success Collaborative, what's been really nice is in particularly in the advising space, and I'll let Patty talk about it more, we had the enrollment management analytics, because really, until that team came on, we were kind of limited. Luckily, the tools for the advisors were so poor, that ours were a brilliant step forward and got them really excited. And then when Patty's team showed them another step forward, that was even better. But it's really these buckets of tools for academic administrators, department chairs, Deans, when they're dealing with, hopefully thinking about evaluating teaching and sending cases forward. But also when they're talking about going to the provost and saying, here's where we're doing really well in changing our structures. As Richard said, it's not the students we recognized, here's these policies in our department that are holding students back, so we're going to change them. So there's, there's that space. Individual faculty, you know, just a lot of universities are doing this now, where you're getting data on the historical gaps in your teaching, or what your grade distributions look like, even the single simple thing is, who are the students in your class, you know? When UCI started really ramping up the number of international students, they don't uniformly go across the campus. So it's one thing to say, Oh, we're 18% International. But your class might be 60% International, and you know that, and if you've never taught in that space, it really changes the class or, you know, your classes now, suddenly 60% First Gen. What does that mean? And then the advisors, there's just so many useful issues and things there. And there's some overlap. But loosely speaking, like our office tends to focus on the faculty and their needs based on what we know for merit and promotion and out of maybe data that Richard's team is doing or research, and Patty's team, I'll let her talk, talk focuses a lot in the academic advising space, and then we collaborate where necessary. So I think it's kind of that collaborative, but also expertise, where there's expertise.

Patty Morales  29:42

Yes. And to extend the metaphor of COMPASS of what we've been doing, we being my team, and also Mike's team and other colleagues and institutional research and Office of Information Technology, is developing a sense of ourselves as navigators. You know, a compass is a great tool, it can tell you where things are, this is north, this is West, but you still in most cases, need a navigator to help guide you to where it is you're trying to get. Especially if there are obstacles. If the terrain is unfamiliar to you, etc. Navigators sort of sit at a higher level, they can see the bigger picture. They can advise on more effective or efficient pathways, etc. So we're thinking about ourselves, we again, being our staff, as those navigators and working with our colleagues directly to have conversations in what they need. And so that engagement with the advisors that my team is doing is very much about hearing from them, what their challenges are, what's on their minds, what they wish they knew. And a lot of times, that's how we open the conversation. We go to all of the academic units. I meet with Deans on a regular basis just to understand their enrollment goals. But underneath their enrollment goals are a whole set of layers that are up, you know, they're not just choosing a number out of a hat and not just saying that they want a certain sort of distribution of majors just because if you know, they happen to have this many faculty here and that many there That's part of it. But usually they have a vision in mind for what they want their school to be known for and to achieve. And the undergraduate enrollment and the graduate enrollment all have to complement that, they have to support it. And underneath all of that is a whole lot of people who are charged with ensuring that the students get there and are successful there, retain there, unless actually the students discover that's not the major that they may want to be in and then to effectively guide them elsewhere. But in order to do that, they have to know who their students are, they have to understand what their challenges might be. So when my team talks with those advisors, and others, those are the questions we're asking. When you're trying to advise your students or thinking about planning how to distribute your resources, what do you wish you knew? And then they'll tell us that, and we know the data we have available. And so we'll look at the data and think, okay, well, how about this? How about if we showed you this view? And you were able to filter it in these ways? And then also compare it with this other piece of information? Would that be helpful? And you see their faces light up. But then we iterate. So the reports and the views, we call them views, that we're building are informed by what their needs are, we're not just saying, hey, wouldn't it be cool if we develop this tool and then expect that folks are going to immediately know what to do with it or want to use it, they might. But a lot of times, they already know what they need, they just may not know the best way to arrange the data or even what data is available. And that's been very gratifying. And so ideally, all of those pieces work well with one another to support the vision of the dean, and most importantly, to support the student and ensure that things make sense to the student, that their experience has a fluidity to it. They feel supported, they feel seen and understood. And so it's working in both of those areas. And again, in order to do that you have to navigate, you have to be able to see the whole picture. And then we can help our folks use their compass, use their tool.

Tom Andriola  33:40

So I think a question we're going to ask ourselves a lot in the future is, what opportunity did the pandemic give us to change the way the world works? So we've got a little bit of time that has passed now, what's the pandemic give us the opportunity to do and what are we taking forward from it?

Patty Morales  34:02

I'll jump in here, if that's okay. I think it gave us an opportunity to or permission to work differently, to feel that you really can change something and it will still not only be okay, it might even be better. I think a lot of times, there are folks who just love change who want to do things differently, but it's hard to do if not everybody is on board, etc. But what the pandemic demonstrated for us is, you can take something, radically change it, turn it inside out, put it in a whole different kind of structure, and it can still work. It might even work better. It may work in unexpected ways. And maybe some things don't. But it gave us an opportunity to pull back and take that into consideration, and see what we want to do moving forward. Because there are now things that we will continue, how we develop syllabi, how modalities considered, etc. But then there are other things where we really learned that relationships are hard to do exclusively over zoom. They're not impossible, but they're hard. And then you investigate, well, why does that interpersonal moment matter so much? And how do you get the best of both worlds?

Richard Arum  36:21

I think that just to add something, I think I think the pandemic, you have to recognize it in the context of all the other stuff that's been happening over the past few years. So the Black Lives Matter movement, the pandemic, the insurrection in, in, in DC, the California wildfires, there's so much unrest and churn in the system, that in institutions facing that much noise, you better be very focused on the data, and to be able to adapt to the changing environment. And so as an example of that, in the in the middle of this, we we asked the students in our surveys, when we come out of the pandemic, what kind of education do you want, you want everything back in the classroom, you want everything online, you want a mix of this, they said we want to mix, we don't want to go back to the old ways, nor do we want to embrace this. But we knew that, because we were asking the students, we knew that probably a year before most of our peer institutions. And so we then kind of I think, to a certain extent, got out in front of that and started to move some courses to online, some hybrid, to respond to the changing environment. And so I think, going forward, forget the pandemic, going forward, we're going to be facing this turbulent environment for higher education, for society. And in that context, we better figure out how to use data in a way to kind of address the challenges.

Tom Andriola  37:08

So data is not just about insight, it's also about providing for and agility to deal with uncertain situations. Diane, you want to add?

Diane O'Dowd  37:17

I think the thing that made the biggest impact on me was something that Michael said and has been promoting is to really be thinking differently about how we grade our students and evaluate their performance. And in traditional times, it's relatively easy to have a final term paper or a final exam. And that wasn't the case in the pandemic times where things had to be moved quickly. And the idea that, well, how can I change this, when I have my structure set in stone, in a week? And to be thinking about, should we be using those as our tools in the first place? And if we are, can we use them more effectively? And this idea that I just heard the other day, I thought was so brilliant, and I thought I would like to use that which was, you know, okay, somebody just, had just given a paper exam with, you know, 300 students and they didn't have people to help grade them. And so what do I do? And the idea was you would then give the students the answer key, they would look through the answer key and try and decide, should I ask for my paper to be graded? Because I think I did super well. And I understood what I was supposed to write in this paragraph. Or say, I might not have done as well as I thought, I think I missed some points. But they will have gotten so much more out of that exercise than if they just got a grade, and they moved on to their next class, because I got a B minus, I don't know what I missed, but I'm not going to look at it, because I don't want to. And so this idea that you could do things differently in a way that was maybe more efficient and more effective and got students, again, this is about an engagement piece. But I think Michael has been pushing, you know, having less of this sort of final exam, final paper type idea in higher education at research institutions and broaden that out to think how can we improve student learning in a way that is better than those tools?

Tom Andriola  39:35

If we, if we think about this, from our students perspective, a lot has been talked about here that you could say is, you know, being student centered, right, where we're really shifting and becoming more student centered. Let's put ourselves in the students' shoes for just a second. How is the student experience changing from their perspective? And how would we say it's improved for them in terms of the experience they get here at UCI?

Michael Dennin  39:58

I'm hoping Richard's surveys tell us. I mean, I meet a lot with many students, mostly these days with student government, we have 30,000 students. So I'm serious. It's going to take something like the surveys Richard does, because otherwise, it's very, analytic, anecdotal. And it's very, you know, I mean, it was what I learned during the pandemic, no matter what you did with 30,000, students, 5,000 don't like it. And so you get a lot of complaints. And so even just this question of are we supposed to be online or remote, the surveys helped us with. What I do think is because we've realized, we can change our policies and our procedures and our structures, as Diane said, we will make a lot of things that the students immediately see as better. At the same time, there's even a really great paper on this, as Diane pointed out with her class, the students will probably never like those cards, as much as they know that it'll help them learn, but you'll have an agreement with the students and a clear understanding of, they will know why you're doing this, because you can point to the data, they will still know why they don't like it, but they will be much more likely to accept it than just some arbitrary change, you know, that Professor O'Dowd is mean and did to us. So it's going to be rich, it's going to be complicated, there's going to be things they don't like that they know are good for them. And there'll be some things if we don't get the message out, well, they won't know why we're doing it. But this is why I mean, I jumped in and said it and took Richard’s thunder, but something like those surveys is just such a key piece of the whole strategy.

Patty Morales  41:36

But I would also throw in the administrative friction that is so present for so many students and becomes a big reason that they may face challenges. We have lots of data now to help us ameliorate that. And it's also a technology. And we need the technology to also enable the things that we see and want to be able to do in some cases, we have been able to do that in recent years. And others, we're still kind of working our way through legacy systems that do handcuff us a bit. But we already know what we're going to do as soon as we get that new system, we're already planning for it. And the way that we're able to do that is because when students interact with each of our sort of administrative, whether it's the student billing office, financial aid, registrar's office, check in with their advisors, etc. They're leaving data breadcrumbs, and we're gathering those and trying to see what we can discern from them. I think the metaphor ends there I don't know. (laughing) But we are able to redesign our processes, reimagine them, usually it involves working across divisions, working cross functionally, we say, right, because a student doesn't think of the university in structural ways, the way it tends to be set up. The billing services, the Office of Student Financial Services, for example, does not report through me. But there's a very close relationship with the Office of Financial Aid and Scholarships, which does report through me. But for a student, they get a bill, and the way that bill gets paid is largely for many students through financial aid, these two things have to make sense, they have to work together. Great colleagues are on each side and they get along well, and can do a lot to facilitate if something goes, you know, awry, but it makes so much more sense if we're able to get our processes lined up. And how will we know what to do? Well, we have to analyze the data, we have to understand who's getting snagged up in these processes, where are those friction points, and it is all able to be seen just in the data, and then we can make those adaptations.

Richard Arum  43:57

So Michael, set me up for this. So you know, in addition to tracking students and seeing, did they get the degrees they wanted? Do they get them in a certain amount of time, you know, four years, five years, for some reasonable amount of time from when they started, did they get the jobs they want? You can use traditional forms of data to answer those questions. But we owe it to these students to ask other questions that in the past, were only asked at elite private universities. Questions like, do develop meaning and purpose in your life from this experience at UCI? Did you develop social networks and friends that persisted after you graduated, into help support your transition into the into adult roles in society? These are things you can ask with surveys. And I think it's essential to the mission of this university. Because we believe in the holistic definition of a student, and think that they deserve that type of education at a place like UCI.

Tom Andriola  46:15

Okay, last question. We're five years out, what do you want UCI to be known for and for our peers to say about us?

Michael Dennin  1:03:34

So I don't know if we can do this in five years Tom, but really, you know, Richard mentioned, there's a lot going on, including not just the pandemic, but Black Lives Matter. And there's a real, and he said, you know, it's not the students at fault, it's the institution, there's a, there's a growing recognition of whether you call it, structural racism, institutional barriers, whatever kind of label you put on the fact that structures of institutions do matter. And when I talk, I'm a physicist, I can't help but talk about the transfer function, we have an input, which is our students, something happens to them while they're here, and we have an output, which happens to also be our students, hopefully transformed in the way Richard mentioned, right? It's just a well known fact of systems. If you change the input, and you're using the same transfer function over and over, you're not going to get the output you wanted when you optimize it for a different set of inputs. And we take great pride in the diversity of our incoming students and serving all students. I would like to be known as the research university that has figured out the new transfer function that takes a diverse set of students as an input, and creates the same amazing output we've always done. And people talk about the Harvard Model for research universities. We all follow the model that Harvard supposedly created in the 1800s. In five years, I'm going to be ambitious because you told me that's what the timeframe was, there will be the UCI model of being a research university, and everybody will be scrambling to follow that. That's my goal. Sorry, it's really probably my 20 year goal, if I'm honest, but you know, you ask for five.

Richard Arum  47:05

Well, let me just jump in. So I'm gonna go back to the American Dream theme. So you know, if in five years, we could, UCI could earn that moniker doing the most to support the American dream, but the American Dream is defined, not just about upward mobility, which upward, which is part of it, the upward social mobility, but also about civic engagement, and a defensive democracy, and community. And it's not just for the elites, but it's for all students. The American dream is for students, including undocumented students here in California, that are at UCI It includes the incarcerated students that we're now providing education to, for the first time this year. That's the American Dream, for me, and if data can be a small part of understanding how to do that work, and then also telling the story to a larger public about the value of what we're doing, then I think we'll have been successful.

Tom Andriola  48:21

You know, I had a chance to spend two days with leaders from about 40 or 50 organizations that was sponsored by our business school, through what they call the Center for Digital Transformation. And I can't tell you that I had a conversation over two days, and this is, you know, financial institutions, pharmaceutical companies, manufacturers, entertainment companies, that the future and everything they were trying to do ultimately came back to really one word, and it was not strategy. It was culture. And one of the things that really strikes me about you know, underlying everything that's said here is really about building a culture here top to bottom, left to right, across the entire community, about a culture that focused on helping people succeed. You know, technology and data, they're just enablers for us to achieve that. But the culture, the people, the commitment, is what really, really matters. I want to thank you so much for spending the time today to talk about the past, the history, how we got here, and also painting a vision for where we're going. I'd like to thank you, for those of you are listening. UCI is an amazing place. In the three years that I've been able to join this community and this family, I've learned so much and they've broadened my understanding. You got to hear from some of our leaders today who are making this transformation and this change happen, and building the culture for tomorrow so that no student gets left behind and that every student gets the opportunity to fulfill their American dream. So thank you for joining us today.