SO I'M GOING TO GO AHEAD AND GET OUR RECORDING STARTED HERE AND WELCOME FIRST TODAY'S MODERATOR.
IT'S BEEN SUCH A PLEASURE TO WORK WITH MARIE RADFORD WHO COMES TO US AS PROFESSOR OF LIBRARY AND INFORMATION SCIENCE AT RUTGERS UNIVERSITY'S SCHOOL OF COMMUNICATION AND INFORMATION.
AND I'M GOING TO LET MARIE WELCOME OUR GUEST PRESENTER FOR TODAY.
WELCOME MARIE.
>> OKAY THANK YOU VERY MUCH JENNIFER.
IT'S MY PLEASURE TO BE HERE WITH ALL OF YOU THIS AFTERNOON.
I HAVE TO THANK LIN, MY PARTNER IN CRIME IN A LOT OF ACTIVITIES, INCLUDING OUR RESEARCH METHODS BOOK WHO INVITED ME TODAY TO BE YOUR MODERATOR.
I DID HAVE A CONVERSATION WITH LIN TO TALK TO HER A LITTLE BIT ABOUT WHAT SHE MIGHT EXPECT ME TO SAY PRIOR TO INTRODUCING MELISSA AND SHE SENDS HER GREETINGS.
SHE'S IN COLOMBIA, SOUTH AMERICA, THESE DAYS.
SO ONE OF THE THINGS THAT SHE ASKED ME TO REMIND ALL OF YOU IS TO THINK IN TERMS OF ASSESSMENT AS A DYNAMIC PROCESS THAT DOESN'T ACTUALLY TRAVEL IN AS MUCH OF A STRAIGHT LINE AS WE WOULD LIKE IT TO AT TIMES.
BUT THAT SHE ASKS YOU TO MAKE SURE THAT AS YOU MOVE INTO THE FINISHING TIMES OF YOUR RESEARCH AS MELISSA IS GOING TO BE TALKING TO US TODAY ABOUT, PRESENTING YOUR RESEARCH FINDINGS AND MAKING YOUR CASE.
TO REFLECT ON DO YOU HAVE THE RIGHT RESEARCH QUESTIONS AND APPROPRIATE METHOD.
WHAT SHE WANTED ME TO TALK A LITTLE BIT ABOUT WAS THIS IDEA BEGINNING WITH THE END IN MIND, THAT AS WE EMBARK ON ANY ASSESSMENT PROCESS, WE WANT TO THINK ABOUT WHAT IS IT THAT WE WANT TO BE ABLE TO SAY AT THE END OF THE PROJECT.
WHAT KINDS OF CASES ARE WE MAKING TO OUR PARTICULAR STAKEHOLDER GROUPS?
AND AS WE'RE MOVING THROUGH THE ASSESSMENT PROCESS, HAVE WE ACTUALLY USED THE APPROPRIATE METHODS?
ARE WE ASKING THE RIGHT QUESTIONS?
HAVE WE BITTEN OFF A CHUNK THAT IS ACTUALLY, YOU KNOW, MANAGEABLE, THAT'S DOABLE, THAT'S REALISTIC SO THAT WHEN WE GET TO THE END THAT WE DO HAVE SOMETHING THAT WE ARE PROUD TO TALK ABOUT AND WE HAVE COLLECTED THAT APPROPRIATE INFORMATION ALL ALONG THE WAY.
SO ONE OF THE THINGS TO THINK ABOUT IS HOW THIS PROCESS FOR YOURSELF, WHEREVER YOU ARE IN IT, HOW IT IS UNFOLDING AND THINKING ABOUT THIS AND THAT WE HAVE IN MIND SO WE'RE VERY EXCITED TO HEAR WHAT MELISSA HAS TO SAY TODAY.
I ALSO WOULD LIKE TO INVITE YOU ALONG THE WAY AS JENNIFER MENTIONED, IF YOU HAVE QUESTIONS TO MELISSA ALONG THE WAY PLEASE DON'T HESITATE TO CHAT THEM INTO THE CHAT BOX AND I KNOW JENNIFER IS GOING TO BE HELPING US TO WRANGLE THESE AND LOOK AT THESE ALONG THE WAY AND ALSO, WE WILL BE ALLOWING TIME FOR SOME QUESTIONS AT THE END.
SO LET ME MOVE TO THEN INTRODUCING MELISSA.
MELISSA IS HEAD OF EDUCATIONAL INITIATIVES AT THE UNIVERSITY OF NEVADA FOR THE LAS VEGAS LIBRARIES WHERE SHE HOLDS THE POSITION OF ASSOCIATE PROFESSOR.
SHE ALSO TEACHES AT THE SYRACUSE SCHOOL AND SHOUT-OUT TO SYRACUSE, THAT'S WHERE I GOT MY MLIS.
MELISSA IS TEACHING LIBRARY PLANNING, MARKETING AND ASSESSMENT THERE.
SO WELCOME, MELISSA.
>> OKAY.
THANK YOU SO MUCH, MARIE AND JENNIFER.
I AM REALLY PLEASED TO BE ABLE TO OFFER THE THIRD PART OF THIS SERIES ON ASSESSMENT, WHICH I THINK OF AS THE "CLOSING THE LOOP" PART.
AS MARIE JUST SAID ASSESSMENT IS A DYNAMIC PROCESS WHICH KINDS OF MEANS THAT YOU'RE NEVER DONE.
SO WE DO TALK ABOUT CLOSING THE LOOP AND SORT OF FINISHING OUT WHATEVER RESEARCH QUESTION THAT YOU'RE WORKING ON, BUT WE WILL TALK AT THE END ABOUT WHAT OTHER QUESTIONS MIGHT BE REVEALED AS YOU GO THROUGH THE PROCESS BECAUSE A LOT OF TIMES AS YOU'RE DOING AN ASSESSMENT PROJECT, YOU'LL COME UP WITH MORE QUESTIONS THAN ANSWERS.
SO I'M EXCITED TO TALK ABOUT THAT WITH YOU TODAY.
BUT WHAT OUR AGENDA IS FOR TODAY IS TO TALK ABOUT ONCE YOU'VE DESIGNED AN ASSESSMENT PROJECT, YOU'VE GATHERED DATA, YOU'VE ANALYZED THAT DATA, WHAT DO YOU DO WITH IT?
THE WHOLE POINT OF THE ASSESSMENT PROCESS IS TO LEARN SOMETHING ABOUT IMPROVING RESOURCES, CREATING A SERVICE OR BETTER SERVING A POPULATION.
SO WE'RE GOING TO TALK ABOUT HOW YOU MIGHT DO THAT.
ONE WAY I THINK ABOUT THIS IS WHAT STORY DOES YOUR ASSESSMENT DATA TELL?
AND I KNOW THAT MANY OF YOU WHO ARE ENGAGED IN THIS SERIES HAVE WORKED ON VARIOUS TYPES OF PROJECTS.
FOLKS HAVE BEEN WORKING ON THINGS LIKE TESTING SEARCH TOOL USABILITY, DOING USER NEEDS ASSESSMENTS, DOING POSTWORKSHOP SURVEYS TO FIND OUT WHAT KIND OF IMPACT DIFFERENT PROGRAMS MADE AND FOR ALL OF THOSE TYPES OF PROJECTS, YOU'RE GUIDED BY A QUESTION AS YOU GATHER INFORMATION.
WHAT DOES THE DATA TELL YOU?
SO IN OUR WEBINAR TODAY, I'M GOING TO WALK THROUGH A SERIES OF SEVERAL QUESTIONS THAT WILL HOPEFULLY HELP YOU FIGURE OUT HOW TO TELL A REALLY COMPELLING STORY BASED ON THE DATA THAT YOU CAN ALSO USE TO MAKE MEANINGFUL CHANGES.
AND I'LL JUST CLARIFY THAT I DON'T MEAN STORYTELLING IN TERMS OF MAKING UP FIX BUT TAKING USAGE STATISTICS OR QUALITATIVE SURVEY RESPONSES AND MAKING THEM COMPREHENSIBLE AND COMPELLING FOR DIFFERENT TYPES OF AUDIENCES, BOTH INTERNAL AND EXTERNAL TO YOUR LIBRARY OR YOUR INSTITUTION.
SO JUST A NOTE, I'LL CALL BACK TO WHAT JENNIFER SAID IN THE INTRODUCTION.
IF YOU'RE INTERESTED, I WOULD LOVE FOR YOU TO FOLLOW ALONG WITH THE WEBINAR LEARNER GUIDE.
IT LISTS SOME OF THE REALLY IMPORTANT QUESTIONS THAT YOU NEED TO ADDRESS AS YOU'RE THINKING ABOUT WHAT TO DO WITH THE RICH INFORMATION THAT YOU HAVE COLLECTED.
SO THE FIRST QUESTION TO ASK, WHICH YOU PROBABLY THOUGHT ABOUT AS YOU WERE STARTING TO GATHER DATA IS WHAT WILL BE -- WHO AND WHAT WILL BE IMPACTED BY THE INFORMATION YOU'VE GATHERED?
NOW, YOU MAY HAVE MEANT ORIGINALLY TO JUST GATHER INFORMATION ABOUT ONE THING.
SAY IT WAS YOUR SEARCH TOOL IN ORDER TO IMPROVE IT, BUT IT'S IMPORTANT AFTER YOU'VE GATHERED THAT DATA TO CONSIDER THE BIGGER PICTURE OF WHICH SERVICES THAT'S GOING TO IMPACT.
SO, FOR EXAMPLE, WITH A SEARCH TOOL, ANY CHANGES THAT YOU DECIDE TO MAKE TO THAT MAJOR LIBRARY SEARCH INTERFACE WILL CERTAINLY IMPACT INSTRUCTION AND REFERENCE AND THERE ARE MANY DOMINO EFFECTS THAT WE MAY NOT PREDICT SO THE FIRST REALLY IMPORTANT THING TO CONSIDER IS GIVEN THE DATA THAT YOU'VE GATHERED, WHICH LIBRARY SERVICES ARE LIKELY TO BE IMPACTED?
AND IT MAY GO ON FROM HERE SO IF YOU HAVE THOUGHTS ABOUT THIS AND SOME OF THE RESOURCES AND SERVICES THAT YOU ARE LIKELY TO CHANGE, PLEASE DO GO AHEAD AND POST THOSE IN THE CHAT.
I'M INTERESTED TO SEE WHETHER WE HAVE OVERLAP IN THE TYPES OF PROJECTS THAT FOLKS ARE DOING.
NOW, BEFORE I MOVE ON TO THE OTHER QUESTIONS, I'M GOING TO TALK A BIT ABOUT SOME EXACTLY FROM MY OWN RECENT EXPERIENCE ON ASSESSMENT PROJECTS THAT HAVE LED TO CHANGED OR REVISED SERVICES AND HAVE ALSO SHAPED A LIBRARY MESSAGE GOING FORWARD.
BEGINNING WITH THE END IN LINED AS MARIE SAID, I'LL JUST GIVE YOU SOME THOUGHTS AND EXAMPLES OF THINGS WHERE WE'VE TAKEN FINDINGS AND PUT THOSE INTO ACTION IN DIFFERENT WAYS.
SOMETIMES, THE ACTION IS MORE OF A MESSAGE AND SOMETIMES, IT'S AN ACTUAL ACTION OR EVENT OR ACTIVITY.
SO ONE EXAMPLE IS REGARDING TRANSFER STUDENTS AT OUR LIBRARY AT UNLV.
WE ARE VERY FOCUSED ON STUDENT SUCCESS AND STUDENT SERVICES, BUT WE REALIZED FAIRLY RECENTLY THAT WE HAD NOT DONE SPECIFIC OUTREACH TO TRANSFER STUDENTS AND WE WERE INTERESTED IN FINDING OUT WHAT THEY NEEDED OR WHAT THEY MIGHT EXPECT FROM THE LIBRARY GIVEN THAT THEY MIGHT BE COMING FROM DIFFERENT LOCAL COMMUNITY COLLEGES.
THEY MIGHT BE COMING FROM UNIVERSITIES FROM OUT OF STATE.
THEY MIGHT BE RETURNING TO SCHOOL AFTER A LONG ABSENCE.
SO WE WERE INTERESTED TO LEARN WHAT WE COULD DO TO PROMOTE THEIR SUCCESS.
SO MY COLLEAGUES DID A SURVEY OF ALL TRANSFER STUDENTS AND GOT QUITE A FEW RESPONSES, AND THEN THEY DID FOLLOW-UP INTERVIEWS AS WELL AND ONE THING I HAD SORT OF EXPECTED TO LEARN -- I THOUGHT THERE WOULD BE GAPS IN WHAT THEY KNEW ABOUT LIBRARIES OR THEIR EXPERIENCE WITH RESEARCH AND THAT IS NOT ACTUALLY WHAT WE FOUND.
WE FOUND THEY WERE PRETTY EXPERIENCED.
THEY HAD LIBRARY INSTRUCTION IN DIFFERENT PLACES AND THEY FELT VERY COMFORTABLE AND CONFIDENT WITH DOING RESEARCH, BUT WHAT THEY WANTED AND WHAT THEY WISHED MIGHT HAPPEN WITH THE LIBRARIES WAS THAT THEY WOULD HAVE A CHANCE TO CONNECT WITH OTHER STUDENTS.
THEY FELT SORT OF DISCONNECTED FROM PEOPLE, SERVICES, ETC.
AND SO WHAT WE DID WITH THAT INFORMATION, THE ACTION THAT WE TOOK WAS TO HAVE TRANSFER STUDENT MIXER NEAR THE BEGINNING OF THE SEMESTER WHERE WE INVITED STUDENT SUCCESS SERVICES FROM ACROSS CAMPUS TO COME TO THE LIBRARY.
WE INVITED ALL OF THE TRANSFER STUDENTS TO COME AND JUST MEET THEIR LIBRARIANS, MEET OTHER FOLKS FROM CAMPUS WHO ARE THERE TO HELP WITH VARIOUS THINGS.
AND, YOU KNOW, WE HAD SOME SORT OF ICE BREAKER TYPE OF GAMES.
AND IT WAS REALLY VERY POSITIVE.
THE FEEDBACK WE GOT FROM STUDENTS WAS THAT THAT WAS HELPFUL.
THEY MET PEOPLE.
MANY OF OUR STUDENTS, AND I THINK TRANSFER STUDENTS IN PARTICULAR IF THEY'RE A LITTLE BIT OLDER HAVE COMPLICATED LIVES AND THEY'RE NOT ON CAMPUS ALL THE TIME AND SO JUST CARVING OUT SOME TIME TO MEET OTHER STUDENTS AND CONNECT WAS REALLY VALUABLE.
SO THAT'S ONE EXAMPLE WHERE WE GOT ASSESSMENT RESULTS THAT WE DID NOT EXPECT, BUT WE WERE ABLE TO TRANSLATE THOSE FINDINGS INTO AN ACTION THAT WAS HELPFUL FOR OUR STAKEHOLDERS.
ANOTHER EXAMPLE IS WITH A FIRST YEAR COMPOSITION PROGRAM, WE WORKED CLOSELY WITH FIRST YEAR COMPOSITION AS I'M SURE MANY OF YOU ESPECIALLY IN ACADEMIC LIBRARIES DO AND A COUPLE OF YEARS AGO WE LOOKED AT STUDENT RESEARCH TOPICS.
WE WANTED TO FIND OUT WHAT KINDS OF TOPICS STUDENTS WERE WRITING THEIR VERY FIRST RESEARCH PAPER ABOUT AND WHETHER THOSE RESEARCH TOPICS WERE SORT OF APPROPRIATELY SCOPED AND RESEARCHABLE FOR FIRST YEAR STUDENTS.
SO WE TOOK A LOOK AT 1,400 STUDENT TOPICS THAT WE COLLECTED AND LOOKED AT THE SCOPE, THE DEVELOPMENT AND THE RESEARCHABILITY OF THE TOPIC.
AND WE FOUND THAT FOR THE MOST PART, STUDENTS WEREN'T REALLY GETTING THE IDEA OF SCOPE.
THEY WERE EITHER TRYING TO APPROACH SOMETHING TOO BIG OR APPROACHING SOMETHING TOO LOCAL, TOO, SORT OF NARROWLY FOCUSED AND SO THAT GAVE US SOMETHING THAT WE WANTED TO ACT ON.
WE ALSO FOUND OUT THAT THE MOST POPULAR TOPICS SORT OF BROADLY SPEAKING FOR THE FIRST YEAR STUDENTS WERE EDUCATION, HEALTH AND THE ENVIRONMENT, THOSE THREE SORT OF GENERAL AREAS.
SO THE ACTIONS THAT WE WERE ABLE TO TAKE WERE TO MAKE SOME GUIDES ON THOSE TOPICS, EDUCATION AND HEALTH AND ENVIRONMENT, TO HELP STUDENTS GET STARTED BECAUSE MOST OF OUR GUIDES HAD BEEN FOCUSED ON CLASSES OR SUBJECTS RATHER THAN RESEARCH TOPICS.
SO WE DID THAT, AND THEN WE ALSO DEVELOPED SOME FURTHER LESSON PLANS AND ACTIVITIES TO SHARE WITH THE ENGLISH COMPOSITION INSTRUCTORS AND TO DO IN OUR LIBRARY INSTRUCTION TO LOOK AT -- TO HELP STUDENTS DEFINE THE SCOPE.
>> MELISSA, CAN I INTERRUPT YOU FOR A SECOND?
>> YES, INDEED.
>> WE HAVE A QUESTION AND ACTUALLY I HAVE THE SAME QUESTION.
WENDY ASKS, HOW DID YOU ASSESS RESEARCHABILITY?
AND I ACTUALLY HAVE THAT SAME QUESTION, TOO.
>> YEAH, THAT WAS -- WE WEREN'T SURE IF WE WERE MAKING UP THAT WORD OR IF THAT WAS A REAL WORD.
BUT IT SORT OF INCAPSULATED WHAT WE MEANT IN TERMS OF -- IS IT SOMETHING THAT A FIRST YEAR STUDENT CAN FIND INFORMATION ON WITHOUT TOO MUCH TROUBLE?
LIKE IS INFORMATION READILY ACCESSIBLE TO THEM ON THAT TOPIC?
AND I'M GOING TO POST IN THE CHAT AN ARTICLE THAT WE WROTE ABOUT IT.
>> YOU KNOW, ONE OF THE THINGS IT REMINDS ME OF, TOO, I THOUGHT YOU WERE PAUSING SO IF I INTERRUPT YOU, THIS WILL BE A TEENY WIENIE DELAY.
WHEN I READ SOME RESEARCH THAT WAS DONE BY MICROSOFT WHERE THEY WERE TRYING TO FIGURE OUT HOW GOOD PEOPLE WERE AT ASSESSING THE TOPIC OF THEIR QUESTION IN SEARCH BOXES, WHEN PEOPLE WERE ASKED TO LABEL THEIR QUESTION, MANY OF THEM -- I THINK THEY FOUND SOMETHING LIKE 74% PUT EDUCATION BECAUSE THEY THOUGHT BECAUSE I'M DOING THIS FOR COLLEGE, THE TOPIC IS EDUCATION.
AND I WAS WONDERING THE SAME THOUGHT.
>> THAT'S INTERESTING.
>> WHAT THEY FOUND WERE PEOPLE WERE NOT -- YOU SAID EDUCATION, HEALTH AND ENVIRONMENT.
THEY FOUND THAT THE REGULAR PERSON ON THE STREET, NOT WE LIBRARIANS AND INFORMATION SPECIALISTS, WERE PRETTY POOR AT THIS SO THAT'S WHY IT WAS -- THAT WAS AN INTERESTING KIND OF THING TO ASSESS.
I WOULD LOVE TO SEE YOUR RESULTS.
>> YEAH.
I'VE POSTED THE LINK TO THE ARTICLE AND ALSO OUR DEFINITION OF RESEARCHABILITY HERE, WE DEFINED IT AS THE FINAL TOPIC SELECTION CAN BE CHALLENGED, EXAMINED, ANALYZED BY A NOVICE RESEARCHER WITH A VARIETY OF READILY AVAILABLE RESOURCES IN A FEASIBLE AMOUNT OF TIME BECAUSE THEY'RE FIRST YEAR STUDENTS.
>> THANK YOU, THANK YOU.
>> ABSOLUTELY.
>> WE ARE REALLY INTERESTED THAT THOSE WERE THE TOP THREE TOPICS AS YOU'RE SAYING AND YOU KNOW, THIS BRINGING UP A LOT OF QUESTIONS THAT WE HAVEN'T THOROUGHLY EXPLORED.
AS I SAID, ASSESSMENT WILL USUALLY JUST REVEAL MORE QUESTIONS TO YOU, BUT WE DID WONDER LIKE WHY ARE THOSE THE TOP THREE TOPICS?
AND ONE HYPOTHESIS THAT WE HAVE IS THAT MOST OF OUR STUDENTS ARE 18 AND THEY DON'T HAVE A LOT OF EXPERIENCE WITH, YOU KNOW, POLITICS OR ANY CAREER FIELD NECESSARILY, BUT THEY ALL HAVE SOME EXPERIENCE WITH EDUCATION AS THEY HAVE GONE THROUGH A K-12 EDUCATION.
THEY HAVE HAD EXPERIENCES WITH, YOU KNOW, DOCTORS AND HEALTHCARE, AND THEY HAVE SOME EXPERIENCE IN THE NATURAL ENVIRONMENT.
SO THAT'S SORT OF OUR HYPOTHESIS ABOUT WHY THEY WERE LOOKING AT THAT.
SO WE WERE ABLE TO TAKE SOME ACTION BASED ON THOSE RESULTS THAT WAS KIND OF MORE DIRECT AND WE WERE ABLE TO APPLY THAT TO OUR OWN INSTRUCTION PRACTICES AND TO SHARE IT WITH THE COMPOSITION INSTRUCTORS, AS WELL.
AND I'LL JUST GIVE YOU ONE MORE QUICK EXAMPLE.
I'VE BEEN WORKING WITH OTHER LIBRARIANS FROM THE GREATER WESTERN LIBRARY ALLIANCE ON THE STUDY TO LOOK AT THE IMPACT OF LIBRARY CONSTRUCTION ON STUDENT SUCCESS MEASURED BY GPA, RETENTION, CREDIT HOURS.
YOU'VE SEEN PLENTY OF ARTICLES ON THAT KIND OF TOPIC.
AND OURS INCLUDES 12 UNIVERSITIES LOOKING AT IT TOGETHER.
SO IT'S A LITTLE DIFFERENT.
IT'S NOT JUST ONE INSTITUTION.
MULTIINSTITUTION STUDY HERE.
AND WITH THAT STUDY, WE'RE ABOUT HALFWAY THROUGH OUR SIX-YEAR LONGITUDINAL STUDY.
SO WE'RE NOT DONE YET, BUT IN OUR FIRST YEAR OF ANALYSIS, WE DID FIND THAT THERE WAS A STATISTICALLY SIGNIFICANT CORRELATION BETWEEN STUDENTS HAVING A LIBRARY INSTRUCTION INTERACTION AND BEING RETAINED, FALL TO FALL, HAVING A HIGHER GPA.
AND ALSO EARNING MORE CREDIT HOURS.
SO WITH THAT PARTICULAR ASSESSMENT EFFORT WE HAVEN'T IDENTIFIED CHANGES TO MAKE YET, ALTHOUGH WE THINK WE WILL BY THE END OF OUR SIX YEARS.
BUT WE HAVE BEEN ABLE TO COMMUNICATE A STORY ABOUT OUR IMPACT TO ADMINISTRATORS.
SOME OF THE PARTICIPANTS AT THE DIFFERENT INSTITUTIONS HAVE WORKED ON ACCREDITATION EFFORTS AT THEIR UNIVERSITIES BASED ON THIS CORRELATION STUDY THAT WE'RE DOING.
THEY'VE BECOME MORE INVOLVED IN CAMPUS ASSESSMENT EFFORTS AND IT'S GIVEN US A STORY TO TELL OUR PROVOSTS ABOUT HOW IMPORTANT THE WORK THAT WE DO IN OUR LIBRARIES WITH STUDENTS, YOU KNOW, THAT IT REALLY MAKES A DIFFERENCE AND I'LL SHARE A LINK IN THE CHAT ABOUT THAT, AS WELL.
I'M SHARING THESE EXAMPLES TO POINT OUT THE DIFFERENT TYPES OF STORIES THAT YOU MIGHT TELL AND DIFFERENT ACTIONS YOU MIGHT TAKE BASED ON YOUR FINDINGS.
YOUR AUDIENCE MIGHT NOT BE WHAT WE EXPECTED IT TO BE IN THE BEGINNING.
SO ONCE YOU'VE NOTIFIED -- IDENTIFIED SERVICES AND RESOURCES THAT ARE IMPACTED BY YOUR DATA THINK ABOUT WHO ARE THE STAKEHOLDER FOR THE SERVICES AND THE RESOURCES THAT YOU MIGHT END UP CHANGING?
THEY ALL NEED COMMUNICATION REGARDING THE DATA THAT YOU'VE GATHERED.
THE CHANGES THAT WILL BE MADE BASED ON THAT DATA AND HOW THAT'S GOING TO IMPACT THEM.
SO THIS IS GOING TO INCLUDE INTERNAL AND EXTERNAL STAKEHOLDERS.
AND THEY MAY HAVE PARTICIPATED IN YOUR DATA GATHERING IN WHICH CASE THEY MAY HAVE HAD A HEADS-UP THAT THIS IS HAPPENING.
AND IN THAT SORT OF A CASE IT'S VERY POWERFUL TO LET THEM KNOW WHAT'S HAPPENING WITH THE INFORMATION THAT YOU RECEIVED FROM THEM, LETTING THEM KNOW THEIR VOICES WERE HEARD AND CHANGES ARE BEING MADE BASED ON THE SURVEY OR THE FOCUS GROUP THAT THEY'VE PARTICIPATED IN IS A REALLY NICE WAY TO CLOSE THE LOOP AND LET THEM KNOW HOW IMPORTANT THEIR CONTRIBUTIONS WERE.
BUT THEN ON THE OTHER HAND, SOME STAKEHOLDERS MAY HAVE BEEN DE-CENTERED FROM THE DATA COLLECTION PROCESS.
AND IT'S JUST AS IMPORTANT FOR THEM TO KNOW THAT THE LIBRARY'S ASSESSING ITS SERVICES, CHANGES ARE BEING MADE BASED ON EVIDENCE, AND THE LIBRARY'S ENGAGED IN CONTINUOUS IMPROVEMENT.
SO IT'S IMPORTANT TO IDENTIFY WHO NEEDS TO HEAR ABOUT THE RESULTS OF YOUR ASSESSMENT PROJECT.
THIS IS A TOOL FOR IDENTIFYING STAKEHOLDERS THAT COMES FROM THE ACADEMIC LIBRARY VALUE IMPACT STARTER KIT FROM MEGAN OAKLEAF.
THE BOOK HAS A FEW MORE CATEGORIES OF STAKEHOLDERS IN THIS KIND OF A CHART, BUT I THINK THIS IS A PRETTY GOOD STARTING LIST FOR AN ACADEMIC LIBRARY.
SO AS YOU'RE THINKING ABOUT YOUR ASSESSMENT PROJECT, THE DATA YOU'VE GATHERED, IS IT GOING TO IMPACT ANY OF THESE STAKEHOLDERS?
AND ALSO CONSIDER THERE ARE SUBGROUPS WITHIN THESE GROUPS SO AMONG STUDENTS, YOU HAVE FIRST YEAR STUDENTS, TRANSFER STUDENTS, FIRST-GENERATION STUDENTS, STUDENTS WHO ARE IN DIFFERENT MAJORS AND COLLEGES.
THERE ARE GRADUATES AND UNDERGRADUATES.
SO THEY'RE NOT A MONOLITH.
YOUR ASSESSMENT PROJECT MAY NOT IMPACT ALL STUDENTS.
SO CONSIDER THE DIFFERENT SUBGROUPS THERE.
IT'S THE SAME FOR FACULTY, OF COURSE.
YOU MAY CONSIDER THE NEEDS OF ADJUNCTS, PART-TIME FACULTY, TENURED VERSUS TENURE-TRACK FACULTY SO THINK ABOUT THAT.
AND THEN INTERNAL TO THE LIBRARY, ESPECIALLY IF YOU HAVE A VERY LARGE ORGANIZATION, WHAT DIFFERENT DEPARTMENTS OR GROUPS WILL BE IMPACTED IF YOU MAKE THE CHANGE THAT YOUR DATA SUGGESTS?
IT'S IMPORTANT TO COMMUNICATE THAT CLEARLY WITH THEM.
AND THEN ADMINISTRATION IN THIS CASE MIGHT BE DEANS, ASSOCIATE DEANS, PROVOSTS, ETC.
SO THE IDEA HERE IS TO CIRCLE THE GROUPS THAT ARE MOST IMPORTANT AND MAYBE YOU PRIORITIZE THE TOP 5 THAT YOU REALLY WANT TO BE SURE TO COMMUNICATE WITH, BUT YOU MAY WANT TO COMMUNICATE WITH ALL OF THE STAKEHOLDERS TO MAKE SURE THAT FOLKS KNOW WHAT'S HAPPENING WITH CHANGE TO SERVICES OR RESOURCES.
NOW, HERE'S AN EXAMPLE FOR A PUBLIC LIBRARY.
I'VE PUT THE INTERNAL STAKEHOLDERS IN THE CENTER, THE LIBRARY STAFF AND BOARD AND LIBRARY USERS AND THEN CONCERN STAKEHOLDERS ARE ON EACH -- THE EXTERNAL STAKEHOLDERS ARE ON EACH SIDE.
THE IDEA IS YOU MAY NOT HAVE CONSIDERED ALL THESE GROUPS WHEN YOU BEGAN YOUR ASSESSMENT PROJECT.
YOU WERE PROBABLY THINKING ABOUT STAKEHOLDERS AND USERS BUT HAD YOU CONSIDERED ALL OF THE NECESSARY STAKEHOLDERS?
CONSIDERING WHAT YOU'VE LEARNED FROM THE ASSESSMENT PROJECT, YOU MAY HAVE A NEW AND A REFRESHED MESSAGE FOR ANY AND ALL OF THESE COMMUNITY GROUPS AND THAT CAN BE REALLY POWERFUL FOR YOU AND FOR YOUR LIBRARY.
SO WHAT I'LL DO NOW IS HAVE YOU GO TO CHAT, POST YOUR TOP STAKEHOLDER GROUP AND/OR ANY QUESTIONS THAT YOU HAVE SO FAR ABOUT IDENTIFYING YOUR STAKEHOLDERS.
AND MARIE, DO YOU HAVE ANYTHING TO ADD ABOUT IDENTIFYING THE SERVICES OR RESOURCES OR STAKEHOLDERS THAT ARE IMPORTANT AT THIS POINT?
>> OH, SORRY.
[LAUGHS]
I WAS JUST TRYING TO SEE ALL THE PARTICIPANTS' POSTS.
WE GOT A QUESTION, WE GOT A COUPLE OF QUESTIONS, BUT NOT ABOUT THE STAKEHOLDERS.
SO I THINK I'M GOING TO HOLD ON TO THOSE OTHER QUESTIONS FOR NOW TO SEE IF WE SEE ANY QUESTIONS ABOUT STAKEHOLDERS BUT ONE THING I DID WANT TO MENTION WHILE WE'RE HAVING THIS -- THIS KIND OF Q&A A LITTLE BIT PAUSED IS THAT ONE OF THE THINGS WE LEARN ALONG THE WAY -- SO WHAT YOU WERE TALKING ABOUT, HOW THAT ASSESSMENT IS THIS LEARNING PROCESS, IS THAT WE LEARN WHETHER OR NOT WE HAVE OVERLOOKED ANY STAKEHOLDER GROUPS.
SO I THINK WE FIND OUT ABOUT THIS WHEN LATER ON, SOMEONE MIGHT SAY WE HAD NO IDEA THAT THIS WAS HAPPENING.
THAT HAS HAPPENED TO ME SOMETIMES IN A LARGE ORGANIZATION TO MAKE SURE THAT THE COMMUNICATION GETS OUT.
THAT'S JUST ONE COMMENT THAT I HAVE.
>> GREAT.
>> I DON'T KNOW IF YOU CAME ACROSS THAT WHERE YOU THOUGHT EVERYONE'S IN THE LOOP, BUT SUDDENLY YOU REALIZE THEY'RE NOT ALL IN THE LOOP.
>> UH-HUH.
AND I THINK PARTICULARLY THAT'S COME UP FOR ME WITH ADMINISTRATORS.
WE'LL COME UP WITH SOME REALLY GREAT IDEA ABOUT LIKE A TRANSFER STUDENT EVENT AND IT'S LIKE OH, WELL OTHER PEOPLE MIGHT BE INTERESTED IN THIS.
WHAT ABOUT THE ADVISING CENTER OR WHAT ABOUT, YOU KNOW, DIFFERENT -- DIFFERENT GROUPS SO I THINK IT IS PRETTY IMPORTANT TO CONSIDER.
OKAY SO YOUR NEXT STEP IN THIS PROCESS WILL BE TO IDENTIFY YOUR TOP FIVE OR SO STAKEHOLDER GROUPS AND COME UP WITH A STORY OR A MESSAGE FOR THEM ABOUT THE CHANGE YOU'RE GOING TO MAKE OR ABOUT THE DATA THAT YOU GATHERED.
I'M USING THE 27-9-3 RULE HERE SO BASICALLY, WHEN YOU'RE COMING UP WITH A MESSAGE FOR DIFFERENT STAKEHOLDER GROUPS, KEEP IT REALLY NICE AND FOCUSED.
THIS IS 27 WORDS, NINE SECONDS, THREE POINTS MAXIMUM SO THAT'S A VERY SHORT ELEVATOR PITCH.
THIS COULD BE SOMETHING THAT YOU DELIVER ON A FLIER OR IN AN E-MAIL, IN A PUBLIC SERVICE ANNOUNCEMENT OR JUST WHEN YOU RUN INTO SOMEBODY AT THE CAMPUS COFFEE SHOP OR SOMETHING SO I'M GOING TO GIVE YOU SOME EXAMPLES OF THIS, AND THEN WE'LL COME BACK TO THIS SLIDE SO THAT YOU CAN DRAFT ONE KEY MESSAGE FOR ONE STAKEHOLDER GROUP.
AND I'M SHARING SOME EXAMPLES FROM THE STUDENTS IN MY SYRACUSE HIGH SCHOOL CLASS, PLANNING, MARKETING AND ASSESSMENT FROM THEIR REALLY GREAT WORK.
SO THIS WAS AN ASSESSMENT PROJECT THAT IDENTIFIED A NEED FOR A BOOKMOBILE THAT WAS BASED ON THE RURAL NATURE OF THIS COMMUNITY AND A BIG GROUP OF OUTLYING NON-USERS WHO ARE MAYBE NOT VERY MOBILE AND SO WE IDENTIFIED A NEED FOR A BOOKMOBILE, BUT THEN IF WE'RE GOING TO START THAT OUTREACH SERVICE WE NEED TO COMMUNICATE WITH INTERNAL AND EXTERNAL STAKEHOLDERS ABOUT THIS NEW SERVICE.
SO THE MESSAGE FOR LIBRARY STAFF WHO MAY BE UNSURE ISN'T THIS KIND OF AN OLD-FASHIONED KIND OF THING TO DO, WHY ARE WE DOING A BOOKMOBILE?
THE MESSAGE FOR THEM MIGHT BE SOMETHING LIKE IMPROVING OUTREACH CAN INCREASE THE USAGE OF MATERIALS THAT WE'VE PURCHASED AND WE WANT TO MAKE AVAILABLE TO THE COMMUNITY.
IT SPEAKS TO OUR STRATEGIC PLAN, IT'S IMPORTANT FOR OUR MISSION.
THAT CAN BE THE MESSAGE FOR SORT OF YOUR FRONT-FACING LIBRARY STAFF.
IT'S IMPORTANT FOR PUBLIC RELATIONS IN THE LIBRARY TO KNOW AS WELL SO THAT THEY CAN CREATE A SOCIAL MEDIA CAMPAIGN AND THE SORT OF BRIEF PITCH FOR THEM MIGHT BE CREATING A SOCIAL MEDIA CAMPAIGN CAN HELP WITH INFORMING PEOPLE ABOUT OUR OUTREACH ACTIVITIES, PROGRAMS.
SPEAKING TO OUR STRATEGIC PLAN.
IT CAN BE REALLY IMPORTANT TO MAKE SURE THOSE THINGS ARE IN ALIGNMENT AS YOU ARE MESSAGING THIS INTERNALLY AND EXTERNALLY, BUT MAYBE, ESPECIALLY INTERNALLY.
IT'S IMPORTANT TO LET PATRONS KNOW ABOUT THIS, TO LET THEM KNOW THAT YOU'RE PROVIDING BOOKMOBILE SERVICES, THAT IT WILL PROVIDE MORE RESOURCES AND ACCESSIBILITY FOR THEM AND THEN TO SELL THIS TO A DIRECTOR OR ASSISTANT DIRECTOR, LETTING THEM KNOW THAT THIS WILL INCREASE THE USE OF OUTREACH SERVICES AND LIBRARY RESOURCES IS REALLY IMPORTANT.
SO THESE MESSAGES MIGHT BE DELIVERED FACE TO FACE IN STAFF MEETINGS OR THROUGH A NEWSLETTER, THROUGH A FLIER.
THERE ARE A VARIETY OF WAYS THAT THESE VERY SHORT MESSAGES MIGHT GO OUT TO THE RELEVANT STAKEHOLDERS.
ANOTHER PUBLIC LIBRARY EXAMPLE IS AN ASSESSMENT PROJECT THAT IDENTIFIED ELIMINATING FINES AS A WAY TO BRING USERS BACK TO THE LIBRARY TO LOWER BARRIERS TO USING LIBRARY MATERIALS, ESPECIALLY FOR TEENS AND THERE'S A NEED, OBVIOUSLY, HERE TO COMMUNICATE THE POLICY CHANGE BOTH INTERNALLY AND EXTERNALLY.
SO ONE GROUP OF LIBRARY STAFF WHO NEED TO KNOW ABOUT THIS AND MAKE SURE THAT THEY BUY INTO THIS IS A GROUP OF PROGRAMMERS SO LETTING THEM KNOW THAT THE ELIMINATION OF FINES ALLOWS US TO REMAIN TRUE TO OUR STRATEGIC PLAN AND WILL BRING NON-USERS BACK INTO THE LIBRARY IF THEY'RE NOT AFRAID OF FINES IS HOPEFULLY A POWERFUL MESSAGE FOR THEM.
THE CIRCULATION STAFF MIGHT ALSO HAVE CONCERNS ABOUT ELIMINATING FINES.
THIS CHANGES THEIR WORK PROCESS SO LETTING THEM KNOW THAT PATRONS WILL BE HELD RESPONSIBLE, THEY'LL EXPECTED TO RETURN MATERIALS IN A TIMELY MANNER MIGHT BE HELPFUL IN LETTING CIRCULATION STAFF KNOW THAT THIS IS NOT GOING TO BE A HUGE DISASTER IF WE ELIMINATE FINES.
THERE'S A MESSAGE TO GO OUT EXTERNALLY, OF COURSE, TO LIBRARY USERS AND FORMER LIBRARY USERS TO LET THEM KNOW ABOUT THIS CHANGE.
AND AGAIN, THESE MESSAGES CAN BE COMMUNICATED INTERNALLY THROUGH TRAINING OR STAFF MEETINGS, AND THEN THEY COULD GO OUT EXTERNALLY TO THOSE EXTERNAL STAKEHOLDERS THROUGH A NEWSLETTER, A FLIER, IN LIBRARY BOARDS OR OUT OF LIBRARY SIGNAGE, A LOT OF WAYS THAT THAT MESSAGE COULD BE DELIVERED.
>> YOU KNOW, ONE OF THE STAKEHOLDER GROUPS AS YOU'RE TALKING IT REMINDS ME OF SOME OF THE TERMINOLOGY THAT LIN AND I HAVE STARTED USING IN OUR RESEARCH.
WE WERE ALWAYS TALKING PREVIOUSLY ABOUT USERS AND NON-USERS AND I SEE YOU HAVE FORMER LIBRARY USERS AND WHAT WE DECIDED TO DO WAS TO CHANGE OUR THOUGHTS ABOUT OUR STAKEHOLDER GROUPS TO INCLUDE LIBRARY USERS AND POTENTIAL LIBRARY USERS.
[LAUGHS]
SO THAT NON-USERS, NOT FORMER USERS OR LOST SHEEP OR WHATEVER, BUT TO THINK OF PEOPLE AS POTENTIAL WHICH I THINK THEN THAT'S ANOTHER POWERFUL -- MAYBE STAKEHOLDER GROUP WE DON'T THINK ABOUT.
THIS IS A HOT TOPIC, BY THE WAY, ELIMINATING FINES.
IT'S GETTING A LOT OF TRACTION NOW BECAUSE IT IS ACTUALLY A QUESTION OF EQUITABLE SERVICES AND INCLUSION.
>> YEAH, IT CERTAINLY I THINK SPEAKS TO YOUNGER USERS IN PARTICULAR, LIKE A TEENAGER WHO'S RETURNED SOMETHING LATE, THEY MAY NEVER BE ABLE TO PAY A FINE AND COME BACK TO USING THE LIBRARY SO I THINK IT CAN BE PRETTY POWERFUL.
>> YEAH, ONE OF MY COLLEAGUES WHO WAS AT QUEENS COLLEGE, MARY SHELTON, SHE'S A FAMOUS PERSON IN YOUTH SERVICES, SHE WROTE AN ARTICLE BASED ON HER DISSERTATION CALLED THE OVERDUE KID AND IT WAS EXACTLY ABOUT THAT, SOMEONE FEELING SHAMED AT THE LIBRARY BECAUSE OF AN OVERDUE FINE.
>> GREAT.
OKAY.
SO THOSE ARE MY PUBLIC LIBRARY EXAMPLES.
I HAVE A COUPLE OF ACADEMIC LIBRARY EXAMPLES FOR YOU, AS WELL.
THIS IS FROM AN ASSESSMENT PROJECT THAT IDENTIFIED A PRETTY EASY, PAINLESS WAY TO DRIVE MORE ONLINE STUDENTS TO THE LIBRARY, JUST EMBEDDING A LINK TO LIBRARY RESOURCES IN ALL ONLINE CLASSES.
AND THAT'S SOMETHING THAT, YOU KNOW, STUDENTS MAY NOT EVEN NEED VERY MUCH COMMUNICATION ABOUT.
YOU CAN JUST LET THEM KNOW THAT THE LIBRARY IS GOING TO BE EMBEDDED IN ALL THEIR COURSE PAGES.
IT WILL GET YOU TO THE LIBRARY'S DIGITAL RESOURCES WITH A SINGLE CLICK.
NO NEED TO OPEN NEW TABS, PERFORM INTERNET SEARCHES, AND THAT CAN -- THAT CAN BE PRETTY EASY.
IT MIGHT BE TRICKIER TO COMMUNICATE WITH FACULTY TO TELL THEM THAT YOU'RE BARGING INTO THEIR CLASS.
BUT THINKING ABOUT A MESSAGE FOR THEM LIKE A COLLABORATIVE RELATIONSHIP WITH LIBRARY STAFF WILL IMPROVE THE FOCUS OF THE CURRICULUM, TWO-WAY COMMUNICATION WITH THE LIBRARY WILL GIVE YOU ACCESS TO THE MOST UP-TO-DATE RESOURCES TO DEVELOP YOUR COURSE.
JUST TO LET THEM KNOW WHAT THEY GET, WHAT THEY GAIN FROM HAVING THAT AUTOMATIC LINK IN AN AUTOMATIC COURSE, AND THEN COMMUNICATING THAT INTERNALLY TO LIBRARY STAFF IS IMPORTANT AS WELL ALTHOUGH HOPEFULLY, IT WON'T BE A TOUGH SELL WITH THE LIBRARY STAFF.
SO THIS COULD BE COMMUNICATED, YOU KNOW, AT THE BEGINNING OF THE SEMESTER OR AS QUESTIONS COME UP TO LET FOLKS KNOW WHY WE'RE DOING WHAT WE'RE DOING.
AND FINALLY, ONE MORE ASSESSMENT PROJECT.
THIS IDENTIFIED STUDENT GROUPS WHO ARE NOT USING THE LIBRARY.
FIRST-YEAR STUDENTS, STUDENTS FROM MINORITY GROUPS AND THE LIBRARY MADE A PLAN TO ADD OUTREACH EFFORTS -- EVENTS, ESPECIALLY FOR NON-TRADITIONAL STUDENTS, FIRST-GENERATION STUDENTS, AND DIFFERENT STUDENT ORGANIZATIONS THAT WERE ORGANIZED AROUND VARIOUS DIVERSE IDENTITIES AND THE WHOLE IDEA WAS ABOUT MAKING THE LIBRARY A MORE COMFORTABLE SPACE FOR EVERYONE.
AND SO COMMUNICATING INTERNALLY WAS IMPORTANT TO MAKE SURE THAT LIBRARY STAFF KNOW THE REASON FOR THIS, WHY WE ARE DOING SOME OUTREACH EVENTS.
COMMUNICATING TO STUDENT GROUPS AND FIRST-YEAR STUDENTS THROUGH OUTREACH IS REALLY IMPORTANT, TO LET THEM KNOW WHY THE LIBRARY MIGHT BE IMPORTANT TO THEIR DIFFERENT GROUPS, AND THEN THIS IS ONE THAT MIGHT HAVE SOME POLITICAL IMPACT, AS WELL, IF YOU CAN COMMUNICATE TO THE VICE PROVOST'S OFFICE TO LET THEM KNOW THAT THE LIBRARY IS A PARTNER IN THAT EFFORT.
SO THOSE ARE JUST A FEW EXAMPLES OF THESE VERY SHORT, VERY FOCUSED MESSAGES FOR DIFFERENT STAKEHOLDERS.
SO I PROMISED WE WOULD COME BACK TO THIS.
AND WE ARE JUST GOING TO TAKE A MINUTE TO DRAFT A MESSAGE.
YOU CAN FEEL FREE TO SHARE IT IN THE CHAT BOX BECAUSE IT'S LESS THAN 27 YEARS AND -- 27 WORDS AND YOU CAN ASK ANY QUESTIONS YOU HAVE AT THIS POINT ABOUT KEY MESSAGES.
I'M GOING TO STOP AND TAKE A DRINK FROM MY WATER BOTTLE WHILE YOU MAKE A NOTE FOR YOURSELF OR SHARE IN THE CHAT.
>> I THINK THAT HERE, WHAT JENNIFER POSTED, WWW.POWERPRISM.ORG, 27913, I OPENED THAT UP IN MY BROWSER.
IT GIVES THREE EXAMPLES OF A 27-9-3 MESSAGE THAT ARE VERY, VERY POWERFUL, IF ANYBODY WANTS TO LOOK AT THAT IN DRAFTING YOUR OWN, I THINK IT WOULD BE VERY HELPFUL, VERY CONCRETE.
>> GREAT.
OKAY.
SO AS YOU'RE THINKING ABOUT HOW TO TELL THIS STORY, THE OTHER THING TO THINK ABOUT IS WHERE ARE THE GAPS?
SO, AS I SAID, AT THE BEGINNING, OFTEN AN ASSESSMENT PROJECT WILL REVEAL JUST AS MANY QUESTIONS AS IT DOES ANSWERS.
AND THIS IS SORT OF THE CREATIVE AND CHALLENGING PART OF CLOSING THE LOOP.
WHERE ARE THOSE GAPS AND HOW ARE YOU GOING TO FILL THEM IN?
SO THIS IS A CHANCE FOR YOU TO CIRCLE BACK TO YOUR ORIGINAL RESEARCH OR ASSESSMENT QUESTIONS.
ARE THEY THOROUGHLY ANSWERED?
IF THE ANSWER FOR YOU IS YES, THEN THAT'S GREAT.
YOU CAN USE THE INFORMATION THAT YOU GATHERED TO IMPROVE OR TO CREATE NEW SERVICES.
BUT IF YOUR QUESTIONS ARE NOT THOROUGHLY ANSWERED, HOW CAN YOU GET MORE INFORMATION THROUGH A DIFFERENT METHOD OR THROUGH A DIFFERENT SOURCE?
SO IF YOU DID A SURVEY TO BEGIN WITH AND YOU DON'T HAVE ALL THE INFORMATION YOU NEED, ARE YOU GOING TO GO BACK TO THAT SAME POPULATION AND TRY TO DO SOME FOLLOW-UP INTERVIEWS OR FOCUS GROUPS TO GET MORE INFORMATION ON THE TOPIC?
THAT WOULD BE A DIFFERENT METHOD OR ARE YOU GOING TO TREE GOING TO A DIFFERENT SOURCE.
IF YOU SURVEYED STUDENTS TO BEGIN WITH, FOR EXAMPLE, YOU MIGHT TRY SURVEYING OR INTERVIEWING FACULTY MEMBERS ON -- ON THE SAME QUESTION, BUT JUST TO GET AT IT FROM A DIFFERENT PERSPECTIVE.
SO TO GO BACK TO MY STUDENT RESEARCH TOPIC EXAMPLE, LOOKING AT STUDENT RESEARCH TOPICS RAISED A WHOLE NEW SET OF QUESTIONS FOR US, LIKE WHY ARE HEALTH AND EDUCATION AND THE ENVIRONMENT SUCH POPULAR TOPICS?
THAT WAS ONE OF OUR QUESTIONS.
ALSO, ANOTHER QUESTION FOR US WAS OKAY, SO IF STUDENTS DO IDENTIFY A WELL-SCOPED RESEARCH QUESTION, THEN DOES THAT MEAN THEY HAVE A BETTER RESEARCH PAPER IN THE END OR NOT?
SO WE DID ACTUALLY END UP DOING SOME FOLLOW-UP TO LOOK AT STUDENT RESEARCH PAPERS, BOTH ANNOTATED BIBLIOGRAPHIES AND THEN FINAL RESEARCH PAPERS TO SEE -- AGAIN TO LOOK AT TOPICS, BUT ALSO TO SEE HOW STUDENTS FOUND INFORMATION AND USED IT AND TO SEE IF THERE WERE INTERESTING, YOU KNOW, CORRELATIONS BETWEEN WHETHER THEY HAD A GOOD TOPIC OR WHETHER THEY HAD A GOOD PAPER.
AND THAT'S BEEN INTERESTING FOR US TO PURSUE ALONG WITH THE COMPOSITION PROGRAM TO SEE HOW WE CAN IMPROVE STUDENT RESEARCH FROM THE VERY BEGINNING WHEN THEY CHOOSE THEIR TOPICS.
>> MELISSA, SINCE YOU'RE REFERRING BACK TO THAT, WE ACTUALLY HAVE THREE QUESTIONS THAT WERE POSTED ABOUT THAT.
THE FIRST ONE WAS HOW DID YOU GET THE STUDENT PAPERS?
>> HOW WE GOT THE STUDENT PAPERS WAS WORKING WITH THE COMPOSITION PROGRAM.
THEY AGREED TO WORK WITH US ON THIS FROM THE BEGINNING AND BASICALLY, THEY SAMPLED THE PAPERS FOR US.
THEY GIVE US THE FIFTH, TENTH, AND 15th PAPER FROM EACH SECTION OF THE CLASS AND THEY DEIDENTIFY IT FOR US SO WHAT WE GET IS -- IS RANDOM AND DEIDENTIFIED.
>> SO I GUESS THAT ANSWERS THE SECOND QUESTION WAS ABOUT PRIVACY ISSUES.
SO IF EVERYTHING -- IS EVERYTHING INDEED TOTALLY ANONYMOUS OR IS IT JUST CONFIDENTIAL?
>> IT'S ANONYMOUS FOR THE LIBRARIANS WHO ARE DOING THE SCORING, YEAH.
SO THE INSTRUCTORS OF THE CLASS HAVE ALREADY GRADED THE PAPER.
THEY HAVE DONE WHAT THEY NEEDED TO DO, TO ASSIGN STUDENT GRADES, AND THEN THEY ANONMIZE IT AND SEND IT TO THE LIBRARIES AND WE DON'T HAVE A CONNECTION TO WHICH COURSE OR WHICH SECTION IT CAME FROM.
WE DON'T KNOW THE INSTRUCTOR OR THE STUDENT NAME.
>> SO I HAVE A QUESTION.
WERE YOU DROWNING IN DATA THEN?
DID YOU HAVE A LOT OF DATA TO ANALYZE?
>> YEAH.
QUITE A LOT!
AND LUCKILY, WE HAD A GRADUATE ASSISTANT WHO HELPED US WITH THE -- WITH THE RESEARCH TOPIC PAPER.
YOU'LL SEE IF YOU LOOK AT THAT THAT I POSTED FROM COLLEGE AND RESEARCH LIBRARIES WE HAD 1,400 STUDENT TOPICS.
>> WOW!
WOW!
WOW.
YEAH, I THINK THIS GOES BACK TO ONE OF THE THINGS ABOUT DESCRIBING DATA.
WHEN WE HAVE A LOT OF QUALITATIVE DATA, ONE OF OUR HUGE CHALLENGES IS, YOU KNOW -- THE TECHNICAL TERM IS DATA REDUCTION, BUT, YOU KNOW, IF YOU HAVE 1,400 -- THAT'S A LOT OF STUDENT PAPERS RIGHT, AND THEN HOW DO YOU GO ABOUT REDUCING THAT TO THE POINT WHERE YOU CAN STILL FOR YOUR STAKEHOLDERS MAKE SENSE OF IT, RIGHT?
SO YOU CAN'T REDUCE IT TOO SMALL, OTHERWISE YOU KIND OF LOSE A LOT OF THE TEXTUAL AND CONTEXTUAL INFORMATION THAT'S PRESENT IN QUALITATIVE DATA AS OPPOSED TO JUST NUMBERS YOU KNOW WHAT I MEAN?
TO REDUCE IT -- SO HOW DID YOU GO ABOUT DESCRIBING THOSE RESULTS THAT WAS UNDERSTANDABLE TO YOUR STAKEHOLDERS AND YET WAS KIND OF TRUE TO YOUR DATA AND ALL THIS ANALYSIS THAT YOU DID?
WAS THAT A CHALLENGE?
>> WHEN WE WERE LOOKING AT THE RESEARCH TOPICS THEMSELVES, NOT AT THE WHOLE PAPER, WE CREATED A RUBRIC THAT JUST DESCRIBED THE -- THE TOPICS.
SO WE WERE LOOKING AT RESEARCHABILITY AND SCOPE.
AND WE SHARED THAT RUBRIC ARE THE ENGLISH DEPARTMENT AND THEY WORKED WITH US ON DEVELOPING IT.
AND SO WE HAD A SHARED UNDERSTANDING OF WHAT WE MEANT BY RESEARCHABILITY AND WHAT THEY MEANT.
THAT WAS IMPORTANT TO SORT OF SHARE OUR DEFINITIONS AND MAKE SURE WE WERE TALKING ABOUT THE SAME THING.
>> AND ONE OF THE OTHER THINGS THAT I WAS ALSO THINKING ABOUT IS KIND OF MEMBER CHECKS.
THAT IS AFTER YOU COLLECT A LOT OF QUALITATIVE DATA AND DO YOUR ANALYSIS TO INVOLVE PERHAPS A GROUP OF STUDENTS IN A FOCUS GROUP, THIS IS SOMETHING THAT CAN BE DONE AS A FOLLOW-UP TO INVOLVE -- TO INVOLVE SOME OF THE PARTICIPANTS THEMSELVES, IF THAT'S POSSIBLE, SINCE YOU DON'T KNOW WHO THEY ARE, RIGHT?
MAYBE WORKING THROUGH YOUR CENTER.
THE ADMINISTRATORS THAT YOU WERE WORKING WITH TO HELP YOU IDENTIFY, TO SEE IF THEY COULD PUT TOGETHER A FOCUS GROUP WHERE YOU COULD PRESENT THE RESULTS TO THE STUDENTS O -- AND GET SOME FEEDBACK FROM THEM.
IT'S ANOTHER WAY TO ENGAGE STAKEHOLDERS AND TO KIND OF COMMUNICATE ADDITIONAL VALUE, TO ADD ADDITIONAL VALUE TO YOUR FINDINGS.
>> YEAH, THAT'S A REALLY GOOD POINT AND WE HAVEN'T DONE THAT IN A REALLY COMPREHENSIVE WAY, BUT SEVERAL OF US HAVE DEFINITELY USED THOSE RESULTS IN INSTRUCTION SESSIONS WITH COMPOSITION STUDENTS TO SAY LIKE YOU KNOW WHAT WE LEARNED FROM LOOKING AT A WHOLE BUNCH OF STUDENT WORK?
WE LEARNED THAT THIS IS CHALLENGING.
WE LEARNED THAT IT'S NOT EASY FOR FIRST YEAR STUDENTS TO SELECT A RESEARCH TOPIC THAT IS APPROPRIATELY SCOPED SO YOU KNOW, BUILDING ON THE EXPERIENCE OF YOUR PEERS, HERE ARE SOME TIPS THAT WE HAVE FOR YOU, WHICH IS MAYBE MORE BELIEVABLE TO HAVE THAT KIND OF STUDENT DATA FOR THEM THAN JUST TO SAY LISTEN TO THE LIBRARIAN, I WILL TELL YOU WHAT YOU NEED TO KNOW, YOU KNOW.
>> ABSOLUTELY.
I WOULD SAY THIS EXTENDS TO GRADUATE STUDENTS IN MY EXPERIENCE, NOT JUST THE UNDERGRADS, THAT SOMETIMES JUST AS WE IN ASSESSMENT MIGHT BITE OFF MORE THAN WE CAN CHEW, SOMETIMES STUDENTS DO AS WELL IN THEIR PROJECTS, KIND OF NAIVELY, MAYBE MORE NAIVELY THAN WE ARE ABOUT THINKING WELL, I COULD JUST RESEARCH EVERYTHING ABOUT WORLD WAR II, RIGHT?
THIS IS A REASONABLE TOPIC.
>> RIGHT.
OKAY.
THIS IS -- THIS IS BASICALLY JUST A BIG QUESTION FOR YOU ALL IN TERMS OF WHERE ARE THE GAPS?
AND SOMETHING FOR YOU TO IDENTIFY AS YOU CIRCLE BACK, AS YOU THINK ABOUT CLOSING THIS LOOP AND THEN STARTING A WHOLE NEW ASSESSMENT LOOP, THAT WILL -- THAT WILL REALLY ENRICH YOUR PROJECT.
SO THAT SORT OF TAKES ME THROUGH MY BIG QUESTIONS FOR YOU ALL AND I'LL JUST OPEN IT UP NOW TO THE QUESTIONS MARIE IS GOING TO SORT OF SHARE THOSE WITH ME, BUT I DID WANT TO SAY, TOO, IF YOU HAVE QUESTIONS OR COMMENTS YOU CAN FEEL FREE TO TWEET ME OR E-MAIL ME AND YEAH, WE'LL OPEN IT UP TO THE QUESTIONS AND DISCUSSIONS.
>> SO THIS ONE CAME ACTUALLY TWICE, AND BEFORE I MISSED IT.
IS THERE SUCH A THING AS STAKEHOLDER CREEP?
DOES IT MAKE SENSE TO PHASE YOUR MESSAGE OUT AT DIFFERENT TIMES THAN ADJUST IT BEFORE RELEASING TO THE NEXT GROUP OF STAKEHOLDER?
>> I THINK THAT'S A REALLY GOOD QUESTION, AND I THINK IT IS IMPORTANT TO PRIORITIZE YOUR STAKEHOLDERS.
YOU MAY WANT TO MAKE A REALLY COMPREHENSIVE LIST OF LIKE EVERYBODY WHO'S INVOLVED OR TOUCHED BY THIS CHANGE, BUT THEN TAKE YOUR TOP THREE OR TOP FIVE AT THE MOST AND COMMUNICATE WITH THEM FIRST.
AND THEN MAYBE BUILD OUT FROM THERE.
>> AND I WONDER ALSO, IF -- IF YOU DID HAVE A PHASED APPROACH TO TALKING TO STAKEHOLDERS, HOW MUCH EFFORT -- TO ME, THINKING ABOUT IT FROM THE POINT OF VIEW OF -- I'VE NEVER WORKED IN A PUBLIC LIBRARY, BUT I WORKED IN SCHOOL LIBRARIES AND I WORKED IN AN ACADEMIC LIBRARY, THINKING ABOUT THE EFFORT THAT GOES INTO ASSESSMENT, THE EFFORT THAT GOES INTO THE PROJECT, TO THE ANALYSIS, TO THE WRITING AND SO FORTH AND THEN THE FINAL ROLL-OUT, SOMETIMES, THERE IS KIND OF A LITTLE FATIGUE AT THE END, TO THINK ABOUT A PHASED -- SO TO THINK ABOUT EFFICIENTLY ROLLING OUT YOUR RESULTS IN TERMS OF A PHASED ROLL-OUT.
I WONDER HOW THAT'S ACCOMPLISHED, IF YOU HAVE ANY TIPS ABOUT THAT, MAYBE MELISSA, THAT WOULD DO THIS, THAT IS LESS OF A ONEROUS TASK.
DO YOU KNOW WHAT I'M ASKING?
YOU GET WHAT I'M ASKING?
>> I THINK SO.
I THINK THAT IF YOU THINK ABOUT IT IN TERMS OF CONCENTRIC CIRCLES, YOU KNOW, LIKE WHO'S IN THE IN CIRCLE?
WHO HAS TO KNOW ABOUT THIS OR ELSE YOU KNOW, YOUR PROJECT IS GOING TO FAIL BASICALLY?
AND START WITH THAT CONCENTRIC CIRCLE AND MOVE OUT FROM THERE TO PEOPLE WHO ARE MAYBE IMPACTED, BUT IT'S NOT GOING TO SORT OF CHANGE THEIR DAILY USE OF THE LIBRARY.
>> YES, YES.
SO JENNIFER PETERSON SAYS I WAS THINKING THAT SOME OF THE EXAMPLES MELISSA SHARED COULD BE TRYING TO REACH MORE STAKEHOLDERS THAN NEEDED AT THE TIME OR WITH CERTAIN MESSAGES.
DOES THAT MAKE SENSE ALONG THE LINES OF SCOPE CREEP?
I THINK WHAT JENNIFER IS SAYING IS IS IT -- IS IT POSSIBLE TO HAVE TOO MANY -- HAVE THAT SCOPE -- SO YOU'RE -- I LIKE YOUR IDEA OF THE CONCENTRIC CIRCLES AND FIGURING OUT WHO FIRST AND AS YOU MOVE OUTWARD RIGHT, HOW THAT MESSAGE MIGHT -- HOW YOU MIGHT WANT TO CONTROL THAT MESSAGE OR CHANGE THAT MESSAGE AND ALSO SOMEONE ELSE COMMENTED THAT THE -- I IMAGINE THAT THE MODE IN WHICH YOU SHARE THE INFORMATION MIGHT VARY DEPENDING ON STAKEHOLDER AUDIENCE?
>> CERTAINLY, YEAH.
SOME OF THE INTERNAL MESSAGING MIGHT BE FACE TO FACE IN TRAINING WITH LIBRARY STAFF AND EXTERNALLY IT CAN GO OUT IN VARIOUS FORMS.
E-MAILS, MARKETING MESSAGES, ALL KINDS OF THINGS.
>> OKAY.
GREAT.
ARLENE, SHE CREATED HER STATEMENT FOR US RELATED TO FINES.
>> OH, AWESOME.
>> SHE SAID, SINCE WE HAVE MORE EFFECTIVE REVENUE SOURCES, WE ARE ELIMINATING LIBRARY FINES SO ALL MEMBERS OF THE COMMUNITY CAN HAVE EQUAL ACCESS TO LIBRARY MATERIALS.
>> I LOVE IT.
THAT IS A GREAT, SHORT MESSAGE.
>> SHE GOT IT AT 25!
I JUST COUNTED!
[LAUGHS]
I WAS ACTUALLY THINKING THAT 27 IS REALLY TOUGH!
>> IT'S VERY SHORT.
>> YOU HAVE TO GET WHAT'S THAT KEY TAKEAWAY?
AND ALSO WRITTEN IN COMPELLING -- THAT'S A VERY GOOD ONE, ARLENE, BECAUSE IT'S VERY COMPELLING.
ARE THERE ANY OTHER COMMENTS OR QUESTIONS THAT YOU HAVE FOR MELISSA?
BECAUSE, YOU KNOW, ONE OF THE THINGS -- I'LL WAIT TO SEE IF PEOPLE DO TYPE IN, BUT ONE OF THE THINGS THAT I THINK WAS EXTREMELY IMPORTANT FROM -- ACTUALLY, MELISSA YOU TALKED ABOUT THIS IN THE BEGINNING AND THEN THROUGHOUT.
WHAT DID YOU LEARN?
IN THE PROCESS OF ASSESSMENT, SO IF WE THINK ABOUT IT REALLY QUITE -- KIND OF CEREBRALLY, WE'RE TRYING TO LEARN SOMETHING FROM OUR USERS, FROM OUR STAKEHOLDER COMMUNITY, FROM OUR ASSESSMENT ACTIVITIES, BUT IN THE END, I THINK IF WE -- IF WE VIEW THE WHOLE PROCESS AS A LEARNING PROCESS, THAT NOT ONLY ARE WE LEARNING THE ANSWERS TO OUR RESEARCH QUESTIONS OR THE QUESTIONS THAT WE HAD, WE'RE ALSO LEARNING MORE ABOUT THE PROCESS.
WE'RE LEARNING MORE ABOUT OUR STAKEHOLDER GROUPS.
WE'RE LEARNING MORE ABOUT ACTUALLY HOW, FOR EXAMPLE, HOW STUDENTS MIGHT SELECT A RESEARCH TOPIC AND YOU THINK THIS WOULD BE WELL RESEARCHED AND IT'S NOT THAT WELL RESEARCHED.
HOW STUDENTS REALLY DO DECIDE WHAT THEIR GOING TO RESEARCH ABOUT AND A LOT OF IT IS ACTUALLY MORE RANDOM THAN WE THINK, IS WHAT I GATHERED FROM THE RESEARCH THAT I READ, BUT IF WE LOOK AT THIS, I THINK HERE'S THE BIG TAKEAWAY, MELISSA.
SEE WHAT YOU THINK.
IF WE THINK ABOUT THIS AS A LEARNING PROCESS FOR OURSELVES, IT ALLOWS US TO MAKE MISTAKES AND NOT FEEL TERRIBLE ALONG THE WAY IF WE HAVE MADE MISTAKES IN OUR ASSESSMENT PROCESS, THAT THEN, AND THEN IF WE LOOK AT IT AS A DYNAMIC PROCESS, THE NEXT TIME, WE'RE GOING TO CORRECT FOR THAT MISTAKE.
RIGHT?
>> YEAH.
I THINK THAT'S REALLY IMPORTANT, AND I THINK JUST, YOU KNOW, KEEPING AN OPEN MIND AND KEEPING A SPIRIT OF INQUIRY ABOUT THE PROCESS, YEAH JENNIFER SAYS ASSESSMENT IS LEARNING.
THAT'S WHAT YOU SAID.
JUST KNOWING THAT YOUR ASSESSMENT PROJECT IS GOING TO REVEAL MORE QUESTIONS FOR YOU AND THAT'S EXCITING, THAT'S NOT A BAD THING BECAUSE, YOU KNOW, YOU'RE NOT GOING TO ANSWER ALL YOUR QUESTIONS IN ONE PROJECT.
YOU'RE JUST GOING TO KEEP ON COMING UP WITH MORE AND MORE.
AND SO YOU KNOW, ONE OF THE AWESOME THINGS ABOUT LIBRARIANSHIP IS IT'S SUCH A LEARNING PROFESSION.
AND I THINK THAT APPROACHING ASSESSMENT WITH THAT KIND OF A MINDSET IS REALLY PRODUCTIVE.
>> YEAH, I AGREE.
AND ONE OF THE THINGS -- WHENEVER I HAVE DONE ASSESSMENT AND AT THE END, ACTUALLY, SOME OF MY ARTICLES THAT I'VE WRITTEN AND GOT PUBLISHED ON ASSESSMENT EFFORTS, I CONCENTRATE MOSTLY ON REFERENCE SERVICES AND USER SERVICES IS AT THE END, YOU ARE LEFT WITH MORE QUESTIONS THAN YOU STARTED WITH, BUT TO ACTUALLY CAPTURE THOSE AND EVEN IN REPORTS TO STAKEHOLDERS, AND IN TALKING ABOUT YOUR ASSESSMENT PROJECT, YOU KNOW, RATHER THAN SAYING WE HAVE ACTUALLY -- LIN SAID THIS TO ME WHEN I WAS TALKING TO HER THE OTHER DAY.
THIS IDEA THAT WE'RE NOT GOING TO -- WE'RE -- YOU KNOW, IT'S NOT -- EVERYTHING YOU EVER WANTED TO LEARN ABOUT X.
[LAUGHS]
IT'S MORE LIKE A PIECE OF THAT AND THEN EVEN COMMUNICATING THAT, LIKE WE STILL HAVE THESE OTHER QUESTIONS.
OR THAT THE ASSESSMENT REVEALED THESE OTHER KEEP QUESTIONS THAT MAYBE WE DIDN'T KNOW THAT THAT WAS PART OF THE PROCESS THAT WE WERE GOING TO UNCOVER THIS.
I THINK THAT'S VERY POWERFUL.
>> I TOTALLY AGREE.
>> AND I'LL POINT OUT THAT BONNIE SHARED A STATEMENT, AS WELL.
SHE HAS A STATEMENT THAT SHE SAID SHE WOULD CONSIDER HER STAKEHOLDERS AS LIBRARY ADMINISTRATORS, COUNTY OFFICIALS WITH PURSESTRINGS AND THE LOCAL COMMUNITY AND PATRONS.
THE STATEMENT IS PUBLIC LIBRARIES NEED TO MOVE INTO THE DIGITAL WORLD WITH THEIR MARKETING EFFORT FOR THEIR LOCAL COMMUNITIES, STAKEHOLDERS UTILIZING COMMUNITY ENGAGEMENT AND TARGET MARKETING ON VARIOUS PLATFORMS.
>> OKAY.
I THINK IT COULD BE AS ARLENE IS POINTING OUT, I THINK IT COULD BE A LITTLE MORE SPECIFIC TO SAY PUBLIC LIBRARIES NEED TO ENGAGE WITH THE COMMUNITY ON SOCIAL MEDIA OR I DON'T KNOW IF THAT'S WHAT YOU MEAN THERE, BUT IT COULD BE A LITTLE SIMPLER AND MORE SPECIFIC, BUT THAT'S A GOOD START.
>> YEAH, I THINK THAT IDEA -- ALTHOUGH AGAIN, IT DEPENDS.
IF SHE IS TALKING TO LIBRARY ADMINISTRATORS, THAT MIGHT WORK, FOR THE COUNTY OFFICIALS AND THE LOCAL COMMUNITY, YOU MIGHT WANT TO REWORD A LITTLE BIT.
YOU MIGHT WANT TO TWEAK IT JUST A LITTLE BIT TO MAKE IT MORE SPECIFIC, BUT I THINK THAT AS IT'S WRITTEN IT WOULD BE FINE FOR LIBRARY ADMINISTRATORS, ABSOLUTELY.
27 WORDS, BONNIE SAYS!
OKAY.
WONDERFUL.
I SEE THAT WE'RE KIND OF ENDING THE TIME THAT WE HAVE TOGETHER.
I WANT TO THANK MELISSA FOR YOUR, YOU KNOW, WONDERFUL DISCUSSION AND VERY PROVOCATIVE AND I ESPECIALLY ENJOYED HEARING ABOUT ALL YOUR EXAMPLES, ESPECIALLY THE EXAMPLES FROM YOUR STUDENTS, THOSE ARE REALLY VERY, VERY ILLUSTRATIVE OF SOME OF THE CONCEPTS THAT YOU WERE TALKING ABOUT AND I WANT TO THANK JENNIFER, EVERYONE AT OCLC FOR YOUR HELP WITH BRINGING THIS SERIES THAT NOW IS CLOSING, BUT I THINK THE DISCUSSION IS JUST STARTING.
>> YES, I THINK YOU'RE RIGHT, ACTUALLY!
THAT'S A VERY GOOD POINT.
YES, MANY OF YOU ARE MAYBE NOW READY TO DIVE IN DEEPER SO WE LOOK FORWARD TO PERHAPS HEARING BACK FROM YOU HOW YOU'VE APPLIED YOUR LEARNING IN THIS SERIES AND THANK YOU TO EVERYONE WHO HAS PARTICIPATED IN THIS SERIES AND A SPECIAL EXTRA THANK YOU TO LIN FOR PULLING OUR PRESENTERS TOGETHER AND A REMINDER THAT I WILL SEND YOU AN E-MAIL LATER TODAY ONCE THE RECORDING IS AVAILABLE AND I'LL ALSO SEND YOU ALL A CERTIFICATE OF ATTENDANCE WITHIN THE WEEK AND WE ARE EXCITED TO BE WRAPPING THIS UP AND AGAIN, THANK YOU TO ALL OF YOU, THANK YOU, MELISSA AND THANK YOU, MARIE AND THANK YOU TO OUR CAPTIONER.
EVERYONE HAVE A FANTASTIC REST OF YOUR WEEK.
THANKS.
>> BYE EVERYONE.
>> BYE THANKS.