The Role of Institutional Research in External Reporting and Responding to Internal Requests

This document attempts to outline the areas and extent of Institutional Effectiveness, Research & Assessment's involvement in responding to internal and external requests for information about Swarthmore College. The "General Principles" presented here are followed by very important "Exceptions," and examples are provided to help clarify. It is our hope that these guidelines will be a starting point for discussions about the best way of accomplishing the goal of providing appropriate and meaningful information that is accurate to those who need it.

While reporting is often done through the Institutional Effectiveness, Research & Assessment Office, it should be remembered that the office is first and foremost a research office. The office seldom "owns" the data that it uses. While all data on campus is theoretically open to IR, the IR officer must take special care to use it correctly and wisely. There will necessarily be frequent consultations with the data stewards. Therefore, the IR officer must depend on the expertise, guidance, cooperation, and good will of the stewards of the data in each area.

External Requests

Except for requests from the news media, which should go through the Communications Office, all external requests for official or public data should go through IR, with specific exceptions noted below. Whether the IR office completes the request or coordinates the request with another office will be based on their expertise in the topic area, access to relevant Swarthmore data, and expertise in working with the particular Swarthmore data (discussed below).

What is data? In this context "data" refers to numerical summaries of aspects of our institution, for example, number of students enrolled, graduation rates, or percentage of faculty who are women. Examples of information that would generally not be considered data are directory information about staff or faculty, or narrative descriptions of programs, policies, or facilities.

What are official or public data? Figures are considered official or public if they are mandated by state or federal guidelines (e.g. the Pennsylvania Department of Education, or "PDE"; the national Integrated Postsecondary Education Data System, or "IPEDS"); are collected routinely by recognized higher education agencies or consortia (e.g. AAUP); or will appear in publications that are widely available (e.g. admissions guidebooks).

Exceptions: With the exception of PDE and IPEDS reporting, mandated routine reporting specific to an individual area should be conducted by that area. (The responsible official in that area should, however, consult with IR whenever figures outside of that area (e.g. enrollments) are required for a form. Examples of reporting that should be conducted not by IR, but by the appropriate office include: athletics reporting, crime reporting, tax forms, human resource work utilization reporting, etc. In general, straightforward office-to-agency reports should go through the appropriate office. If the information requested on a form may be of general interest or may be made public, it would be helpful if a copy were provided to IR. When information may be used in various other ways, the IR office should be involved.

Who gathers the data? Whether a particular external request should be handled by IR versus sent to another office for completion will be based on the IR office's knowledge of the area, access to the data, and knowledge of the data. For example, currently most state and federal reporting are coordinated through IR. This does not mean that the IR officer completes every form herself. A number of forms are sent to other officials on campus for their response and returned to IR for forwarding to the requesting agency. On these forms, it is generally the official who completes the form who signs off on it. Examples of this are the IPEDS-Finance Form and the IPEDS Library Form. The IR officer does complete forms for other kinds of data, including student enrollments, degrees, degree programs offered, graduation rates, and student residency.

Area Knowledge: This guideline will depend on the staff in IR and the role of institutional research at the College. An institutional researcher is bound by a professional code of ethics to not accept assignments outside of his or her realm of expertise. At the same time she should seek training in areas deemed by the institution to be important to the Institutional Research role. In areas where the IR officer has some knowledge or expertise, it may be reasonable for him or her to fulfill a request. In areas where the IR officer does not have expertise, it should be sent to the appropriate expert on campus.

Access to Data: In the situations where the IR officer has little or no access to the particular type of data, she should not attempt to complete a request, but rather forward it to the appropriate office. One example of this is data about the Library. Currently, these data are not centrally maintained and are not accessible to the IR office. Requests for such information will be sent to the Librarian or her designee. Access involves not only being able to retrieve data, but also access to the data coding and field definitions. Though the IR office theoretically has access to all data in the Banner system, she does not necessarily have a complete "dictionary" of what all the data fields mean, how they are populated, and how they relate to each other. Where this kind of mapping has not been made available, data, though technically accessible, cannot be used.

Knowledge of the Data: It makes sense for the IR office to be the one reporting data that is commonly used by the IR office, because in order to use these data effectively the IR officer should be very knowledgeable about it. This is more than knowledge or expertise in a particular area - it is about knowing the particular data on this campus very well, both in form and in content. The IR officer is therefore in a position to accurately provide meaningful information about these areas to outsiders.

Non-Routine External Requests for Study Participation

Increasingly we are asked to participate in research on higher education by everyone from dissertation students to higher education agencies to world- renowned scholars. These requests may range from completing a simple one page form to providing very complex historical data to providing data files at the individual level. Swarthmore must balance the value of the research effort to the public, to the higher education community, and to itself against the burden it presents on College resources (especially staff time) in responding. (This discussion is not meant to include the more informal peer-to-peer type of requests, e.g. queries posted on electronic discussion lists).

Because of the expertise of the IR office in conducting higher education research, any request for such a study should go to the Chief IR Officer for a response strategy. In cases where the institutional burden is minimal, the IR officer may make the decision of whether or not to participate. In cases where the institutional burden is higher, she will notify President's Staff1, telling them:

Who is requesting the study
Nature of the data requested
The purpose and intended use of the information
What offices will need to be involved and to what extent
What peer institutions are known to be participating
Deadline(s) for responding
Benefits (if any) of participation
Her recommendation regarding Swarthmore's participation

President's Staff should then inform the IR officer of their judgment, or whether they need to discuss it further. If they decide that Swarthmore should participate in the study, the IR officer will notify all affected offices immediately, providing them with all the relevant materials. If the IR officer will be directly involved in providing data, she may serve as the study liaison, coordinating the responses of other involved offices. If the IR officer is not involved in providing data, a judgment will be made whether the response should go through her or whether it would be more appropriate for another officer on campus to serve as the liaison, using the same guidelines as described above under "exceptions."

Internal Requests

These issues lead to the question of who should respond when an internal requestor needs information. For example, when the IR officer and the Registrar have equal access, expertise, and facility to respond to a particular data request who should do it?

General Principle: IR should conduct research and analysis in support of planning and decision-making critical to the College. Providing raw data for others to use, and providing simple summaries is not research, and should be done by the appropriate office1. For example, a request for simple trends over time in enrollments in science courses can be done by the Registrar's office. Examining those trends to see if differences by gender are statistically significant, and comparing it with national data would normally be done by IR.

Exceptions

Official data: IR should usually assemble data that should be based on "official" figures. (Example: Degrees awarded in the sciences for use in a grant proposal.)

Information across areas: IR should conduct most analyses that involve combining data across areas. (Example: Relating retention and graduation rates to SAT scores and financial aid received.)

Public Data: IR should usually provide data that will appear publicly and therefore ought to be consistent with "official" figures. (Example: Fact Sheet, total enrollments that appear in Admissions publications.)

Conclusions

These guidelines are meant to clarify the role of Institutional Research in reporting College information. These guidelines may continue to evolve over time. However, they should serve as a good starting point. We are fortunate to work in an institution that highly values collegiality. Guidelines such as this are not meant to replace or even reduce the valuable discussion that takes place here, but to help us all to make sure that we are handling requests for information as effectively as possible.

Examples of Information Requests and Recommended Action

Request: An editor at the Philadelphia Inquirer calls to ask for our current enrollments.
Recommended Action: The call should be directed to the Communications Office. They should refer to Fact Book or contact IR for "official" enrollment figures.
Reason: This is an external request for data from the news media, so the Communications Office should handle it. Since the figure will be made public, it should be based on our official data, which IR is responsible for providing.

Request: A department chair would like to know about the enrollments in a particular introductory course in her department over the past decade, in order to plan for the next few years.
Recommended Action: The Registrar can provide this.
Reason: This is an internal request for simple summary data (no analysis or tying to other institutional data). There is no obvious need for it to be based on public/official data, and it will likely not be used for other purposes.

Request: The US Department of Education requests the reporting of crime statistics.
Recommended Action: The survey should go to the Director of Public Safety, who should confer with IR for any enrollment, staff counts, etc. that may be requested. A copy of the completed survey should be sent to IR.
Reason: This is an office-to-agency report that requests data for which the IR office has no access or expertise. A copy should go to IR since these data are publicly available and it's easy to conceive of it being used in ways other than intended by the collecting agency. (For example, a college ranking publication may decide to add crime rate to the factors considered.)

Request: Someone from another college calls to see if our applications for admission are up or down as of a certain point in time.
Recommended Action: The Admissions Office should handle this.
Reason: This is an informal request for unofficial data from a peer. Since data at a point midway through the admissions cycle are not yet official, the Admissions Office is also in the best position to decide whether the information is too sensitive to release.

Endnotes

¹
The IR officer does not have the authority to release to anyone data on individuals (students, faculty, staff, etc.). Requests for data about individuals should go to the responsible office.