Regarding the nature of the objective, this research is classified as exploratory, as it promotes the evaluation and generation of knowledge in the decision maker, the librarian. It seeks to develop knowledge on the subject and, with this, organize a set of aspects, in the form of performance indicators, which allows the “Evaluation of the informational services offered to students by IFSC library – Campus Araranguá”, through the perception and values proposed by the decision maker, in this case, the librarian (Creswell, 2014; Marconi & Lakatos, 2010; Richardson, 2010; Yin, 2010).

As for the nature, this study is considered as an applied research, in the form of a case study, with the objective of solving a real problem, that is, the construction of an evaluation model of informational services in a library of the Federal Institute of Santa Catarina. Regarding the approach to the problem, this research is qualitative. Regarding the qualitative approach, it is found in the structuring stages and recommendations of MCDA-C methodology. The constructivist foundation, followed in this research, takes into account the opinions and preferences of the Decision-Maker, in this case, the librarian. With regard to data collection, it encompassed both primary and secondary data.

Primary data were collected through semi-structured interviews, in accordance with the MCDA-C Methodology protocol, which sought to verify the concerns of the decision maker for the structuring of the model. Under the terms of the steps/activities inherent to the MCDA-C Methodology, the decision maker was encouraged to explain their perceptions, with the help of the facilitator. Thus, as a script, the steps of the MCDA-C Methodology, presented in section 4, were followed, and each semi-structured interview required around two hours, that is, considering that five activities were carried out, under the terms of the Methodology protocol, the duration of the total interviews involved around 10 hours.

The secondary data focused on the analysis of the reports generated in the Sophia system, software for managing the collection of IFSC libraries, where the quatitative and qualitative data of the collection and users are found (Creswell, 2014; Marconi & Lakatos, 2010; Richardson, 2010; Yin, 2010).

In agreement with Ensslin et al. (2010), the implementation of the MCDA-C methodology, as a scientific management instrument, occurred from the 1980s. The scientific bases of MCDA-C appear with the research publications of Roy (1993) and Landry (1995) when deliberating the limits of objectivity for decision support methods; in the studies of Skinner (1986) and Keeney (1992), when adopting that the characteristics (objectives/criteria) are particular for each context, from the perceptions of the manager/decision maker; and, in addition to these with the work of Bana e Costa (1993) when specifying the convictions of MCDA (Ensslin et al., 2010).

The distinction between the MCDA-C methodology and the traditional MCDA methodologies resides, above all, in the fact that the traditional MCDAs restrict the support to the decision in two phases: from formulation, and from evaluation to select, according to a defined group of objectives (with little or no participation of the decision maker) in which, among the previously established possibilities, is the best (optimal solution) (Ensslin et al., 2010). According to Ensslin et al. (2010), the traditional MCDA research logic is rationalistic, while the MCDA-C employs a constructivist logic, taking into account the decision maker's perception and values.

MCDA-C methodology is developed in three main phases, as evidenced in Figure 1: (i) Structuring Phase; (ii) Evaluation Phase; and (iii) Recommendation Preparation Phase.

Figure 1.
MCDA-C methodology phases
Source: Adapted from Ensslin et al. (2001).

The structuring phase, fundamentally, proposes to organize, develop and expand the decision maker's knowledge about the decision-making context. For this to happen, some steps need to be taken to achieve this goal (Ensslin et al., 2000; Ensslin et al., 2001; Bortoluzzi et al., 2011; Ensslin et al., 2020a; Ensslin et al., 2020b; Caldatto et al., 2021). In the first step of this phase, the activities of describing the environment (problem that will be treated), delimiting the actors (stakeholders) involved in the decision-making process, designing a label for the problem and organizing a summary that includes a brief summary of the decision-making context (Ensslin et al., 2001, Bortoluzzi et al., 2011; Dutra et al., 2015) are carried out.

The second step of the structuring phase comprises the activities of identifying the Primary Elements of Evaluation (PEEs), the construction of the Concepts (present pole and opposite pole) for the PEEs, the construction of the Family of Viewpoints (FPV) and the validation of PEEs as sufficient and necessary (Cardoso et al., 2016).

The third and final step of the structuring phase refers to the construction of Cognitive Maps, with the identification of Clusters and Sub-clusters (if any); the arrangement of the value tree, with the respective Elementary Points of View (EPVs), which are the dismemberment of PVFs until the identification of a lower Point of View, which can be subject to ordinal measurement (Ensslin et al., 2000; Ensslin et al., 2001); the constitution of Descriptors, with the distinction of reference levels (ordinal scale), the performance profile ( status quo) and the imputation of performance stages, designated as Excellent, Competitive (market or expected) and Compromising ( Ensslin et al., 2000;Ensslin et al., 2001;Bortoluzzi et al., 2011; Zamcopé et al., 2012; Dutra et al., 2015).

In the evaluation phase, the objective is to explain the qualitative model built in the structuring phase, in a mathematical model, where the quantitative performance of the individual context (in each aspect of the model) or globally (global evaluation of context performance) can be verified (Ensslin et al., 2000; Bortoluzzi et al., 2011; Dutra et al., 2015; Cardoso et al., 2016). It is noteworthy that this phase was not developed in this work, considering that the librarian chose to focus on the development of knowledge in relation to the services offered by the library. Thus, the quantitative approach was not applied in the present research.

In the recommendation phase, we seek to propose actions that the decision maker can put into practice to improve the performance of the studied context, focusing on improving the performance of the descriptors considered in the model (Ensslin et al., 2001; Bortoluzzi et al., 2011; Azevedo et al., 2013; Rolim-Ensslin et al., 2014; Lacerda et al., 2014; Dutra et al., 2015; Cardoso et al., 2016).

The knowledge generated allows the decision maker to understand, graphically and numerically, for each performance indicator, whether the performance is excellent, competitive or compromising. This additional information from the decision maker will provide a subsidy for him to define which action, or strategy, is the most appropriate (Ensslin et al., 2001; Cardoso et al., 2016).

It is noteworthy that the model built and presented, from the next section, based on the Multicriteria Methodology of Support to Constructivist Decision (MCDA-C), focuses on the structuring and recommendation phases, presented in Figure 1. It is evident that, in the recommendation phase, 5W2Hmethodology was used, a method created by the Japanese automotive industry in the course of quality studies. Currently, it is considered an administrative and quality tool, which can be applied in various business areas and in different contexts, within an organization (Napoleão, 2018). This method can also be used to prepare action plans focused on improving services. 5W2H is an acronym, derived from the initial letters of seven questions What? (o quê?), Why? (por quê?), Where? (onde?), When? (quando?), Who? (quem?), How? (como?) e How Much? (quanto custa?) (Silva, 2019).

The library of Araranguá campus of IFSC was created together with the campus, on March 20, 2008. Currently, the campus receives about 900 students in professional qualification courses (FIC), 3 integrated technical courses, 5 subsequent technical courses, 4 specializations, 1 undergraduate degree, 1 technology degree and 1 youth and adult education integrated into technical education (Proeja). (Federal Institute of Education, Science and Technology of Santa Catarina, 2023). In the library, where 4 servers operate, there is a collection of more than 4,000 titles and 9,000 copies in various media, in a physical area of 152.88m². In it, a computerized collection management system is adopted, through software Sophia, and using internet access for consultation, renewal or online reservation of the collection.

The purpose of the libraries of the Federal Institute of Education, Science and Technology of Santa Catarina, according to the respective regulation, is “To offer informational access to the teaching, research and extension programs of IFSC, developing services and products that make it possible to satisfy the informational needs of its users.” (Federal Institute of Education, Science and Technology of Santa Catarina, 2023).

Among the informational services offered by libraries are: Home loan service; Renewal of loan; Reserve of materials; Local and online consultation service to the collection; Internet access for academic research; Bibliographic survey; Reference service; Guidance for standardization of academic works; Guided visits; List of new acquisitions; Preparation of institutional catalog sheets and User training (Federal Institute of Education, Science and Technology of Santa Catarina, 2011). Thus, it is observed that there is concern with the performance of information services, in front of the target public (students).

For the purposes of libraries to be achieved, special care is taken to assess whether there is a need for changes or improvements in the information services performed. In this sense, this research was developed to evaluate the performance of information services made available to students, and that subsidize decision making in IFSC libraries, applying the Multicriteria Methodology of Decision Support – Constructivist (MCDA-C) with the decision maker of the library of IFSC Araranguá campus. The library of this campus follows the same precepts as the other institution libraries.

With the development of this work, it is possible to: identify, organize, establish reference levels judged as relevant by the decision maker. The generated model allows viewing the profile of the current situation, highlighting its strengths and weaknesses and having a process to identify opportunities for improvement. Finally, it remains to inform that the feasibility of the research was first due to the interest of the librarian, who works at the institution, in building a personalized model that provides a basis for decision-making; then, because one of the authors is part of the institution.

The performance evaluation model of information services, offered by IFSC library of the Araranguá campus, was built according to the perception of the campus librarian (decision maker), based on the information acquired in different interviews. According to the decision maker's statements, in these interviews, an attempt was made to produce information regarding the aspects that were considered important and that impact the performance of the informational services offered.

The choice of the campus librarian as a decision maker in this process is due to the fact that she is responsible for organizing the information services in this library, thus being the person with the necessary skills to make the decisions regarding the information services offered by the campus. Thus, based on the main focus of the work, pointed out by the decision maker, the label that best identified the model was defined: “Evaluation of the informational services offered to students by IFSC library – Araranguá campus”. Likewise, the actors involved in the decision-making process of the research context were identified, as shown in Table 1.

Table 1
Actors involved in the decision-making process.

Source: Authors' elaboration.

Thus, once the label and the actors involved were defined, the Primary Evaluation Elements (PEEs) were identified, which are qualities or characteristics that the decision-maker considers important and/or that impact their values and preferences regarding the context (Ensslin et al., 2013; Ensslin et al., 2010). From the identification of the PEEs, the concepts that guide the direction of preference were constructed, pertinent to the PEE and which is followed by its psychological opposite (minimum permissible of the underlying objective), represented by ellipsis (...) that sould be read as “rather than” (Cardoso et al., 2017). However, it is observed that a PEE can unfold in more than one concept (Ensslin et al., 2013; Ensslin et al., 2010). At this stage of the research process, in all, 41 PEEs were counted, from which 75 concepts were built. From these 75 concepts, a cutout was selected and presented in Table 2.

Table 2
Selected PEEs with built-in concepts

Source: Authors' elaboration.

After constructing the 75 concepts, the information generated was weighted and organized by areas of concern chosen by the decision maker, and then given rise to candidates for Fundamental Viewpoints (FPVs). The concepts were divided into five areas of concern, called as follows: FPVs: “1. Student’s service”; “2. Educational levels”; “3. Planning of services"; "4. Financial resources”; and “5. Library staff.” To understand the analyzed context, FPVs were organized in a structure called Family of Points of View, as shown in Figure 2.

Figure 2.
Family of Fundamental Viewpoints (FFPV).
Source: Authors' elaboration.

It is possible to see in Figure 2, that, according to the concerns presented by the decision maker, the five FPVs were identified. Below each of them are listed the concepts that exhibit the same strategic concern for the decision maker. In the same figure, the dismemberment in Elementary Points of View (EPVs) of FPV "1 Students' Service" is visualized. Subsequently, EPV Cognitive Map “1.1.1 Basic Services” was built (Figure 3). With the elaboration of the Cognitive Map, it is sought to structure and expand the decision maker's knowledge, through means and ends relations.

For this construction, the concepts of the EPV "1.1.1 Basic services" were organized in the form of a map, as shown in Figure 3, and at the bottom of the map are the concepts of means (operational) and at the top are the considered strategic final concepts. A cause and effect analysis was performed on each concept and, as the understanding of the problem was expanded, the concepts were modified, and at times it was necessary to build new concepts. The new concepts are highlighted on the map, in blue and without the numbering referring to the PEEs. Based on the organization of the concepts on the map, the clusterswere established, each one being related to a designation that conceives the decision maker's focus of interest. (Ensslin et al., 2013; Ensslin et al., 2010).

Figure 3.
Cognitive Map – EPV 1.1.1 Basic Services
Source: Authors' elaboration.

Figure 3 shows that the EPV map "1.1.1 Basic services" displays three clusters: the first, which represents the "guided visits" service, is offered when the student enters the institution, constituting a way for him/her to get to know the space and the basic services of the library. The second cluster presents the “access to materials”, dealing with access to the password and the loan of materials, and how to continue the loan service. The last cluster related is the “access renewal”, which deals with the students' knowledge in renewing the loan of materials. The process of creating the cognitive map brought the possibility for the decision maker to obtain relevant information for the development of the concept according to her interests and pointed out the means to achieve the desired end. The expansion of EPV “1.1.1 Basic Services”, through this procedure, allowed the aggregation of concepts from the cognitive map, thus giving rise to the Hierarchical Value Structure (HVS) (Ensslin et al., 2013). Figure 4 shows the HVS for EPV “1.1.1 Basic Services”.

The Hierarchical Value Structure, shown in Figure 4, shows the Elementary Viewpoints (EPVs), created from the cognitive map. Below the EPV "1.1.1 Basic services", the identification of Points of View that can be measured can be observed. For each of these EPVs, scales were constructed, called, in the MCDA methodology, as descriptors, which allow the evaluation of the performance of each EPV characteristics, which operationalize the objectives presented in the cognitive map (Cardoso et al., 2017; Rolim-Ensslin et al., 2015).

In the scale presented in Figure 4, the reference levels "Good" and "Neutral" are exposed, which allow the comparison of performance between the descriptors (Ensslin et al., 2000), and the performance presented at one of the scale points above the reference level "Good" will be considered excellent; those below the reference level "Neutral" will be considered compromising performance; and those between the two reference levels will be considered as competitive performance (market or expected) (Cardoso et al., 2017, Ensslin et al., 2001). Based on the defined reference levels, it is possible to describe the performance profile of the basic services described in the research and which are offered by the Araranguá campus library, as illustrated in Figure 4.

Figure 4.
Hierarchical Value Structure (HVS) and descriptors of EPV 1.1.1 Basic Services
Source: Authors' elaboration.

Looking at Figure 4, it is possible to see that, for the EPV descriptor "1.1.1 Basic services", five levels were built, which allowed to evaluate the decision maker's concerns related to this descriptor (ordinal measurement scale). The blue dashed line shows the performance profile (status quo) is demonstrated, indicating that: three descriptors have market performance; one has excellent performance; and one descriptor shows compromising performance. The success of a service will only be achieved if the reality is known and, for this, indicators play a fundamental role in the performance evaluation processes.

Thus, we proceed to describe how the performance level calculations (status quo) of the indicators shown in Figure 4 were performed.

To measure the level of the first descriptor, "1.1.1.1 Guided Visits", the percentage was calculated based on the incoming classes in semester 2019.1, chosen because it was the last semester with fully face-to-face pre-pandemic assistance. Thus, nine groups entered, and four (44.44%) of them carried out the guided visit, leaving this indicator at a neutral level.

In order to verify the second descriptor, "1.1.1.2.1 Registration", a report was generated in software Sophia on the number of enrollments registered in the system in 2019 (752), and it was asked to IFSC Library System (SIBI/IFSC) coordination to report the number of these students who had a password. The report resulted in data that 32.44% (244) of enrolled students created a password for use of the library's loan services, leaving this indicator also at the neutral level.

In order to verify the data of the third descriptor, “1.1.1.2.2 Loan”, the number of 2019 enrollments (752) available in Sophia was used, and a loan report was generated from these applicants in 2019. With the results generated through software, it was found that the percentage of incoming students who took out a loan was 27.92% (210), thus remaining this indicator at a compromising level.

For the fourth descriptor, “1.1.1.2.3 Loan (reference)”, the measurement was made based on the decision maker's personal experience and in the literature on the reference process. The term “reference process” is used to designate the activity involving the person who is performing the consultation and the execution of the reference service provided to it. The reference service is the assistance that is offered to the user who needs information (Grogan, 1995). Also according to Grogan (1995), the normal reference process has eight steps to be defined: The problem; The need for information; The initial question; The negotiated question; The search strategy; The search process; The answer; The solution. In this case, based on the literature and the decision maker's personal knowledge, this indicator was at excellence level, with 95%.

In the case of the fifth and last descriptor, “1.1.1.3 Renewal of access”, the loan renewal report of 2019 was generated, and the result showed that, of the 7,633 renewals carried out in the period, 6,388 (83.69%) were carried out by the student itself, using Sophia's access on the internet, and only 1,245 (16.31%) were carried out by the staff in the library, in person. That is, the indicator was also at excellence level, with 83.69%.

In a performance overview, one indicator performed excellently: “1.1.1.2.3 Loan (reference)”, with the percentage of students who obtained a loan through the reference process. Three indicators remained at the competitive level: (1) “1.1.1.1 Guided Visits”, with the percentage of classes that conducted the guided visit in relation to the classes that entered the period; (2) “1.1.1.2.1 Registration”, with the percentage of incoming students who registered a password in the library; (3) “1.1.1.3 Access renewal”, with the percentage of students who were able to renew the materials loan without the staff's help of the library. And one indicator showed compromising performance: "1.1.1.2.2 Loan", with the percentage of incoming students who borrowed materials.

Most of the descriptors (loan, renewal, guided visits) that resulted from this research were also found as quality indicators in libraries, calculated not based on students, but rather based on the number of collections, on the literature, according to studies by Paula and Vergueiro (2018), Viapiana et al. (2018) and Pimenta (2016) The indicators that were based on password registration and the number of loans made after going through the reference process were not found in the literature.

It is understood that this result comes from the constructivist characteristic of MCDA-C methodology, which analyzes that the proposed model is customized to the application context (Cardoso et al., 2017). In this way, the particularities and attributes of the IFSC library - Araranguá campus and the librarian's perception were respected in the construction of indicators in the model structuring phase.

In the recommendations phase, the descriptor that performed at the compromising level was considered, and improvement actions were proposed for the growth and development of the information services offered by IFSC library, Araranguá campus. As shown in the EPV descriptors “1.1.1 Basic services” (Figure 4), the only performance indicator of this evaluated EPV, which was at the compromising level, was: "1.1.1.2.2 Loan", with the percentage of incoming students who borrowed materials. Table 3 presents the recommendations for improvements suggested to improve the performance of the indicator, causing it to achieve higher levels of performance.

Table 3
Recommendations for improvement actions, using 5W2H.

Source: Authors' elaboration.

To achieve higher levels of performance, increasing the number of loans, four actions were considered. The first (1st action) refers to improving the quality of the collection of technical books, in order to meet the demands of students and course staff. For this, the references of Pedagogical Projects of the Courses (PPCs) would be used and, if they were outdated or exhausted in the book market, new titles would be suggested. The second (2nd action) deals with the updating of the literature collection (fiction, romance, suspense) to meet the interests of the library's reader profile, which could suggest new titles for purchase. In the third (3rd action) it was proposed to disclose what the library already has and what will be acquired, making more people aware of what the library offers. The fourth (4th action) deals with improving the location of the library within the institution, since it is located far from the space with the greatest circulation of people.

The value estimates for updating the collection were made based on the values related to the collection purchases and the suggestions for the purchase of technical and literature books, made by servers and students, in the last two years. And to estimate the value of building a new physical space for the library, an estimate was requested from the responsible budget engineer.

As can be seen in Table 3, improvement actions are presented using 5W2H tool. The proposed improvement actions depend on several actors, and the entire library team is involved, highlighting that the coordination and the librarian (decision maker) manage the activities developed by the team In this case, it is emphasized, in most actions, the need for cooperation between course coordinators, campus management, teachers, all profiles of servers and students, as they involve actions that depend on or are demanded by the academic community. However, as they interfere with the performance of the library, it is up to the librarian, together with the coordination, to seek solutions from those involved.

It is noted that the only EPV descriptor "1.1.1 Basic services", which presented compromising performance, refers to the percentage of incoming students who borrowed materials during the first year at the institution. Thus, this situation now provides the opportunity for improvement, through the proposed actions, cooperating to improve the information services offered by the IFSC library, Araranguá campus, mainly to improve the provision of basic services.