Developing and evaluating cybersecurity competencies for students in computing programs

Cybersecurity education models

According to the ACM Joint Task Force on Cybersecurity Education (2017), there is a need to develop a comprehensive curriculum in cybersecurity education. The primary purpose of cybersecurity programs is to equip the future generation with cybersecurity knowledge and experience (ACM/IEEE-CS Joint Task Force on Computing Curricula, 2013; ACM/IEEE-CS Task Group on Information Technology Curricula, 2017; ABET, Inc, 2018).

The US National Institute of Standards and Technology (National Institute of Standards and Technology (NIST), 2017) commenced a framework for cybersecurity workforce through collaborative efforts by the educational sector, public and private sectors. The framework also outlines the knowledge, skill, and ability (KSA) essential for organizations and entities who wish to implement cybersecurity in their workplace. Knowledge, Skills, and Abilities (KSAs) define the attributes and traits necessary for individuals and organizations to deliver the required level of performance. Such characteristics may be evident in expertise, skills and experience, acquired through performance-based learning.

ABET, Inc (2018) has specified the accreditation criteria for undergraduate cybersecurity degree programs in addition to current accreditation criteria specified for computer science, Information Systems (IS) and IT programs. ABET is an American private organization serving as an accreditation board for approving various intermediate education programs in engineering, computing and technology. ABET specifies and modifies the criteria to be fulfilled by university programs. ABET, Inc (2017) and ABET, Inc (2018) has contributed to incorporating cybersecurity in existing programs in Saudi Universities. ABET introduced the Engineering Accreditation Commission (EAC) program criteria and made all cybersecurity engineering programs mandatory to meet the Computing Accreditation Commission (CAC) standards besides other existing requirements. CAC criteria make it compulsory to equip the curriculum with rules and activities that ensure safe computing (ABET, Inc, 2017; Parrish et al., 2018).

However, ABET, Inc (2018) also acknowledge the concept of developing independent cybersecurity programs. The measures issued by ABET, Inc (2018) has made it mandatory for undergraduate programs to incorporate cybersecurity principles. In addition, ACM ACM Joint Task Force on Cybersecurity Education (2017) specified comprehensive curriculum criteria in cybersecurity education.

Cybersecurity competencies

Envisioning the future of cybersecurity is not a simple task. The current cybersecurity activities have been highlighted in the ACM Joint Task Force on Cybersecurity Education (2017) and Institute of Information Security Professionals (2018). Therefore, every IT degree program must incorporate cybersecurity. Hence, the significance of learning about cybersecurity becomes evident from implementing cybersecurity in multiple disciplines, including science, engineering, and business etc. This also implies that some knowledge and experience of cybersecurity is essential for every workplace.

The NIST Cybersecurity Framework contains five synchronized and continuous functions: identity, protect, detect, respond, and recover. These functions are fulfilled by having cybersecurity competencies (National Institute of Standards and Technology (NIST), 2014). Moreover, there should be focused measures of competency assessments; hence, the focus of the evaluation must be the functional or technical competencies (Succar, Sher & Williams, 2013). It is worth mentioning that the competency level needs to be established while evaluating an individual (Garavan & McGuire, 2001). Precisely, the experts may design the competency assessments to determine the superior performance level (expert) or a threshold level (minimum competency) (Shahidi et al., 2015). Therefore, all IT users must acquire cybersecurity necessary competency, a dynamic combination of cybersecurity knowledge, cybersecurity skills, and cybersecurity abilities (Parrish et al., 2018; Nilsen, 2017).

Knowledge, skills, and abilities (KSAs)

All the combined KSAs formulate competencies, which reveals the performance level of the combined KSAs (Chen et al., 2014). The specific actions needed to complete job tasks are directly associated with the KSAs (Baker, 2013). Therefore, the competency gaps involving additional training will be identified once the KSAs are determined (Chen et al., 2014). Besides discovering competency gaps, Baker (2013) expressed that the measures determining the level of task performance are none other than the KSAs.

Different jobs relate to several KSAs in the context of cybersecurity (Campbell, O’Rourke & Bunting, 2015). Specific jobs require a low level of combined KSAs. In contrast, some need a high level of combined KSAs (Lu et al., 2015). Moreover, KSAs are not certainly exchangeable between job functions or career fields (Conklin, Cline & Roosa, 2014). Hence, while determining cybersecurity KSAs, \an initial set of KSAs for all job functions must be the prime area of concentration (Chen et al., 2014; Conklin, Cline & Roosa, 2014).

Cybersecurity knowledge

According to Alavi & Leidner (2001), knowledge is defined as a reasonable belief that strengthens an individual’s capacity to initiate suitable action. Knowledge can be split into tacit knowledge and explicit knowledge. According to their explanation, the knowledge that can be taught is known as explicit knowledge. In contrast, the knowledge acquired from experience is not easily exchangeable is referred to as tacit knowledge.

As per Parsons et al. (2014), the following cybersecurity knowledge units were described for the students, namely: incident reporting, email use, Internet use, information handling, password management, mobile computing, strong passwords and social networking site use. At the same time, the following IT professionals cybersecurity knowledge units were recorded by Gross & Rosson (2007), i.e., antivirus software, access control, cybersecurity responsibilities, cyber vulnerabilities, cyber threats, file permissions, email encryption, policy compliance, phishing, privacy and sensitive information etc. Another cybersecurity knowledge unit is believed to be password reuse (Ives, Walsh & Schneider, 2004). Table 1 shows cybersecurity knowledge competencies.

Cybersecurity skills

The skills required to avoid a loss to IT infrastructure through the Internet are cybersecurity skills (Carlton & Levy, 2015). Cybersecurity skills can be associated with the required tasks or specific sets of actions (Conklin, Cline & Roosa, 2014). Acquisition of skill to prevent unauthorized access to an IT is critical to ensure access control within an organization. It is realized when the individual controls systems (Gross & Rosson, 2007; Ifinedo, 2012). Unauthorized access to sensitive information can only be reduced through proper access control.

The protection offered by antivirus software can be maximized by gaining skill in antivirus software (Dhepe & Akarte, 2013). The automatic update facility of antivirus software is activated by many organizations (i.e., auto-update configurations). However, organizations can come across the times where the update is required to be facilitated by a cyber-security expert (Dhepe & Akarte, 2013). Students must be well aware of handling an antivirus application, especially when the computer system notifies the user to update the antivirus application (Gross & Rosson, 2007).

The students should know the cookie usage skills because they may have unencrypted sensitive information to track the activity (DISA, 2015). Moreover, the students should be familiar with their Internet browser setting to manage their web cookie storage policy (DISA, 2015). Students must have practical skills in email security (DISA, 2015; Parsons et al., 2014). They must know the skill to configure the Email in such a way that prevents leaking confidential data (DISA, 2015; Parsons et al., 2014). Therefore, students must illustrate how they can control the downloading of malicious items besides carrying out the task of exchanging private information with the help of encryption (Barlow et al., 2013; DISA, 2015). Moreover, digitally signing emails must also be demonstrated to provide added security (DISA, 2015; Foster et al., 2015). In addition, the task of scanning all email attachments before use must also be illustrated (DISA, 2015).

Students must also acknowledge the skills in cybersecurity incident reporting to ensure denial of service to an unauthorized person (Imgraben, Engelbrecht & Choo, 2014; Parsons et al., 2014). Students need to know the personal mistakes required to be reported besides identifying suspicious individuals involved in security breaches (Parsons et al., 2014).

During internet connectivity at the workplace, the students must prevent opening the links to malicious Websites (Carlton & Levy, 2015). Hence, students must responsibly demonstrate that they will not click on malicious pop-up windows (DISA, 2015; Kumar, Chaudhary & Kumar, 2015). In addition, students ought to have skills in preventing such activities, through which the systems become vulnerable to malicious code (Barlow et al., 2013; DISA, 2015). As a result of this malicious code, hackers can access the system/network, corrupt the files, and erase hard drives (DISA, 2015). The worms, viruses, spyware, Trojan horses and scripts are included among the examples of malicious code (DISA, 2015). While most students do freelance works, carry out online studies or perform telework, it is appreciated to have skills in securely operating mobile computing devices (DISA, 2015). The task of locking the mobile computing device when inactive must also be demonstrated by the students (Parsons et al., 2014). Skills required by students are the stoppage of password reuse (Ives, Walsh & Schneider, 2004). Concerning confidential data, the skill to prevent phishing attempts is obligatory for the students (Carlton & Levy, 2015; DISA, 2015; Furnell, Tsaganidi & Phippen, 2008). As exposed in the literature review, students need several cybersecurity skills (Table 2).

Cybersecurity abilities

The physical or/and mental ability to apply skills to execute a task is called the ability (Tobey, 2015). Likewise, the foundation for skills and knowledge application is the ability (Prager, Moran & Sanchez, 1997). The primary cybersecurity abilities are written communication, near vision, advanced written comprehension, written expression and problem sensitivity (Campbell, O’Rourke & Bunting, 2015; Trippe et al., 2014) Nilsen, 2017). The close-up viewing defined for objects almost sixty centimetres or less than two feet from the eyes is described as near-vision or accurate near vision (Colman, 2015). Researchers believe that a cybersecurity ability to view computer screens also corresponds to the near vision concept. According to them, the ability to express when something goes wrong or is likely to go wrong is known as problem sensitivity (Nilsen, 2017). It is nothing to do with problem-solving. Instead, it is only to identify a problem (Trippe et al., 2014).

The ability to read and comprehend government or/and technical documents refers to advanced written comprehension (Trippe et al., 2014). Researchers recommend advanced written understanding as one of the cybersecurity abilities, which can read cybersecurity policies and guidance. The broadcast of a message in written symbols is known as written communication (Terkan, 2013, p. 149). According to Poteet (1980), a visible representation of feelings, thoughts, and ideas is described as written expression, where the writer’s language symbols are used to record or communicate. The experts conclude that written expression is guided as cybersecurity ability. The potential investigators could transcribe cybersecurity incident reports besides speaking to a cybersecurity contact for the relevant issues.

Siponen, Mahmood & Pahnila (2014) support the argument that an entry-level step to follow and implement cybersecurity procedures and policies is the ability to understand cybersecurity terminology. According to the findings of Hagen & Albrechtsen (2009), the ability to anticipate, monitor and respond to cybersecurity challenges are the three fundamental abilities, such as awareness, knowledge and behaviour are needed to measure the information security intervention. Table 3 shows cybersecurity abilities criteria.

Fuzzy linguistic decision-making methods

Acquiring knowledge, skills, and abilities involves a decision-making process. According to Ganin et al. (2020), cybersecurity risk management and assessment is a multi-criteria decision problem. Experts have proposed several techniques for complex decision-making issues in the practical world, particularly those involving multiple choices. The selection of a specific method depends on the nature of info available to decision-making individuals. Dominance technique proves effective in case of lack of information available to decision-makers while maximin or maximax technique is used when pessimistic or optimistic information is available.

Moreover, the selection of the method is further categorized into sub-categories in case of the availability of attributes. The conjunctive and disjunctive methods are usually employed if a standard level of information relevant to each attribute is available. However, simple additive weighting (SAW), analytic hierarchy process (AHP), TOPSIS (a technique for order preference by similarity to ideal solution) and the ELECTRE (elimination and choice expressing the reality) method etc. are used when ordinal or cardinal scales are used in the analysis of attribute weights (Lu et al., 2007). Although various MCDM techniques are available, the TOPSIS method has been extensively used by previous researchers (Sohaib et al., 2019). The main features offered by TOPSIS that make it favourable are that it is based on the logic of an individual’s preference; moreover, it makes use of simple computation processes and considers the ideal and the anti-ideal solutions concurrently (Shih, Shyur & Lee, 2007). TOPSIS has made it possible to consider criteria and alternatives together (the situation in our study), which is not the case with pairwise comparison techniques. Besides these benefits and the extensive popularity of TOPSIS, it has been our preference because of the features of extensions applicable in fuzzy environments. In this proposal, we will highlight and demonstrate the effectiveness of this feature concerning the ranking of alternatives.

The central underlying concept behind the TOPSIS method is that an alternative is deemed the best one. It needs to correspond to the ideal positive solution and to be contrary to the negative ideal solution besides adhering to the subsequently mentioned steps (Hwang & Yoon, 1981).

Fuzzy group decision-making methods

Due to the complex nature of the issues faced in the practical world, it is imperative to consider various viewpoints when solving a case. Usually, a group of experts give their opinion, which serves as the solution to the concerned problem; the process is commonly known as multi-criteria group decision making (MCGDM). Previously, group decision-making involved the participation of multiple experts $E=\left\{e_1,e_2,\dots ,e_k\right\}\left(k\ge 2\right)$ who present their opinion regarding alternative X to solve the issue at hand. The alternative was chosen in two steps, including the aggregation and exploitation stages. The aggregation stage involves using an aggregation operator to compile all the views expressed by different experts. The aggregation operator merges all this data to develop a collective preference matrix containing all the proposed solutions to the concerned problem. This is followed by the exploitation stage, which involves analyzing the joint preference matrix and choosing the most appropriate alternative from all the available options proposed for solving the problem (Rodrıguez, & Martinez, 2013). Sohaib et al. (2019) explain the use of a general scheme which is only possible when experts give their preference from among various available alternatives through the employment of fuzzy linguistic variables. The steps mentioned below are involved in the development of solution scheme (Rodrıguez, & Martinez, 2013):

• The first step involves the selection of a set of linguistic terms that contains semantics. Consequently, linguistics implies meaning and is deemed linguistic descriptors, which experts analyze to determine their preferences from various available alternatives and assign weights to criteria weights in light of their knowledge and experience.

• The subsequent step involves selecting an aggregation operator that compiles all the preferences given by individual experts to present collective linguistic information.

• The final step involves the selection of the best alternative(s) among the available ones.

The aggregation phase involves the application of various models of linguistic computing, including the models proposed by Degani & Bortolan, (1988), Delgado, Verdegay & Vila (1993), Herrera & Martinez (2000). On the other hand, the exploitation phase involves the application of conventional MCDM methods.

A 2-Tuple fuzzy linguistic group TOPSIS model

TOPSIS techniques and the fuzzy extensions offered by this technique are known for their extensive use in various applications (Ju, Wang & You, 2015; Lu et al., 2016; Wei, 2010). However, this technique fails to meet the computing with words (CW) criteria due to the inaccuracy of the linguistic domain or non-linguistic outcomes indicated by applying the domain of the preferences for distances instead of the linguistic domain. Hence, this drawback called for proposing a novel linguistic TOPSIS model. Such a model is presented in this study. Such a model uses a 2-tuple linguistic model wherein the fuzzy linguistic variables are used to assign weightings to each criterion. Thus, this proposed model satisfies the CW criteria since it involves the use of appropriate syntax and semantics for preferences and distances, leading to precise, flexible and understandable linguistic outcomes.

Let’s consider a set of alternatives A = A₁, A₂, …, A_m for the criteria C = C₁, C₂, …, C_n being evaluated by a set of decision-makers D = D₁, D₂, …, D_k. Moreover, considering that the set of the linguistic term for assigning weightage to criteria is U = u₁, u₂, …, u_p andconsidering that the set of the linguistic term for the analysis of alternatives is S = s₁, s₂, …, s_t. Also, consider that the set of the linguistic term to determine the similarity of a linguistic term s_p with another linguistic term s_rbe S′ = l₁, l₂, …, l_t′ and the set of the linguistic term to determine the distance of the linguistic term s_p from another linguistic term (s_r) isS^′′ = r₁, r₂, …, r_t^′′. Let’s consider a weight vector $U_t={\left(u_j^t\right)}_{1\ast n}^T$for which the decision-maker D_t ∈ D proposes a linguistic value preference $u_j^t\in U$ concerning criteria C_j ∈ C. Moreover, consider the decision matrix $X_t={\left(r_{ij}^t\right)}_{m\ast n}$ for which the proposed linguistic value preference is $r_{ij}^t\in S$ as per the decision-maker D_t ∈ D considering the alternative A_i ∈ A and the criteria C_j ∈ C. In such a case, all decision-makers are supposed to carry an equal level of significance. The steps involved in the advanced version of TOPSIS have been mentioned subsequently. Please see Sohaib et al. (2019) previous work on a novel 2-tuple fuzzy TOPSIS method discussed in detail.

Step 1

This research aimed to evaluate the cybersecurity KSAs competencies for computing students of Saudi higher education institutions. The KSA model was used to measure a core set of required cybersecurity knowledge, skills, and abilities (KSAs) essential as cybersecurity competencies (Nilsen, 2017). The three criteria and twenty-seven sub-criteria were determined based on the KSA model. The alternatives are University A (Uni. A), University B (Uni. B) and University C (Uni C.). The criteria structure of the KSA competencies is discussed in the above sections.

Step 2

In the second step, three professors from three different universities attempted to collectively evaluate cybersecurity knowledge, skills, and abilities essential as cybersecurity competencies. A Delphi method using a ’consensus rule’ was used to improve the process of decision making. Delphi method consensus rule in fuzzy group decision-making aims at making a mutual agreement about an opinion (Lu et al., 2007). Thus, a consensus rule was approved using a questionnaire to build interdisciplinary understanding about the different views.

Step 3

In step 3, the relative importance of criteria and the alternatives under study was weighted using linguistic terms. The linguistic terms for weighting the criteria and the alternatives are presented in Tables 4 and 5. Triangular fuzzy numbers denote the membership functions of all linguistic terms for the sake of simplicity. Finally, Table 6 shows the linguistic variables for measuring the distance to choose the best alternative.

Step 4

Finally, in step 4, the novel 2-tuple group TOPSIS method (Sohaib et al., 2019) was used to obtain the desired ranking. Out of the three alternatives, the best alternative was selected as the ideal strategy based on the maximum closeness degree to the ideal solution.

Criteria weights

The criteria weights and the linguistic terms as presented in Table 4; the experts’ judgments have resulted in Table 7.

Alternative evaluation

Table 8 shows an alternative evaluation decision matrix was resulted using linguistic terms (Table 5).

Finally, the novel 2-tuple group TOPSIS method (reference) was applied to deliver a decision as discussed in step 4 of the research methodology. Table 9 shows the 2-tuple linguistic values. The 2-tuple arithmetic mean was to obtain collective values.

Table 10 shows the results of the 2-tuples evaluation matrix and their mean. In the experts’ view, cybersecurity threats and vulnerabilities with a 2-tuple of (u₇, 0 in Table 9) are the most important criteria with the importance of Very High, followed by access control and policy compliance.