|
|
|
|
|
|||||||||||||||
Recent Activity
no record found
|
Validating a Knowledge Assets Measurement Instrument Ron Freeze, Arizona State University Uday Kulkarni, Arizona State University Abstract Measurement of organizational knowledge assets is necessary to determine the effectiveness of knowledge management initiatives. A Knowledge Management Capability Assessment instrument has been developed and operationalized to measure knowledge assets identified as Knowledge Capability Areas. A longitudinal field study is initiated in a large microchip manufacturing company to determine the reliability and validity of the KMCA and to assess the success of KM initiatives. In this paper, we provide the initial validation of the KMCA with empirical evidence from two business units of the company. Confirmatory factor analysis revealed that four Knowledge Capability Areas can be conceptualized in terms of latent descriptor variables. Each capability area is identified as an overall latent factor influencing a set of latent descriptor variables. Second Order and General- Specific structural equation models of each capability area provide evidence of the validity of measurement of these knowledge assets. 1. Introduction The quest to leverage knowledge assets through effective knowledge management (KM) is a strategic initiative for many firms. Management literature has noted the lack of effective management of knowledge and called for establishing quantitative measures for these intangible assets [1, 2]. Unfortunately, most KM initiatives in reality have been information projects that result in only the consolidation of data and not much improvement in products or innovations [3]. In order to initiate effective knowledge management, firms must focus on the identification of specific knowledge assets and the capabilities that they represent within an organization. Only through adequate measurement of these knowledge assets can firms begin to tie their capabilities to value generating metrics and move towards a sustained competitive advantage. Previously identified KM application areas are: knowledge repositories, lessons learned, expert networks and communities of practice [4]. The Cognizant Enterprise Maturity Model - CEMM introduced the concept of measuring 15 Key Maturity Areas within an organization to improve business value [5]. Although each of these frameworks have provided valuable steps toward understanding the nature of knowledge management within an organization, none have identified separate knowledge capabilities that may be individually measured and leveraged within a single organization to more effectively meet a business unit?s objectives. The objective of this research is to develop and validate a set of Knowledge Capability Area (KCA) measures that accurately capture a firm?s knowledge management ability and status. For this purpose, we developed and operationalized a Knowledge Management Capability Assessment (KMCA) instrument to measure the effectiveness within each KCA. A longitudinal field study is underway in a leading microchip manufacturing company to determine the reliability and validity of the KMCA and to assess the success of KM initiatives. In this paper, we provide the initial validation of the KMCA with empirical evidence from two large independent organizational units of the company. We begin by reviewing the composition of a firm?s knowledge asset structure. This includes the human capital, technological factors, knowledge lifecycle, and the tacit/implicit/explicit nature of knowledge. Through the process of identifying the KCAs, we recognize that the various knowledge types significantly affect the composition of each KCA. We identify four important KCA?s that represent distinct capability areas of Knowledge Management - Lessons Learned, Knowledge Documents, Expertise and Data. Each KCA consists of multiple latent descriptor factors that represent an organization?s ability to effectively and efficiently manage this aspect of knowledge. These latent descriptor factors are considered First Order constructs. With data gathered from the two large business units of the company, we conducted Exploratory Factor Analysis (EFA), utilizing maximum likelihood factoring ? ML, and verified the alignment adequacy of the items to the posited latent descriptive factors. Results of the convergent/discriminant validity and the EFA confirm the adequacy of the constructs. Then we tested each KCA 0-7695-2268-8/05/$20.00 (C) 2005 IEEE Proceedings of the 38th Hawaii International Conference on System Sciences - 2005 1 through Confirmatory Factor Analysis (CFA) using General Specific and Second Order measurement models. The CFA establishes each capability area as a significant construct and validates the measurability of the KCA factors. Significance of the measurement models provides the initial evidence of the reliability of measurement for individual capability areas. 2. Knowledge Capability Areas In spite of the recognized need for creation and utilization of knowledge assets, a standard, well-accepted description of what is knowledge continues to be incomplete. Some view knowledge as non-existent without the knower [6], while others claim to have successfully captured it into knowledge objects [7]. In order to make effective use of knowledge assets, organizations must be able to identify and quantify these resources. Only when these knowledge assets are clearly identified can the capabilities associated with them be measured and effective managing of knowledge begin. Knowledge Asset Structure Knowledge assets are intangible assets that encompass the knowledge as well as the ability of an organization to leverage that knowledge. Therefore, the structure of knowledge assets includes the extent to which individuals exploit the knowledge. A firm?s employees, also called knowledge workers, human capital etc., are integral to establishing the capability areas as knowledge assets. This perspective views knowledge assets as organizational resources defined within the resource-based view of the firm. The literature on resource-based strategy treats human capital as one of the key rent-generating [knowledge] assets of a firm [8, 9]. Knowledge assets also include the technology designed to facilitate the interaction of the knowledge through each stage of its lifecycle with the human capital. Davenport, et al [10], identify three major factors ? management and organization, information technology, and work place design ? that influence the performance of knowledge workers and knowledge-based organizations, emphasizing the interplay of organizational, technological and physical factors (in knowledge work). In order to acknowledge the ownership of knowledge and how the organizational, technological and physical factors interplay, companies that want to develop and use knowledge most profitably need to treat it differently according to the stages of its life [11]. KM research takes several views of these stages of knowledge: knowledge flows, steps to knowledge management, architectures for explicit knowledge and knowledge lifecycle [1, 11, 12]. We view the knowledge lifecycle as an acquisition/ storage/retrieval/application cycle. The actual acquisition of knowledge and decision to transfer resides solely with the capabilities provided by a firm?s human capital. Knowledge can only be acquired and utilized if the knowledge worker recognizes and makes use of its value. Processes and technologies to codify and capture this new knowledge in repositories under existing or expanded taxonomies is the next logical stage while leveraging knowledge assets. These organizational and technological factors begin the storage stage of the knowledge assets and represent the potential use that can be made of captured knowledge. The retrieval stage is the culmination of the decision to reuse / apply existing knowledge. The success of any attempt to leverage knowledge assets of a firm is measured by whether a knowledge reuse has occurred. To measure the transfer of knowledge, actual knowledge use by a firm?s human capital must be identified. Tacit/Implicit/Explicit Knowledge Tacit knowledge has engaged researchers for many years and is described in a multitude of ways: practical know-how, difficult to articulate, transferred only via observation and practice, subconsciously understood and applied and rooted in action, experience and involvement in a specific context [1, 5, 13-15]. Similarly, there is a wealth of research about explicit knowledge depicting its essence as: embodied in code or language, knowledge already documented, precisely or formally articulated, codified and communicated in symbolic form and/or natural language [1, 5, 13, 16]. A holistic view of organizational knowledge assets must encompass a view of both the tacit and explicit nature of knowledge. The connection between tacit and explicit knowledge is apparent when one recognizes that tacit knowledge is the means by which explicit knowledge is created, captured, assimilated, and disseminated [6] and where tacit knowledge forms the background necessary for assigning the structure to develop and interpret explicit knowledge [16, 17]. These connections between explicit and tacit knowledge imply a continuum that provides a scale of media richness vs. externalization: face-to-face (tacit knowledge), telephone, written personal, written formal, numeric formal (explicit knowledge) [13]. The continuum of tacit to explicit knowledge hints at a process in which tacit knowledge is converted or transformed into explicit knowledge. This movement of knowledge from tacit to explicit is where the domain of implicit knowledge exists. Organizational Learning (OL) literature defines implicit knowledge as that which results from the induction of an abstract representation of the structure that the stimulus environment displays, and this knowledge is acquired in the absence of conscious, reflective strategies to learn [18]. To place a knowledge management perspective on this definition, the tacit 0-7695-2268-8/05/$20.00 (C) 2005 IEEE Proceedings of the 38th Hawaii International Conference on System Sciences - 2005 2 knowledge of experts has unconsciously been made implicit. This implicitness enables the possibility of transforming what was originally tacit into explicit. The OL literature has researched implicit learning and provided support that ?implicit knowledge can be retained for longer periods than explicit knowledge? [19]. This means that, once tacit knowledge is made implicit, it resides with greater permanence within the human capital and therefore extends the time available for making that knowledge explicit. Implicit knowledge does not have extensive recognition within the MIS literature. However, the literature does imply the existence of implicit knowledge on a continuum from tacit to explicit, and recognizes that implicit knowledge is known to an expert and it can be elicited from the expert and documented [5]. More recently, it has been recognized that externalizing tacit knowledge into explicit knowledge means finding a way to express the ?inexpressible?. Herein resides the realm of implicit knowledge [13]. Avenues for transforming implicit knowledge to explicit knowledge must exist as integral parts of organizations? knowledge capability areas. Capability Areas The framework presented here provides a method to assess the overall capability level of an organization?s knowledge management initiatives within the four KCAs introduced earlier: Lessons Learned, Knowledge Documents, Expertise and Data. The KCAs support organizational knowledge use and affect the overall performance of the organization. We describe each KCA in terms of four elements: 1) the importance of the capability in prior research, 2) the interaction of human capital, 3) the tacit/implicit/explicit nature of the knowledge and 4) the knowledge lifecycle flow. Lessons Learned or best-known methods are defined as useful knowledge gained while completing tasks or projects. Lessons Learned, as internal benchmarking or best practice transfer, are identified as ?one of the most common applications (in KM)? [16]. Internal benchmarking is the process of identifying, sharing, and using the knowledge inside one?s own organization [20]. Lessons Learned are unique individual aspects of knowledge and their identification as best practices imply that they are highly tacit/implicit, singular and specific to situations. Although lessons may be unique and learned in specific circumstances, one can develop a process to facilitate the identification, capture and transfer of such lessons to other similar situations. Codified knowledge that can be described as having a long shelf life, originating from published sources and containing highly explicit knowledge define the capability area of Knowledge Documents. Knowledge Documents may be text-based forms that include: project reports, technical reports, research reports and publications. This ?field of information (codified knowledge) can include statistics, maps, procedures, analyses?? and can include alternative forms such as: pictures, drawings, diagrams, presentations, audio and video clips, on-line manuals, and tutorials [21]. While many of these sources of codified knowledge originate internally, ?knowledge sources may lie within or outside the firm? [22]. An organization?s human capital must recognize the explicit nature, the internal/external origin, and the referential usage of this knowledge source. The interaction with this knowledge source mainly occurs at the search and retrieval stage of the life cycle. Expertise is viewed as the knowledge that may be gained through experience or formal education. In many organizations, corporate directories map internal expertise. Identifying experts and classifying their areas of expertise such that their knowledge can be efficiently tapped is an active research area [16, 23]. Need for expertise initiates the transfer of this highly tacit form of knowledge. Experts are also a source of implicit knowledge that has the potential to be made explicit. Organizations must encourage the sharing of expertise between workers through one-on-one and group collaborations, as well as processes to transform experts? implicit knowledge to explicit knowledge where appropriate. Data provides many complementary benefits to the leveraging of other KCAs. Data can be transformed into decision- and action-relevant meaning. Databases and data warehouses containing aggregated or otherwise summarized historical information are the most basic form of knowledge management tools [6, 24]. The value of this highly explicit form of knowledge is dependent on various dimensions such as context, usefulness, and interpretation [16]. A common view holds that data is raw numbers, information is processed data and knowledge is authenticated information [25]. The dichotomous view reverses the data to knowledge assumption and states that knowledge must exist before information can be formulated and before data can be measured to form information [26]. It?s inclusion as a KCA is justified theoretically as part of the data/information/knowledge chain and practically due to the face validity of its actionable content. 3. Methodology & Results In the first phase of this research, we developed the Knowledge Management Capability Assessment (KMCA) instrument to measure various aspects of the four KCAs described above. We operationalize each KCA using a set of latent descriptor factors. The development of the 0-7695-2268-8/05/$20.00 (C) 2005 IEEE Proceedings of the 38th Hawaii International Conference on System Sciences - 2005 3 descriptor factors was guided collectively by the KCA?s involvement in the knowledge life cycle, its need for technological support, and its interaction with the human capital. For example, Lessons Learned is hypothesized to be composed of four descriptor factors: Repository, Taxonomy, Capture, and Application/Use. Table 1 ? KMCA Instrument - shows the originally hypothesized descriptors and an abbreviated version of the scale items. Table 1 also indicates which latent descriptors and scale items were dropped during the EFA as described later. We assembled a focus group of 12 individuals within the subject organization to evaluate the first version of the instrument. The feedback from the focus group resulted in clarifying the questions to ensure applicability to the target audience. We then surveyed a pilot group of 98 individuals and found that the questionnaire required about 45 minutes to complete. As a result, we shortened and substantially simplified the questionnaire. Another focus group ensured that the meaning of the questions remained intact. The resulting questionnaire consisted of approximately 130 questions and required about 20 minutes to complete. Two large business units, referred to as BU1 and BU2, were selected to undergo the assessment. One of the units, BU1, is responsible for internal material and product quality across the entire organization. The other unit, BU2, is responsible for development and sourcing of system software across all product lines of the organization. The two business units had substantially different functional responsibilities. Even though the two business units resided within a single organization, we believe the differences in their responsibilities, business goals and objectives provided us with some degree of external validity. Each member of the two business units received an introductory email concerning the administration of the survey, its potential impact on knowledge management Table 1 ? KMCA Instrument Expertise Expertise Repository Expertise Taxonomy er1 Availability of repository(ies) et1 Existence of taxonomy er2 Accessibility of repository(ies) et2 Clarity and standardization er3 Usefulness of repository content et3 Comprehensiveness er4 Information about internal & external experts et4 Extensibility er5 Search capabilities Collaboration Tools ** er6 Ease of searching ec1 Routineness of use er7 Multiple search criteria ec2 Ease of use Expert Access/Consulting ec3 Access to internal & external experts ea1 Practice of looking for available expertise ec4 Multiple tool set ea2 Ease of finding experts Communities of Practice ** ea3 Embedded in normal work practices es1 Participation in SIGs Expert Profiling & Registration es2 Encouragement for participation ep1 * Existence of a registering and profiling process es3 Availability of relevant SIGs ep2 Ease to use es4 Participation on company time ep3 Allows self-updating es5 Financial support for participation ep4 Managed for consistency Lessons Learned Lessons Learned Repository(ies) Taxonomy lr1 Availability of repository(ies) lt1 Existence of taxonomy lr2 Accessibility of repository(ies) lt2 Clarity and standardization lr3 Usefulness of repository content lt3 Comprehensiveness lr4 Search & retrieval capabilities Capture lr5 Ease of searching lc1 Practice of capture lr6 Multiple search criteria lc2 Consolidation and management Application/Use lc3 * Individual and group responsibilities la1 Practice of application/use lc4 Existence of a systematic processes la2 * Ease of finding relevant lessons la3 Embedded in normal work practices Knowledge Documents Knowledge Documents Repository(ies) Taxonomy ** kr1 Availability of repository(ies) kt1 * Existence of taxonomy kr2 Accessibility of repository(ies) kt2 * Clarity and standardization kr3 Usefulness of repository content kt3 * Comprehensiveness kr4 Access to internal & external documents Search & Retrieval kr5 Supports rich formats ks1 Ease to use kr6 Clarity of meta-data ks2 Effectiveness of retrieval system Categorization ks3 Multiple search criteria kc1 Existence of a categorization process Reference & Use ** kc2 Ease to use ku1 * Practice of reference/use kc3 Embedded in normal work practices ku2 * Ease of finding documents kc4 Managed to ensure adherence Data Data Repository(ies) Data Relevance dr1 Availability of repository(ies) dv1 Timeliness dr2 Accessibility of repository(ies) dv2 Periodicity dr3 Currency of data dv3 Completeness dr4 Level of detail/summarization dv4 Usefulness of format dr5 Clarity of meta-data dv5 Accuracy Decision Support Tools ds1 * Ease of use ds2 * Sufficiency KCA factors are in bold and underlined, Descriptor factors are in bold * - Dropped Scale Item for EFA, ** - Dropped Factor for CFA 0-7695-2268-8/05/$20.00 (C) 2005 IEEE Proceedings of the 38th Hawaii International Conference on System Sciences - 2005 4 and the importance of the survey. A senior level sponsor within each business unit initiated the email. A second email provided each member a link to the survey instrument and data was collected over a four-week period. Follow-up emails, as well as various incentives, boosted the participation rates of the two business units to 37% (223 useable responses) and 56% (303 useable responses), respectively. The responses were voluntary and therefore may have introduced some bias into the results. We tested for the potential response bias by polling a sample of non-respondents and also making a first & fourth quartile response comparison. Discriminant Analysis of these groups did not provide any evidence of a response bias. Exploratory Factor Analysis Unidimensionality is defined as the existence of one latent trait or construct underlying a set of measures [27]. In evaluating each KCA construct, we hypothesized each latent descriptor factor within an individual KCA to be a trait (first order or specific factor) for the set of measures. We conducted factor analysis in determining the valid descriptor factors within each KCA. For the factor analysis techniques, comparisons of Principal Component Analysis (PCA) and Maximum Likelihood factor analysis (ML) indicate that either is equally accurate for pattern reproduction [28]. However, we chose ML over PCA for the knowledge asset representations since PCA is designed as a data reduction technique that maximizes the extracted variance, whereas ML is designed to estimate factor loadings for populations that maximize the likelihood of sampling the observed correlation matrix [29, 30]. Through the use of ML, an iterative process was applied to each KCA to identify problem scale items for both business units. If a scale item loaded incorrectly on its latent descriptor factor, it was removed. We set four goals for the EFA for determining the inclusion or exclusion of scale items on a factor. The first goal was to retain a minimum of three variables per factor in order to ensure stability [28]. Comrey [31] recommends the following numbers be used as thresholds for determining the loading criteria: 0.71 ? excellent, 0.63 ? very good, 0.55 ? good and 0.45 ? fair. The second goal was that the removal of any scale item should improve the model?s Chi-square and Tucker/Lewis Index (TLI) fit indicators. The third goal was that if a scale item did not load on the hypothesized factor, it was to be removed rather than trying to provide a possible explanation for reorienting that item. This parsimony is especially critical when the removal improves the model?s fit indices. The fourth goal was to achieve a model that is significant for both business units and can guide the development of the structural equation models for confirmatory factor analysis. Summary results of the overall model for each capability area of the two business units (BU1 and BU2) are presented in Table 2 ? Exploratory Factor Analysis ML Results. We discuss each business unit?s scale item results with respect to each KCA, but provide illustrations and actual loadings only for one of the KCAs, Lessons Learned, due to space limitations. The specific final loadings for the Lessons Learned KCA are provided in Table 3 ? Lessons Learned Factors. Table 1 ? KMCA Instrument shows the originally hypothesized descriptor factors and an Table 3 - Lessons Learned Factors Repository Taxonomy Capture Use BU1 BU2 BU1 BU2 BU1 BU2 BU1 BU2 lr1 0.67 0.70 lr2 0.82 0.86 lr3 0.82 0.80 lr4 0.87 0.92 lr5 0.88 0.86 lr6 0.89 0.86 lt1 0.63 0.64 lt2 0.87 0.87 lt3 0.83 0.91 lc1 0.67 0.79 lc2 0.89 0.74 lc3 * * lc4 0.51 0.49 la1 0.83 0.84 la2 * * la3 0.84 0.87 Table 2 - Exploratory Factor Analysis ML Results Variance Explained Chi ? Square TLI Observations Capability Area Descriptor Factors BU1 BU2 BU1 BU2 BU1 BU2 BU1 BU2 Expertise Repository, Taxonomy, Access, Profiling, Collaboration, CoP''s 85% 78% 654 (p<.001) 614 (p<.001) 0.92 0.92 250 301 Lessons Learned Repository, Taxonomy, Use, Capture 84% 80% 139 (p<.001) 143 (p<.001) 0.95 0.95 243 290 Knowledge Documents Repository, Categorization, Use 88% 82% 140 (p<.001) 140 (p<.001) 0.96 0.96 228 283 Data Repository, Relevance 90% 87% 97 (p<.001) 123 (p<.001) 0.97 0.96 224 291 0-7695-2268-8/05/$20.00 (C) 2005 IEEE Proceedings of the 38th Hawaii International Conference on System Sciences - 2005 5 abbreviated version of the scale items. Table 1 also indicates which latent descriptor factors and scale items were dropped during the EFA. The Lessons Learned EFA agreed with the four hypothesized factors. As can be seen from Table 3, most of the scale items had excellent loadings on their respective factors. Two scale items from different factors loaded improperly. These two scale items were lc3 (which did not load on BU1) and la2 (which did not load on BU2). Iterative removal of each scale item (la2, then lc3) improved both the Chi-square statistic and TLI. The descriptor variable Application & Use resulted in only two scale items after the removal of la2. However, both these scale items had excellent loadings and therefore this was not deemed to be a major drawback. In the case of Expertise, ML returned six identifiable factors as was originally hypothesized. The only scale item (of the twenty-seven) that loaded improperly was ep1. Instead of loading on Expert Profiling & Registration, ep1 had the highest loading on Repository. Removal of this scale item increased the TLI for both business units. All descriptor variables for Expertise met our criteria of at least three scale items per factor. Five of the six factors had at least three scale items with excellent loadings for each business unit. The sixth factor, Expert Access, had two items with excellent loading for both business units and the third item with a very good loading (BU1) and a good loading (BU2). Knowledge Document was hypothesized to be composed of five factors. However, the scale items hypothesized for Taxonomy and Reference & Use did not load on a separate factor. Pursuant to our second and third goals, each of the scale items for Taxonomy (kt1, kt2 & kt3) and Reference & Use (ku1 & ku2) were iteratively removed. The TLI improved with each iteration and a more parsimonious three-factor model emerged. Each of the three factors had at least three scale items with excellent loadings. The fourth KCA, Data, hypothesized three factors. The descriptor variable Decision Support Tools and the associated scale items (ds1 and ds2) encountered commonalities greater than one while running the ML and would not complete computation in SAS. Iteratively removing these two scale items resulted in convergence to two factors for Data. Knowledge Capability Area SEM Models An overall KCA factor was hypothesized for each capability area. Second Order and General-Specific models were constructed to perform the CFA (CFA) for the overall KCA factor. In the Second Order model, the descriptors within each capability area correspond to first order factors. In the case of the General-Specific model, the descriptors correspond to specific factors. Although these two models are not mathematically equivalent, the two models provide similar interpretations and represent the factor of the capability area under investigation [32, 33]. The main difference between the two models is that, the Second Order model evaluates the influence of the second order factor (e.g., the overall KCA) on the first order factors (descriptors), whereas the General-Specific model evaluates the influence of the general factor (KCA) directly on the scale items. In both models, the descriptors are considered to be orthogonal. We used LISREL 8.54 for the CFA (CFA) in the investigation of all sixteen measurement models (4 KCAs * 2 Models * 2 business units). The final measurement model results for each capability area are presented in Table 4 ? Confirmatory Factor Analysis Results. Path diagrams for the Lessons Learned KCA representing the two measurement models for BU1 are in Figure 1 ? BU1 ? Second Order model and Figure 2 ? BU1 ? General Specific model. For confirming the factors within each capability area, we began with a model that replicated the initial Table 4 - Confirmatory Factor Analysis Results Capability Area Model Type Group N df ?2 NNFI CFI SRMR Lessons Learned Second Order BU1 223 100 465 0.95 0.96 0.093 Lessons Learned Second Order BU2 303 100 629 0.94 0.95 0.120 Lessons Learned General Specific BU1 223 88 355 0.96 0.97 0.069 Lessons Learned General Specific BU2 303 88 394 0.96 0.97 0.093 Data Second Order BU1 223 51 183 0.98 0.98 0.032 Data Second Order BU2 303 51 198 0.98 0.99 0.034 Data General Specific BU1 223 43 136 0.98 0.99 0.048 Data General Specific BU2 303 43 125 0.99 0.99 0.036 Expertise Second Order BU1 223 131 578 0.96 0.97 0.076 Expertise Second Order BU2 303 131 477 0.97 0.98 0.066 Expertise General Specific BU1 223 117 461 0.97 0.97 0.034 Expertise General Specific BU2 303 117 391 0.98 0.98 0.030 Knowledge Documents Second Order BU1 223 62 263 0.97 0.98 0.052 Knowledge Documents Second Order BU2 303 62 241 0.98 0.98 0.043 Knowledge Documents General Specific BU1 223 52 193 0.97 0.98 0.030 Knowledge Documents General Specific BU2 303 52 201 0.98 0.99 0.024 0-7695-2268-8/05/$20.00 (C) 2005 IEEE Proceedings of the 38th Hawaii International Conference on System Sciences - 2005 6 instrument and included all items and their loading on the hypothesized descriptors (Table 1). This initial confirmation ignores the results of the EFA in order to test the adequacy of the initially hypothesized factors within each KCA. If adequate Second Order and General Specific models are achieved, these are considered the final models. We achieved significant results for this initial confirmation in the capability areas of Lessons Learned and Data. For Knowledge Documents, the hypothesized five-factors would not converge using both the CFA models. We utilized the EFA results to run a three-factor CFA model which achieved a good model fit. Expertise encountered similar problems with convergence which prompted us to compare the descriptor factors with the other KCAs. We concluded that although Collaboration Tools and Special Interest Groups within the capability area of Expertise were confirmed to be latent factors in the EFA analysis, these factors had no corresponding counterparts in the other capability areas. We removed these factors and ran a four-factor model for Expertise. The results in Table 4 represent these final measurement models. The fit indices in Table 4 provide four tests indicating the adequacy of fit for each KCA factor. The overall KCA is represented as a Second Order factor and a General factor. These two representations were each replicated, with similar goodness of fit, in the two business units. NNFI and CFI above a threshold of 0.90 is considered to indicate a good fit for the model. Also, SRMR below the threshold of 0.08 is considered a good fit for the model. As can be observed in Table 4, all models for each business unit represent a good fit and thus validate the KCA constructs. The relationship between the General/Second Order factors and the Specific/First Order factors provides insight into the explanatory power of each model. For brevity, we present the detailed analysis of only the Lessons Learned models (Figure 1 & 2) for one of the business units (BU1). Similar detailed analyses were performed for each business unit, but are not presented here for space considerations. The detailed results may be requested from the authors. Reviewing the Second Order model (Figure 1), we see that for each of the descriptor variable constructs, the loadings for each item are excellent based on the stated loading criteria (0.71 or better). This provides the requisite construct validation (in addition to the EFA), i.e. the instrument has measured the intended separate latent concepts. Each path between the first order factors and the second order factor is significant indicating the influence of the overall KCA on each of its first order descriptor factors. While reviewing the General-Specific model (Figure 2), each item is posited to load on both the General factor and a single Specific factor. In the case of the General- Specific model, the loading criterion is not followed as .83*** .86*** .77*** .68*** Lessons Learned Taxonomy Capture Application/Use Repository lr1.38*** lr2.17*** lr3.14*** lr4.05*** lr5.03*** .79*** .91*** .93*** .98*** .98*** lr6.07*** .96*** lt1.29*** lt2.01** lt3.02*** .99*** .84*** .99*** lc1.22*** lc2.22*** lc3.50*** lc4.43*** .88*** .89*** .71*** .75*** la1.34*** la2.37*** la3.12*** .79*** .81*** .94*** N = 223 df = 100 Chi-Square = 468 NNFI = .95 CFI = .96 SRMR = .093 Figure 1 - BU1 - 2nd Order Model .33*** .71*** .84*** .83*** Lessons Learned Taxonomy Capture Application/Use Repository lr1 lr2 lr3 lr4 lr5 .18*** .39*** .40*** .51*** .50*** lr6 .55*** lt1 lt2 lt3 .61*** .28*** .56*** lc1 lc2 lc3 lc4 .52*** .65*** .51*** .42*** la1 la2 .40*** .67*** .71*** Figure 2 - BU1 - General Specific Model .03*** .06*** .01 .23*** .04*** .11*** .24*** .49*** .46*** .16*** .31*** .14*** .05*** .10** .32*** .62*** .81*** .83*** .85*** .80*** .80*** .83*** .81*** .65*** .70*** .61*** .48*** .50*** N = 223 df = 88 Chi-Square = 355 NNFI = .96 CFI = .97 SRMR = .069 la3 0-7695-2268-8/05/$20.00 (C) 2005 IEEE Proceedings of the 38th Hawaii International Conference on System Sciences - 2005 7 previously stated. Instead, the significance of the loading coefficients has greater meaning since each scale item is hypothesized to load on both a General and a Specific factor. For Lessons Learned (Figure 2), the loadings on the General factor (Lessons Learned) are all significant which indicates the influence of the general factor on all scale items. The scale item loadings for the Specific factors (Repository, Taxonomy, Capture and Application/Use) are significant even though these loadings vary from a low of 0.18 to a high of 0.71. These loadings represent the additional influence and explanation of variance that the Specific factors have on each of the scale items, above and beyond the influence of the General factor [33]. The fact that all items are also significant on the Specific factors provides additional evidence of construct validity of the measurement model. Since the Second Order model is a more restricted model and has been demonstrated to be a nested version of the General-Specific model [34], the Chi-Square difference test indicates whether the two models are significantly different in their representation of the capability area. The form of the Chi-Square difference test statistic and its value for the Lessons Learned KCA is represented as: ?2? = (?22nd ? ?2GS) = (465 ? 355) = 110, df ? = (df2nd ? dfGS) = (100 ? 88) = 12, where: ?2? is the difference between the chi- squared statistics and df ? is the difference between the degrees of freedom of the Second Order model and the General-Specific model respectively. Significance of the Chi-Square test is determined by consulting a Chi-Square table utilizing the resulting chi- square and degrees of freedom values. The Chi-Square test results for BU1 for the Lessons Learned KCA indicate that the two models are significantly different (at p<.001). A review of the fit indices indicates that, although both models indicate the existence of Lessons Learned as a valid KCA, the General-Specific model represents the Lessons Learned capability area more accurately than the Second Order model. This means that, for any investigations involving Lessons Learned as an overall factor (e.g. relationships between Lessons Learned and firm performance), the General-Specific model would be a better choice for representing this KCA. On the other hand, if the theory involves descriptor variables within Lessons Learned, then the Second Order model provides an adequate representation for testing the hypotheses. The Chi-Square difference test and comparison of model fit indices provided similar results for each capability area for both business units. This indicates that the structure of the KCAs and the descriptor variables are consistent across all the capability areas and that both models may be used depending on the theoretical situations to which they are applied. 4. Conclusions and Limitations In the process of establishing capability areas as knowledge assets, we have focused our efforts on establishing measurement consistency and the representation of each capability area as a latent factor. Each capability area was established using the two measurement model forms of: 1) a General-Specific SEM model and 2) a Second-Order SEM model. Both models provided fit indices for all capability areas indicating models of good fit. The significance of the General factor and the Second Order factor representing the overall capability area provides strong evidence supporting these knowledge assets as measurable capabilities. This evidence is further strengthened due to the application of the models to two independent business units in order to confirm the measurability of the capabilities as knowledge assets. By using two measurement models within two business units, we have provided experimental rigor and external validity. While we have demonstrated the measurability of each knowledge asset, we recognize that these results may be limited by the fact that the data originated from a single organization. This limitation needs to be evaluated in light of the vastly different corporate directives and the autonomous nature of the two business units. One must also recognize that while the identification of four capability areas represents an attempt at enumerating diverse knowledge assets within most organizations, these areas may not represent all that is considered as knowledge by every organization. KM is an evolving field and the definition of what is knowledge depends on an organization?s strategic and operational needs. For example, an organization may require that Lessons Learned and Knowledge Documents be considered a single KCA (e.g. unstructured explicit knowledge). Similarly, descriptor variables that we have identified may not represent all the components within each KCA. Depending on the need for operationalizing a specific KCA, an organization may emphasize different aspects of the KCA. 5. Implications and Future Research Immediate implications for managers choosing to utilize the KMCA are in the realization that a method has been provided to assess the capability level for these four knowledge asset areas. Through the measurement of these knowledge assets, the recognition that a specific knowledge asset is low in comparison to both its relative position with regards to other knowledge assets and organizational strategic goals, will allow KM initiatives to 0-7695-2268-8/05/$20.00 (C) 2005 IEEE Proceedings of the 38th Hawaii International Conference on System Sciences - 2005 8 be targeted to that specific KCA. The ability to improve targeted KCA?s that are deficient in both relative position to other KCA?s as well as achieving identified organizational goals ensures that scarce resources are efficiently utilized. As an example of a discrepancy in relative position, an organization may recognize a need for contacting of Experts and using relevant Expertise as a knowledge asset. The Expertise KCA may not be understood or exploited as well within an organization as say, documented knowledge that may be systematically maintained and utilized widely across the organization [35]. If this deficiency/discrepancy is identified and the need recognized, an organization may focus on improving the sharing of relevant expertise to augment the high capability in Knowledge Documents. Another organization may identify a high need for efficiency in the knowledge asset of Lessons Learned but a lower need for utilizing raw Data. The organization?s strategic positioning may drive these differences and the KMCA can identify the relative position of these capabilities. Recognizing that the organization?s knowledge asset capabilities are at appropriate levels with respect to its business goals can assist in directing resources to initiatives that provide the greatest return. Potential business implications of measuring knowledge asset capabilities reside in the ability to tie knowledge management to recognized value metrics, construct targeted knowledge sharing improvements and match organizational/business unit goals to the need for specific knowledge assets. Value metrics that knowledge assets causally affect may range from such soft measures as user satisfaction, perceived usefulness of knowledge, use of knowledge systems and quality of knowledge to hard performance metrics that include time to obtain needed knowledge, knowledge access frequency, and personal job productivity measures. Current research is underway to investigate the relationship of the knowledge assets to each of these value metrics. As an initial avenue of future research, the nature of the interaction between knowledge and human capital implies a potential influence of the organization?s culture with respect to knowledge management and knowledge sharing. While the capability measurement models are considered adequate without taking a cultural metric into account, an organization?s culture may influence a causal relationship to the value achieved from these knowledge asset. Factors, such as, the leadership?s commitment to knowledge sharing, rewards and incentive systems for promoting knowledge sharing behavior, attitudes of co- workers, and importance placed on training while introducing new KM systems are all important aspects to be considered while investigating the causal relationships of KCAs to value indicating metrics of an organization. The research implications are significant. A standardized instrument for measuring capabilities in knowledge areas would not only allow benchmarking, but also allow tracking capabilities over time and linking them to those performance metrics that are deemed appropriate by the organization. The application of the KMCA instrument to multiple organizations will improve the external validation of the KCA measurement models and the identification of knowledge assets across organizations. Within the current organization, KM improvements have been initiated and a longitudinal study is in progress to validate the predictive ability of the KMCA instrument and the level of impact of each KCA to proposed value metrics. 6. References [1] M. H. Zack, "Managing codified knowledge," Sloan Management Review, vol. 40, pp. 45-+, 1999. [2] D. J. Teece, "Research directions for knowledge management," California Management Review, vol. 40, pp. 289- +, 1998. [3] A. H. Gold, A. Malhotra, and A. H. Segars, "Knowledge management: An organizational capabilities perspective," Journal of Management Information Systems, vol. 18, pp. 185-214, 2001. [4] W. R. King, P. V. Marks, and S. McCoy, "The most important issues in knowledge management," Communications of the Acm, vol. 45, pp. 93-97, 2002. [5] U. Harigopal and A. Satyadas, "Cognizant enterprise maturity model (CEMM)," Ieee Transactions on Systems Man and Cybernetics Part C-Applications and Reviews, vol. 31, pp. 449-459, 2001. [6] L. Fahey and L. Prusak, "The eleven deadliest sins of knowledge management," California Management Review, vol. 40, pp. 265-+, 1998. [7] S. A. Carlsson, O. A. El Sawy, I. Driksson, and A. Raven, "Gaining Competitive Advantge Through Shared Knowledge Creation: In Search of a New Design Theory for Strategic Information Systems," presented at European Conference on Information Systems, Lisbon, 1996. [8] J. B. Barney, "Firm resources and sustained competitive advantage," Journal of Management, vol. 17, pp. 99-120, 1991. [9] R. W. Coff, "Human assets and management dilemmas: coping with hazards on theroad to resource-based theory," Academic Management Review, vol. 22, pp. 374-402, 1997. [10] T. H. Davenport, R. J. Thomas, and S. Cantrell, "The mysterious art and science of knowledge-worker performance," Sloan Management Review, vol. 44, pp. 23-30, 2002. [11] J. Birkinshaw and T. Sheehan, "Managing the knowledge life cycle," Mit Sloan Management Review, vol. 44, pp. 75-83, 2002. [12] A. Satyadas, U. Harigopal, and N. P. Cassaigne, "Knowledge management tutorial: An editorial overview," Ieee Transactions on Systems Man and Cybernetics Part C- Applications and Reviews, vol. 31, pp. 429-437, 2001. 0-7695-2268-8/05/$20.00 (C) 2005 IEEE Proceedings of the 38th Hawaii International Conference on System Sciences - 2005 9 [13] K. U. Koskinen, "Evaluation of tacit knowledge utilization in work units," Journal of Knowledge Management, vol. 7, pp. 67-81, 2003. [14] D. J. Teece, "Capturing value from knowledge assets: The new economy, markets for know-how, and intangible assets," California Management Review, vol. 40, pp. 55-+, 1998. [15] I. Nonaka, "A Dynamic Theory of Organizational Knowledge Creation," Organization Science, vol. 5, pp. 14-37, 1994. [16] M. Alavi and D. Leidner, "Review: Knowledge Management and Knowledge Management Systems: Conceptual Foundations and Research Issues," MIS Quarterly, vol. 25, pp. 1-7-136, 2001. [17] M. Polanyi, Personal Knowledge. Chicago: University of Chicago Press, 1975. [18] A. S. Reber, "Implicit Learning and Tacit Knowledge," Journal of Experimental Psychology-General, vol. 118, pp. 219-235, 1989. [19] R. J. Tunney, "Implicit and explicit knowledge decay at different rates: A dissociation between priming and recognition in artificial grammar learning," Experimental Psychology, vol. 50, pp. 124-130, 2003. [20] C. O''Dell and C. J. Grayson, "If only we knew what we know: Identification and transfer of internal best practices," California Management Review, vol. 40, pp. 154-+, 1998. [21] R. McDermott, "Why information technology inspired but cannot deliver knowledge management," California Management Review, vol. 41, pp. 103-+, 1999. [22] M. H. Zack, "Developing a knowledge strategy," California Management Review, vol. 41, pp. 125-+, 1999. [23] K. J. Dooley, S. R. Corman, and R. D. McPhee, "A Knowledge Directory for Identifying Experts and Areas of Expertise," Human Systems Management, vol. 21, pp. 217-228, 2002. [24] J. S. Brown and P. Duguid, "Balancing act: How to capture knowledge without killing it," Harvard Business Review, vol. 78, pp. 73-+, 2000. [25] D. M. Vance, "Information, Knowledge and Wisdom: The Epistemic Hierarchy and Computer-Based Information System," presented at Americas Conference on Information Systems, Indianapolis, In, 1997. [26] I. Tuomi, "Data is more than knowledge: Implications of the reversed Hierarchy for knowledge managment and organizational memory," presented at Proceedings of the Thirty- Second Hawaii International Conference on Systems Sciences, Hawaii, 1999. [27] J. C. Anderson, D. W. Gerbing, and J. E. Hunter, "On the Assessment of Unidimensional Measurement - Internal and External Consistency, and Overall Consistency Criteria," Journal of Marketing Research, vol. 24, pp. 432-437, 1987. [28] W. F. Velicer and J. L. Fava, "Effects of variable and subject sampling on factor pattern recovery," Psychological Methods, vol. 3, pp. 231-251, 1998. [29] B. G. Tabachnick and L. S. Fidell, Using Multivariate Statistics, Fourth ed: Allyn and Bacon, 2001. [30] L. R. Fabrigar, D. T. Wegener, R. C. MacCallum, and E. J. Strahan, "Evaluating the use of exploratory factor analysis in psychological research," Psychological Methods, vol. 4, pp. 272-299, 1999. [31] A. L. Comrey and H. B. Lee, A First Course in Factor Analysis, 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers, 1992. [32] J. Gustafsson and G. Balke, "General and Specific Abilities as Predictors of School Achievement," Multivariate Behavioral Research, vol. 28, pp. 407-434, 1993. [33] F. F. Chen, S. G. West, and K. H. Sousa, "A Comparison of General-Specific and Second-Order Models of Quality of Life," Structural Equation Modeling, vol. in press, 2004. [34] Y. F. Yung, D. Thissen, and L. D. McLeod, "On the relationship between the higher-order factor model and the hierarchical factor model," Psychometrika, vol. 64, pp. 113-128, 1999. [35] M. T. Hansen, N. Nohria, and T. Tierney, "What''s your strategy for managing knowledge?," Harvard Business Review, vol. 77, pp. 106-+, 1999. 0-7695-2268-8/05/$20.00 (C) 2005 IEEE Proceedings of the 38th Hawaii International Conference on System Sciences - 2005 10 Comments
|
