A demonstration of advantages & limitations

Computerized content analysis in management research: a demonstration of advantages & limitations

Rebecca Morris

sugar-coat bad news. The Wall Street Journal, March 31: 31.

Rosenberg, S.D., Schnurr, P.P. & Oxman, T.E. (1990). Content analysis: A comparison of manual and computerized systems. Journal of Personality Assessment, 54(1 & 2): 298-310.

Salancik, G.R. & Meindl, J.R. (1984). Corporate attributions as strategic illusions of management control. Administrative Science Quarterly, 29: 238-254.

Saris-Gallhofer, I.N., Saris, W.E. & Morton, E.L. (1978). A validation study of Holsti’s content analysis procedure. Quality and Quantity, 12: 131-145. Content analysis is a qualitative research technique that uses a set of procedures to classify or categorize communications to permit valid inferences to be drawn (Weber, 1990). Although content analysis may be used to extract data from a wide range of communications media (writing, speeches, movies, radio and television, for example), management research has utilized the technique primarily to draw valid inferences from the textual communications of managers (Cochran & David, 1986; Pearce & David, 1987; David, 1989b; D’Aveni & MacMillan, 1990). As such, content analysis provides researchers with opportunities to unobtrusively study the values, sentiments, intentions, and ideologies of managers generally inaccessible to researchers. This has the potential to be particularly advantageous in strategic management research as access to a large number of top level executives has been difficult to obtain. Additionally, content analysis can be beneficial to management research in general by providing researchers with a methodology for systematic analysis of the information contained in corporate documents, thus opening a rich data source that has been often neglected.

In the social sciences, content analysis is used to accomplish the following objectives:

1. To make inferences about the values, sentiments, intentions or ideologies of the sources or authors of the communications.

2. To infer group or societal values through the content of communications.

3. To evaluate the effects of communications on the audiences they reach. (Williamson, Karp & Dalphin, 1977, p. 291).

As these goals correspond with the focus of many areas of management research, content analysis methodologies may provide an effective tool for gaining access to desired information which does not suffer from tendencies of respondents to answer questions in socially desirable ways, to fail to adequately remember past events or to attribute more rational thought processes to past decisions.

Computer technology has been used in content analysis research in other disciplines such as psychology and sociology to eliminate coding errors and to enable analysis of large volumes of written communication since 1966 (Stone, 1966). By formalizing coding rules through the creation of computer content-coding schemes, perfect coding reliability is obtained (Weber, 1990). The introduction of inexpensive and powerful personal computers, cost effective optical character reading (OCR) devices for converting written text into machine-readable files, the availability of text information in CD-Roms and text management software makes computerized content analysis more accessible to researchers than ever before.

This paper provides a demonstration of the use of computerized content analysis in management research by comparing the analysis of human coders to computerized coding of the same text communications. This comparison will permit a test of whether computerized coding rules can be created that will adequately replicate human judgments regarding the classification of the content of documents. The paper further examines the impact of the unit of analysis (by word, sentence, paragraph or whole document) on the classification of documents. Are the same conclusions reached regardless of the unit of analysis selected? In addition, the advantages and limitations of computerized content analysis and their implications for future management research are discussed.

Content Analysis Methodologies

Deffner (1986) classified content analysis methods into three broad types: (1) human scored schema; (2) individual-word-count systems (usually computerized); and (3) computerized systems using artificial intelligence. Each of these methods will be briefly discussed in the following sections.

Human-scored schema. Steps for this approach are shown as Figure 1. In the human-scored approach, coders are trained to classify the text according to specific classification categories. The validity and reliability of text classification is achieved by demonstrating the extent to which multiple coders code the text in the same way. An iterative process of coding sample text, comparing classifications of multiple coders and revising coding rules is typically used to improve the reliability of the human coders. In disciplines such as psychology where content analysis is a more frequently utilized research technique, specialized coding systems such as the Gottschalk-Gleser method (1969) enable consistent classification of text by coders trained in the method. No classification methods or schema have yet been created for the classification of management variables, thus requiring researchers to create their own coding systems and limiting the cross-comparability of research studies.

Individual-word-count systems. Individual-word-count systems classify the text by assigning words to pre-specified semantically equivalent categories. For example, words such as worker, researcher, associate, staff or human resources are all classified as relating to employees. Frequency counts or occurrences of words in each category are then analyzed to determine the relative concern given each category by the text’s author (Weber, 1990). Although this type of analysis can also be performed by human coders, simple computerized coding systems of this type are considered superior due to their near-perfect reliability and cost effectiveness (Rosenberg, Schnurr and Oxman, 1990).

Artificial intelligence systems. Computerized content analysis via artificial intelligence approaches differs from individual-word-count systems primarily in the way in which problems with the classification of words that have more than one meaning are resolved. Computerized systems incorporating artificial intelligence features attempt to consider both the syntax and lexicon of words (Rosenberg et al., 1990). Programs in this category such as General Inquirer with the Harvard IV Psychosociological Dictionary (Kelly & Stone, 1975) incorporate rules that distinguish among the various meanings of the word according to the context of its usage. The expectation is that by increasing the precision of text classification, these systems are more accurate in coding the symbolic meaning of the text (Weber, 1990). A comparative analysis of the classification performance of human-scored systems with more simple computerized individual-word-count techniques and the context sensitive computerized approach did not support this expectation (Rosenberg et al., 1990). Although both computerized methods were more accurate than human scoring, there was no significant difference between the classification accuracy of the computerized approaches.

Reliability and validity issues. If valid inferences about the symbolic content of the message are to be drawn, the content analysis classification scheme must be reliable in terms of consistency and reproducibility (Weber, 1990). Reliability problems in text classification are typically due to the ambiguity of word meanings, category definitions or other coding rules. Studies utilizing human raters or coders as described above, have relied upon multiple coders to deal with reliability concerns by permitting a quantitative assessment of the interrater reliability achieved. Although the use of multiple coders may provide an acceptable solution to reliability issues, the costs involved (in time, tediousness and perhaps monetary compensation) may result in sacrifices to research design and rigor. For example, the validity of the content analysis results rests in part upon the qualifications and expertise of the coders. Because a content analysis of a sufficiently large sample may take considerable time (D’Aveni and MacMillan’s (1990) six coders coded more than 20,000 sentences from 490 letters to shareholders), the coders are often masters degree or doctoral students. Although the researchers have taken considerable care to prepare and train the student coders, their inexperience and lack of expertise in the subject area might lead one to question their ability to make the fine distinctions required to unambiguously categorize the message contents. Additionally, human coders have difficulties in remembering complex coding rules and applying them consistently to all documents to be coded.

In addition to reliability, the classification scheme must be valid–in terms of the extent to which the variables that result from it are correlated with some other measure of the same construct (construct validity), the extent to which it “behaves” as it is supposed to in relation to other variables (hypothesis validity) or the extent to which the categorization scheme appears to measure the construct it is intended to measure (face validity) (Weber, 1990, pp. 18-21). In content analysis, semantic validity or the extent to which persons familiar with the language and texts agree that the list of words placed in the category have similar meanings or connotations (Krippendorff, 1980:159ff), must also be demonstrated. The validity of the human-coded content analysis schemes has primarily rested on the establishment of semantic validity through the use of multiple coders. The extent to which the coders agree on the categorization of the text is presumed to provide an indication that the process used in the categorization is valid. Establishment of stronger forms of validity such as construct validity is difficult in some areas of research as often no good alternative measures exist for the construct in question (perhaps due to the newness of the field and divergent research foci).

Validity problems are created by the attempt to retrieve concepts through the use of imperfect surrogates for those concepts–words or strings of words (Pfaffenberger, 1988, p. 40). Because concepts can be represented by many different words and words have different meanings in different contexts, valid content analysis schemes must incorporate rules which specify the pertinent connotations for the context under investigation (Adams, 1979, p. 374). For example, the coding instructions used by D’Aveni and MacMillan (1990) require almost three pages to define what is meant by the category “customer” in the context of their study. The definition proceeds from words commonly associated with this concept such as purchaser, consumer, and buyer to a detailed description of when the words “market” or “industry” might be considered to refer to customers.

Content Analysis in Strategic Management Research

Content analysis has been used in the strategic management literature to analyze the content (Pearce & David, 1987; David, 1989b), readability and tone (Cochran & David, 1986) of corporate mission statements. In these studies, panels of independent coders evaluated the mission statements to determine whether they contained desirable mission statement attributes as identified in a review of the literature. Pearce and David (1987) found a positive association between the inclusion of desirable mission components and a firm’s financial performance (measured by profit margin). Cochran and David (1986) found that less than half of the mission statements in their sample were written in a tone that was inspiring and motivating. The content analysis methodology thus enabled the researchers in these studies to systematically examine the messages communicated to organizational stakeholders via the mission statement document.

D’Aveni and MacMillan (1990) used content analysis of corporate annual reports to compare the managerial attention to environmental factors in bankrupt firms with a matched sample of surviving firms. Panels of MBA and business doctoral students coded each sentence in the letters to the stockholders with regard to their reference to a particular element of the environment such as owners, customers, general economic conditions, etc. The study results indicated that managers of surviving firms focused more on external environmental factors than did managers of bankrupt firms. Lack of responsiveness to externally induced crisis was suggested as a partial explanation for firm’s failure to survive. Content analysis provided an effective means for unobtrusively analyzing the perceptual focus of strategic managers and yielded important insights in the shifts in their perceptions over time in response to crisis.

In all these studies, panels of human coders evaluated texts according to a categorization scheme derived from a review of the literature. The coders were trained to classify the content of each sentence (D’Aveni & MacMillan, 1990) or the complete mission statement (Cochran & David, 1986; David, 1989b; Pearce & David, 1987) using researcher-specified coding rules. The reliability of the coding procedure was established by including an assessment of an overlapping sample of the text materials by multiple coders.

Computerized content analysis using the individual-word-count or artificial intelligence approaches has not been observed in the strategic management literature to date. This may be due to the emphasis on psychology and sociology variables in the classification schemes utilized by programs such as the General Inquirer III system (as documented in Zuell, Weber & Mohler, 1989) or the Oxford Concordance Program (as documented in Hockey & Martin, 1987). Until recently, few software programs were available that would permit users to easily define their own classification schema for content analysis. A new class of personal computer software called text retrievers (Tesch, 1990) or text management software provides powerful, readily accessible alternatives for analysis using user created classification schemes. This paper utilizes two text management software programs to conduct a content analysis of corporate reports similar to the analysis in Pearce and David (1987) and David (1989b).

Text Management Software

Text management systems do for text what database management systems do for numeric data–index the material to permit rapid location of specific strings of text in large text files. Technically, the text manager software builds a complete index containing every meaningful word in the full text. Users can typically search the index using single words, combinations or lists of words, and substrings such as by a suffix or stem utilizing Boolean operators such as AND, OR, BUT, and NOT. The result of this search is a listing of every file that contains the word or phrase and the number of times the item appears in each file. Typically, the software then permits the user to access each of the files to display the text and the highlighted search term in context.

A 1990 review of text management software in the Data Based Advisor described twenty-eight different products providing a range of speed, text indexing and text retrieval capabilities (Perez, 1990). Although not specifically marketed to perform content analysis, many of these programs provide users with the necessary tools to accomplish individual-word-count methodologies and to create key-word-in-context (KWIC) concordances of words and phrases. KWIC concordances are useful in refining categorization schema and for intensive study of a few specific symbols or messages (Weber, 1990). Some programs also provide capabilities for creating word-frequency lists. These lists also permit researchers to draw conclusions about the concerns of the author of the text as it is assumed that more frequently used words or phrases reflect a higher concern for a particular theme (Weber, 1990).

Two such programs were utilized in this study–ZylNDEX (Information Dimensions, 1992) and The Text Collector (O’Neill Software, 1987).(1) ZyINDEX is a personal computer-based text management software program that provides sufficient indexing, retrieval and text manipulation procedures for effective content analysis methodologies. The Text Collector’s automatic and interactive search capabilities allow users to search by sentences, lines, paragraphs or whole documents to collect references to user-defined categories. The content analysis capabilities of these two programs will be shown by conducting an analysis of corporate reports for elements typically associated with corporate mission statements.

Method

Description of components. As described earlier, the contents of corporate mission statements have been analyzed in previous research to determine the organizational concern for each of eight components: customers, geographic domain, products or services, technology, corporate philosophy, self-concept, public image, and concern for survival, growth and profitability (Pearce, 1982). The methodology used by Pearce and David (1987) and others doing research in this area has been to use human coders to indicate whether TABULAR DATA OMITTED a mission statement includes a given component. A “1” was recorded if the coder believed the component to be communicated in the full text of the mission statement. A “0” was recorded if the component was not included.

Steps of the analysis. The analysis described in this section proceeds in three steps. First, the creation and testing of the computerized content analysis scheme is discussed. Secondly, the validation of the computerized content analysis program is presented. The validation and reliability assessment process includes a comparison of the classifications of the computerized content analysis program with those of human coders classifying the same documents. Lastly, an investigation of different units of analysis (by word, paragraph, whole document, etc.) is presented. A summary of these analyses, descriptions of the data sets and number of documents analyzed is provided in Table 1.

Creation and Testing of Computerized Content Analysis Scheme

The steps outlined in Weber (1990) for creating and testing a coding scheme were used to develop the computerized scheme for ZyINDEX and the Text Collector. First, the word sense approach was chosen as the basic unit for text analysis in the computerized program. The word sense approach classifies each word or phrases designated by the programmer as a semantic unit (such as “competitive advantage”). The classification of phrases was expected to improve the reliability of coding by providing a more context-sensitive analysis of the text. Although the word sense approach was used for sensing or detecting the presence or absence of text matching a specified category definition, the computerized capabilities of ZyINDEX and Text Collector permit the user to determine whether these matches are aggregated by sentence, paragraph, whole document or some other text block.

Next, category definitions were created. The eight mission statement elements outlined in Pearce (1982) were identified as categories for analysis. A review of the mission statement literature and many corporate mission statements (excluding those used in this study) produced a list of words and phrases commonly used to refer to each of the eight categories. This list was further expanded upon consulting a thesaurus. Once a satisfactory list was obtained, the next step was to enter the words and phrases for each category as a search concept. Concepts allow users to enter one word in a search command to represent a list of words. Sample ZyINDEX concepts and conceptual definitions for the mission element categories are shown as Figure 2. Boolean operators such as “OR” separate one semantic unit or phrase from the next. Note the use of wild cards (the asterisk) to permit classification of variations of a word with the same prefix (for example, capabilit*classifies both capability and capabilities). The positional operator (w/n) classifies words that appear within n words in either direction of the first word. For example, “first w/4 industry or market” requires that the words “industry” or “market” must be within four words of the word “first” to indicate concern for self-image. The positional operator thus ensures that the words are contextually related.

The coding rules as specified in the concepts were then tested on a sample of abbreviated mission statements obtained from David (1989a). The results TABULAR DATA OMITTED of the computerized analysis were compared to those established by David (1989a) and the concepts were revised to remove ambiguities and to improve the coding accuracy. The mission statements used for testing were not used in the subsequent analysis. Additional testing and refinements were made by using the computerized content analysis programs to classify mission statements obtained from other sources. The results were then compared with the researcher’s own evaluation of the mission statements and refinements to the classification scheme were made as needed.

Differences in the search algorithms of the two software programs required additional refinements to the programs to ensure that they were evaluating the mission statements in a similar fashion. For example, ZyINDEX does not include plural forms of words unless wild card characters are used (employee* retrieves employee and employees) where the Text Collector does not require wild cards. The Text Collector’s retrieval of all forms of a root word necessitated the use of the Boolean “NOT” to prevent undesired retrievals such as retrieval of the word “productivity” when searching for references to “product”.

The mission documents were then analyzed using both ZyINDEX and the Text Collector. The resulting whole-document classifications were then compared using Kendall’s W to show the extent to which the two software programs agreed on the evaluation of the documents. The Kendall’s W was 1.00 (chi-square = 24.000, p [is less than] .001) for each of the eight mission elements. This was as expected. Despite differences in the search operations of the two programs, given correctly specified search concepts, the programs should yield the same outcomes. The only difference should occur if the unique characteristics of the search algorithms are not fully taken into consideration. Put simply, given the functionally equivalent set of key words, the two programs should find the same number of occurrences of the key words. For this reason, only the ZyINDEX analysis is presented in the rest of the paper. ZyINDEX was preferred over the Text Collector due to some of its reporting and ease of use features, the discussion of which are beyond the scope of this paper.

Validation and Assessment of Reliability

The reliability and validity of the computerized analysis approach utilizing ZyINDEX and the Text Collector software was established by comparing the computerized classifications of thirteen documents obtained from David (1991) with those of panels of human coders. Because the unit of analysis (whole document, paragraph or sentence) was of interest in this paper, computerized document classifications by whole document and sentence were compared with classifications of human coders using the whole document and sentence analysis approach. Two independent panels of human coders analyzed the thirteen documents–the first panel analyzed the statements as whole documents using the methods of Pearce and David (1987), the second panel analyzed each sentence of each statement.

Whole Document Analysis

Human-coded system. The first panel of human coders was comprised of six graduating MBA students who had completed the capstone course in Strategic Management/Business Policy. The students were given the Pearce and David article (1987) outlining the evaluation criteria and a packet of the word-processed documents. They were instructed to read the article and to independently score each document using the criteria in the article. Their instructions specifically stated, “Our goal is to score the documents using the same procedures as the authors of the article.” The instructions further required students to read and complete their evaluation of each document before proceeding to the next. They were also asked not to discuss their evaluations with any of the other students participating in the project. The documents were sorted randomly in the packets to avoid any biases due to the order in which they were evaluated.

Table 2. Extent of Agreement Between Human Coders on Whole Document

Classifications (Coefficients of Concordance for Mission Elements Coded by 6

MBA Students)

Mission Element Kendall’s W Chi-Square Significance

Customers 0.5722 20.6000 0.0022

Products 0.5298 19.0714 0.0040

Geographic Domain 0.7813 28.1250 0.0001

Technology 0.4400 15.8400 0.0146

Survival, Growth & Profitability 0.5161 18.5806 0.0049

Company Philosophy 0.3016 10.8571 0.0929

Self-Image 0.5179 18.6429 0.0048

Public Image 0.5402 19.4483 0.0035

Students were told to indicate by making a mark if they believed the document to contain each of the eight mission components. If the document did not contain a specific component, the students were to leave the space corresponding to that component blank.

Kendall’s W (coefficient of concordance) was run to show the extent to which the six coders agreed on the evaluation of the documents. Kendall’s W for each mission element is shown in Table 2. The coefficients of concordance ranged from a low of W = .3016 for the company philosophy mission element to a high of W = .7813 for the geographic domain mission element. The chi-square statistic was statistically significant at the .01 level or better for six of the eight mission elements. The chi-square for the technology element was significant at the .05 level while the chi-square for the company philosophy element was significant at the .10 level. Statistically significant chi-squares indicate a high level of agreement between coders on the evaluation of each of the elements of the documents.

For the remainder of the analysis, the evaluations of the student coders were aggregated according to a procedure used in D’Aveni and MacMillan (1990), David (1989b), David and Cochran (1986) and Pearce and David (1987). Documents were coded as containing an element if at least four of the six coders indicated that the statement referred to that specific element. The aggregated data was used for the remainder of the analysis.

Computerized content analysis. The output of the ZyINDEX computerized content analysis of the same thirteen documents were recorded utilizing three different units of analysis: by sentence, by paragraph and by whole-document. Pearson correlation coefficients comparing the whole-document computerized analysis to the whole-document analysis of the panel of six MBA students are presented in Table 3.

The computerized content analysis scores and the scores obtained by the panel of human coders were significantly correlated in a positive direction for seven of the eight mission statement elements. The two methods of content analysis reached substantially the same conclusions for seven of the eight mission elements. Only the score for the company philosophy element was not significantly related between the computerized and human coding methodologies. The company philosophy element illustrates some of the difficulties in operationalizing content analysis schemes on computerized systems. The company philosophy component discloses a firm’s “basic beliefs, values, aspirations and philosophical priorities” (Pearce & David, 1987). Human coders indicated difficulties in distinguishing statements of a firm’s philosophy from statements regarding its self-image while the computerized methodology recognized only statements containing specific words such as “believe”, “dedicated,” “commitment” or “committed” (among others) as relating to company philosophy. Difficulties such as this will be discussed in more depth in a later section.

Table 3. Comparative Analysis of Computerized and Human Coded Content Analysis

Methods

Pearson Correlation

Mission Element Coefficients

Customers 0.7303(**)

Products 0.6455(**)

Geographic Domain 0.7303(**)

Technology 0.7500(**)

Survival, Growth & Profitability 0.5477(*)

Company Philosophy -0.1667

Self-Image 0.6455(**)

Public-Image 1.0000(***)

Notes: Pearson Correlation Coefficients between Whole-Document Classifications

of Human Coded and Computerized Methods

n = 13

*p [is less than] .10

** p [is less than] .05

*** p [is less than] .001

Sentence Analysis

Human-coded system. A second panel of three MBA graduates followed a similar coding procedure to complete a sentence analysis of the thirteen word-processed documents. The content analysis instructions were expanded to encompass situations unique to content analysis by sentences such as the definition of a sentence (statement ending with a period, question mark or exclamation point) and implicit references to an element discussed in a previous sentence. For example, both of the following sentences would be coded as referring to employees: “Employees are a critical component of our success. Their creativity and hard work are the keys to our continued good performance.” A coding worksheet was created to simplify the coding process. The worksheet contained columns for each element and rows for each sentence. Coders were instructed to read each sentence separately, to decide if it referred to each of the eight mission statement elements and to mark the corresponding worksheet cells.

Table 4. Extent of Agreement Between Human Coders on Sentence Analysis

Classifications (Coefficients of Concordance for Mission Elements Coded by 3

MBA Graduates)

Mission Element Kendall’s W Chi-Square Significance

Customers 0.6865 24.7131 0.0162

Products 0.7542 27.1502 0.0074

Geographic Domain 0.8797 31.6684 0.0016

Technology 0.6806 24.5008 0.0174

Survival, Growth & Profitability 0.8582 30.8953 0.0020

Company Philosophy 0.6210 22.3565 0.0337

Self-Image 0.7569 27.2482 0.0071

Public Image 0.7334 26.4027 0.0094

Kendall’s W (coefficient of concordance) was run to show the extent to which the three coders agreed on the evaluation of the documents. Kendall’s W for each mission element is shown in Table 4. The coefficients of concordance ranged from a low of W = .6210 for the company philosophy mission element to a high of W = .8582 for the survival, growth and profitability mission element. The chi-square statistic was statistically significant at the .01 level or better for five of the eight mission elements. The chi-square for the remaining mission elements was significant at the .05 level, thus indicating a high level of agreement between coders on the sentence-by-sentence evaluation of each of the mission statement elements.

As with the whole-document coding, mission statements were coded as containing an element if at least two of the three coders indicated that the statement referred to that specific element.

Computerized content analysis. In comparing the sentence analysis of the thirteen documents using the computerized and human coder approaches, an acceptable level of agreement in classification is obtained for five of the eight mission elements as shown in Table 5. Kendall’s coefficient of concordance ranged from a low of W = .4158 for the customers element to a high of W = .9025 for the survival, growth and profitability element. The chi-square statistic for the geographic domain, survival, company philosophy, self-image and public image elements was significant at the p [is less than] .10 level or better, indicating that the computerized sentence classifications and those of the panel of three MBA graduates were in agreement. The products element approached a level of acceptable agreement with a W of .7500 and a chi-square significance of .1157. The computerized classifications did not concur with those of the panel of coders for the technology and customers elements. The lack of agreement in these two areas may be due in part to their infrequency in the documents. Customers were mentioned in two sentences or fewer in the documents. The technology element merited mention in only one sentence in the few statements that incorporated this element at all. The infrequency of mention of these two elements may make their reliable classification difficult.

Table 5. Extent of Agreement Between Human Coders and Computerized Content

Analysis on Sentence Analysis Classifications of Mission Elements

(Coefficients of Concordance and Significance Levels)

Mission Element Kendall’s W Chi-Square Significance

Customers 0.4158 9.9798 0.6177

Products 0.7500 18.0000 0.1157

Geographic Domain 0.7757 18.6156 0.0982

Technology 0.6098 14.6341 0.2621

Survival, Growth & Profitability 0.9025 21.6603 0.0415

Company Philosophy 0.8742 20.9816 0.0507

Self-Image 0.9143 21.9429 0.0382

Public Image 0.8253 19.8072 0.0708

In summary, the computerized content analysis scheme for mission statements was shown to have an acceptable level of semantic validity for the analysis of most mission elements in that the computerized and the human coded categorization of the text was in agreement. The agreement between the human coded and computerized categorizations was stronger when the analysis was conducted at the level of the whole document as seven of the eight mission elements were significantly correlated. The sentence analysis results indicated less agreement among panel members and between the human coded and computerized results. Categorizations of the human coders and the computerized content analysis were correlated at an acceptable level of significance for six of the eight mission elements, thus providing an indication of the semantic validity of the computerized approach.

Comparison Of Different Units Of Analysis

Does the unit of analysis make a difference in the way the mission statement elements are classified? This question was addressed by comparing the classifications for whole-document, paragraph, and sentence units of analysis for a sample of 159 documents.

The text to be analyzed. As noted in previous empirical studies of mission statements, many companies do not have formal mission statements. Cochran and David (1986) found that 40.4% of the 218 Fortune 500 firms responding to their request had no mission statement. Lack of a formal mission statement document should not be interpreted as the firm’s failure to consider its mission, purpose or reason for existence. Individuals within the firm may have a clear idea of its mission without creating a formal mission document. A sense of a corporation’s mission may be contained in the annual report (Ireland & Hitt, 1992). In particular, the letter to shareholders section serves as a good indicator of the major topics attended to by organizational managers (Watzlawick, Beavin & Jackson, 1967; Bradley & Baird, 1977). As discussed earlier, letters to shareholders have previously been used in strategic management research by D’Aveni and MacMillan (1990).

A random sample of letters to shareholders and mission statements for Fortune 500 firms was extracted from the Compustat Corporate Text CD-Rom database for 1990. Only those firms which are publicly held are included in the Compustat database. The final sample included 159 firms representing a wide variety of industries. If the firm included its mission statement in the annual report, as did 65 of the 159 firms (41%), the mission statement was utilized in this study. For this study, the letters to shareholders and mission statements were converted into word-processed documents of identical layout and appearance.

Method of analysis. The difference in unit of analysis requires some adjustment to permit this type of comparison. For example, the customers mission element was included in 10 of 13 mission statements, 12 of 61 sentences, or 16 of 1,564 words. Does this indicate conformity of the classifications despite differences in the classifying unit or are different conclusions being reached depending on the unit of analysis? As the example suggests, a direct comparison of frequencies may not resolve these issues. Therefore, two approaches were used: dichotomous classification (element is present or not present) and actual frequencies of occurrence for each of the different units of analysis.

Dichotomous classification. To permit comparability of the results of the classification despite differences in the classification unit, the data were recoded as a simple dichotomous classification. That is, if the mission statement contained one sentence or word phrase that referred to a specific element, it was coded as having that element. The result of the recoding was to convert the sentence and word classifications to essentially a whole document classification for comparison purposes. The recoding step was performed for the sentence analysis of human coders by coding a document as containing the element if one or more sentences in the document were coded as containing the element.

Because the computer content analysis software is essentially performing a word/phrase analysis upon chunks of text determined by the investigator and because of the computer’s perfect repeat reliability, it was predicted that no difference would be observed between the whole-document, sentence and word analyses. An analysis of the recoded data supported this prediction. The detection of the mission element in the statements by the computerized content analysis software was identical regardless of the unit of text analyzed. Kendall’s coefficient of concordance (W) for all eight mission elements was 1.000 (Chi-square = 36,000, p [is less than] .001) indicating perfect agreement. Only a clerical error in recording or entering the data, a software glitch or hardware malfunction would yield differences in the classification of the text by computerized content analysis software.

Because the sample of documents analyzed in this portion of the study contains a mixture of mission statements and letters to shareholders, analysis of variance (ANOVA) was used to determine if there was any difference in the inclusion of the mission elements that could be attributed to differences in the type of document analyzed. No significant difference was detected in the computerized whole document classification of the documents that could be attributed to the type of document (whether mission statement or letters to shareholder). Since the results do not vary whether letters to shareholders or mission statements are used, all subsequent analysis utilized the combined sample of 159 documents.

Table 6. Extent of Agreement Among Human Coders on Classifications of Mission

Elements When Analyzed by Whole Document and by Sentence (Coefficients of

Concordance and Significance Levels)

Mission Element Kendall’s W Chi-Square Significance

Customers 0.8750 10.5000 0.1051

Products 0.8125 9.7500 0.1356

Geographic Domain 0.8636 10.3636 0.1102

Technology 0.7222 8.6667 0.1932

Survival, Growth & Profitability 0.7727 9.2727 0.1588

Company Philosophy 0.4167 5.0000 0.5438

Self-Image 0.5000 6.0000 0.4232

Public Image 0.7083 8.5000 0.2037

A separate comparison of the classifications of the human coders for whole-document and sentence analysis is presented as Table 6. When the recoded sentence classifications are compared with the whole-document classifications, the Kendall’s coefficients of concordance show a lack of acceptable agreement in the classification of all eight mission elements. The coefficients of concordance range from W = .8750 (p = .1051) for the customers element to a low of W = .4167 (p = .5438) for the company philosophy element. Unlike the computerized content analysis results, the classifications of the human coders are divergent for different units of analysis.

Frequency of Occurrence Classification. This analysis was performed by computerized content analysis of the 159 documents. The documents were analyzed for the number of sentences and paragraphs containing the mission element. The frequency of “hits” or the number of terms from the classification rules contained in each document was used as an indication of the number of times an element was contained in the whole document. In addition, unit densities were used for the word/phrase, sentence, and paragraph analyses. Unit densities were computed by determining the percent of sentences, for example, within each document which referred to each of the mission statement components. ZyINDEX calculated a unit density measure called the “hit” density which was used to measure the relative density of an element in the whole document. According to the program manual, hit density is “a function of the number of hits and the size of the file” (Information Dimensions, 1993). Documents with higher hit density scores contain more hits as a percentage of bytes of file storage.

Means, standard deviations and inter-method correlation coefficients are provided in Table 7. In all cases, the correlation coefficients relating frequencies TABULAR DATA OMITTED for one unit of analysis with another or with the unit density measures were positive and significant at p [is less than] .01, thus indicating a strong relationship between all measures for each mission element. Stronger inter-method correlations are observed between the simple frequencies measures than when the frequencies are correlated with the density measures. Density measures are also more strongly correlated with each other than with the simple frequency measures. The hit density, which measured the number of hits relative to the storage size of the document, was positively correlated at the p [is less than] .001 level with all other measures.

Discussion

Differences in the classification of the documents by the panels of human coders and those obtained by the computerized content analysis are due to two possible sources of error. The first source of error is in the instructions and training given to the human coders. Inadequately specified conceptual definitions and/or poor directions for coding may have resulted in inconsistencies in the coding of the mission elements. There is some evidence that better specification of the conceptual definitions for the self image and company philosophy mission elements in the directions given to the human coders may have improved their ability to adequately distinguish these two elements and more properly code them in their analyses. The second source of error is in the computer coding instructions. If the essence of the mission elements or classification categories is not adequately captured in the list of keywords and phrases used in the computerized coding rules, the content analysis will not yield an accurate classification of the document’s contents. Because it is difficult to define all possible keywords and phrases for concepts as broadly defined as some of the mission elements, the computerized content analysis programs should continue to evolve and be revised throughout their usage so that coding accuracy improves. In this way, the evolution of the computerized content analysis program mimics the improvements humans make in coding documents as the they receive additional training and gain experience in the process.

Interpretation of the comparison of the different units of analysis must proceed with some caution. Although it may appear in the dichotomous classification that the computerized approach is superior to classification by human coders due to the complete agreement of the computerized classifications across whole-document, sentence and word units of analysis, one may question whether the perfect reliability is at the expense of validity. A closer examination of the recoded classifications of the human coders indicated that in some cases, the coders indicated that a mission element was present when analyzing the whole document and not present when analyzing by sentence. This suggests that the coders may have been using a broader context when analyzing whole documents. For example, the overall statement may have been more effective in communicating an impression of the firm’s corporate philosophy than each sentence read without regard to its context. As will be discussed in a later section, the computerized content analysis software is incapable of matching human capabilities to consider the broader context.

In the frequency of occurrence classification using the computerized content analysis program, positive and highly significant inter-method correlations indicate strong agreement in the conclusions one might draw utilizing the number of times the mission element was contained in each sentence, paragraph or whole document. Differences in the length of the documents did not significantly alter the results of the analysis as the sentence, paragraph and hit densities were all strongly correlated with each other and with each of the frequency measures. Both of these findings may be due to the writing style typically used in annual reports and mission statements. After a number of formal research studies found that about half of annual reports were consistently classified as “very difficult” to understand (i.e. required a college education) and the remainder were classified as “difficult” (i.e. required a high school education), companies have attempted to improve and simplify the writing style of the reports (Means, 1981; Lebar, 1982; Heath & Phelps, 1984). Frequently, simplification of the text is accomplished by shortening sentences and reducing the number of sentences contained in a paragraph (Heath & Phelps, 1984). This approach was observed in the 159 corporate reports included in this study as the average number of sentences per paragraph was 2.24 (0.9571 STD DEV). When fewer sentences are included in each paragraph, one would expect stronger correlations between sentence and paragraph frequencies as observed in this study. As the number of sentences per paragraph increases, wider variation in frequency classifications by paragraph or by sentence would be expected. Therefore, researchers utilizing the content analysis technique must carefully consider the writing style used in the documents to be analyzed to select the most appropriate frequency measure for correct classification of the documents. Smaller text units are expected to yield more reliable analysis than larger units such as sentences or paragraphs since these larger units may contain references to more than one of the classification variables. Evidence that the level of reliability varies by size of the unit of analysis is provided by Grey, Kaplan and Lasswell (1965) and Saris-Gallhofer, Saris and Morton (1978).

As Grey et al. (1965) suggest, the choice of unit for analysis depends upon the research question under investigation. Use of too large a unit may result in missing phenomenon of interest, while too small a unit may result in obscuring more subtle interpretations of statements in context. Although the above analysis of different units indicated that this choice is more important for content analysis using human coders, careful consideration must also be given to this aspect when using computerized content analysis. In some instances, the investigator may wish to draw conclusions about intensity of concern the communicator has for each category within a given document. In this type of research, it would be more important to know the frequency with which an issue was addressed rather than its mere presence or absence in the text. While word/phrase analysis would provide the best indication of the intensity of concern, it is not often done with human coders due to the tedious coding it requires. The “number crunching” capabilities of the computer make this type of analysis readily available. However, care must be taken in performing and interpreting word/phrase frequency analysis. As Pfaffenberger (1988) indicates multiple meanings of words and the inability of the computer to recognize the communicative intent of word usage may result in an overstatement of the communicator’s message. Word frequencies analysis is best applied to text where the communicator is deliberately trying to emphasize specific themes such as propaganda, political speeches and political songs (Pfaffenberger, 1988). Because effective mission statements are intended to “arouse readers’ emotions to be supportive of the organization” (Cochran & David, 1986, p. 109) in much the same way that political rhetoric is used to bring in votes, word frequency analysis would be an appropriate method for examining the intensity of a firm’s concern for specific mission elements.

Implications for Management Research

Human coding systems are by their very nature difficult and expensive to use. Human coded content analysis is also difficult to reproduce without extremely specific classification rules and comparable background and training for the coders. When coupled with the inter-rater reliability and repeat reliability problems associated with content analysis, the expense and difficulty of use discourage widespread application of human coded content analysis techniques. Unfortunately, content analysis remains an important research technique for the analysis of management attitudes, values and concerns due to its unobtrusive analysis of publicly available communications. The use of documents or speeches also permits a longitudinal analysis of management thoughts, beliefs and attitudes which is unmatched by other research techniques.

Advantages of computerized content analysis approaches over human coded content analysis techniques include:

1. Perfect stability of the coding scheme due to the computer’s ability to always apply the coding rules in the same way (Weber, 1990, p. 15).

2. Explicit coding rules yielding formally comparable results. (Weber, 1990, p. 41).

3. Perfect coder reliability of the computerized approach thus freeing the researcher to concentrate on other aspects of the inquiry such as validity, interpretation and explanation (Weber, 1990, p. 41).

4. Easy manipulation of text to create word-frequency counts, key-word-in-context listings and concordances that would require considerable effort to create manually. These listings allow researchers to explore the way in which certain terms or themes are presented in the texts.

5. Ability to process larger volumes of qualitative data at lower cost (Gephart & Wolfe, 1989).

Because computerized systems such as ZyINDEX and the Text Collector can approach the accuracy of human-coded content analysis as evidenced in this study and result in improvements in reliability and stability, they merit consideration as a more cost effective alternative research methodology. The relatively inexpensive cost ($500 or less) and standard hardware requirements of most text management software packages make it possible for anyone with a personal computer and minimal programming skills to perform content analysis of large volumes of textual material at a minimal cost.(2)

Computerized content analysis is also not without limitations. Care must be taken in the implementation of computerized content analysis methodologies to ensure the method’s validity. Limitations that may impact validity include:

1. Lack of natural language processing capabilities in the software. Because the computer performs the content analysis of the text literally, any meaning that is imputed by human readers by reading with a broader context will not be properly categorized by the computer. Ambiguous concepts, such as the self-image mission element in this study, which cannot be represented adequately by a listing of key words or phrases may not be appropriate for computerized content analysis.

2. Inability of the software to recognize the communicative intent of word usage when performing content analysis. Simple word frequency analysis may provide misleading information about the intent of the communicator’s use of certain words due to the insensitivity of the software to negation, irony or the tone of voice (Krippendorff, 1980). This limitation can be reduced by examining key-word-in-context listings to determine whether the words are used in the desired context.

3. Inability of the researcher/programmer to provide an exhaustive listing of key words for a category that is by nature indeterminate. This limitation was encountered in developing the classification rules for the product mission element. Because developing an exhaustive list of every possible product or service that could be provided by a business was not possible, the computerized content analysis programs used a more restricted list of general references to products and services. As a result, human coders recognized things such as “molybdenum, coal, iron ore, copper, lead, zinc, petroleum and natural gas” (AMAX mission statement as cited in Pearce & David, 1987) as products, but the computerized content analysis programs did not.

4. Inability of the software to resolve references back or forward to words appearing elsewhere in the text. Pronouns referring to nouns in other sentences would not be interpreted correctly by computerized content analysis software. D’Aveni and MacMillan (1990) included forward and backward references as special coding exceptions in their instructions to their panel of coders. Specification of similar rules would be extremely difficult in computerized content analysis programs. One possibility for eliminating this problem would be to have a human coder enter the correct referent for each pronoun encountered in the text prior to submission of the text to the content analysis software. Pfaffenberger (1988, p. 31) describes a similar approach for coding sociology field notes for computerized analysis.

5. Inability of the software to analyze qualitative data as strips instead of imposing its own artificial definition of the retrieval unit (words, paragraphs, etc.) on the data (Pfaffenberger, 1988). A strip is defined as a bounded segment of qualitative data whose boundaries are demarcated by the source themselves or the researcher (Agar, 1986). Proper analysis requires examination of the strip in its entirety to determine the ideas, behaviors or concepts the strips represent. Pfaffenberger (1988) suggests that this problem can be countered by using content analysis software that permits the user to demarcate a unit of coherent data using special symbols that the program then uses to define the unit of retrieval.

6. The capabilities of content analysis software may result in “word crunching”, transforming meanings into numbers that are meaningless. Without the development of theoretical rationales for the classification of textual data, computerized content analysis risks becoming yet another form of abstracted empiricism (Gephart & Wolfe, 1989). The literal processing algorithms used by computerized content analysis software must be carefully considered in deciding the appropriateness of the application of this methodology to the research in question. Research which requires analysis of causal reasoning or appreciation of linguistic phenomenon such as irony or sarcasm cannot be fully accomplished using automated data analysis with content analysis software. However, the indexing, storing and retrieval capabilities of the software may prove extremely useful in supporting the reSearcher’s use of his/her judgment in performing this type of analysis by making the textual data readily available in an infinite variety of combinations.

7. Continued reliance on human coders. Some use of human coders for classification of at least a sub-sample of text will probably continue to be necessary to establish the reliability and validity of the computerized content analysis.

Although computers are not capable of making value judgments, the text manipulation capabilities presented by content analysis software may enable researchers to view the data in different ways, thus leading to improved ability for the researcher to draw conclusions about the values contained in the communications. Pfaffenberger (1988) suggests that the construction of culturally specific and culturally valid content category classification schemes is an important research end in and of itself. The recursive process of defining content categories, testing the usefulness of the categories on textual data and further refining the categories puts the researcher into a process of discovery that may yield a growing awareness of the values, attitudes and sentiments represented by the text (Pfaffenberger, 1988).

Researchers mindful of these limitations may find computerized content analysis a useful and appropriate technique requiring careful implementation. The following approach is recommended for development of computerized classification schemes. The steps shown in Figure 1 for human-scored systems should be followed to establish the effectiveness of the computerized content analysis program in accurately classifying the message contained in the text. These steps specify an iterative process of specifying classification rules (writing the computer program), coding of a sample of text by multiple coders (both human and computer) and assessment of reliability. If the reliability is unacceptable, re-specification of the rules and further testing is required. Once acceptable reliability and accuracy are obtained, the revised content analysis program can then be applied to all the text.

Future Applications in Management Research

Content analysis methodologies add an effective tool to management research by permitting researchers to utilize data sources that have not been widely used in previous research. Data from corporate reports and other firm generated texts could be utilized to address a variety of management research questions such as:

1. Do firms behave in a manner consistent with their stated intentions or do they adopt socially responsible stances in their annual reports just to interest investors in their stock? Content analysis has been previously used to assess the level of a firm’s social responsibility (Bowman, 1973; Gephart and Wolfe, 1989; Wolfe, 1989). Computerized content analysis of annual reports could be compared with Fortune magazine’s annual “most admired corporations” community and environmental responsibility scores (as published in January issues such as Davenport, 1989) to detect differences in the firm’s stated social responsibility position and that perceived by the professional investors that submit the Fortune rankings.

2. Does the strategic focus or environmental scanning focus of strategists shift with changes in external environmental factors? The unobtrusiveness of the content analysis technique proves especially important in this type of investigation. Researchers could utilize content analysis methodologies to study the focus of attention of top managers in response to external events. This research would use content analysis in a similar manner as used by D’Aveni and MacMillan (1990) to study differences in the attention of top managers to environmental factors in surviving and bankrupt firms.

3. Is there an annual report credibility problem? Do the various audiences of these corporate reports believe that statements contained in them are more for public relations value than a reflection of the firm’s actual practice or intended future direction? What implications does mistrust in annual report statements have on commitment to organizational goals, employee job satisfaction, and competitor strategies? Content analysis could be utilized to isolate themes contained in the documents which then could be presented to stakeholders for their reactions. An investigation into the ways firms represent or address issues in different contexts and medias for different audiences would provide answers to these and many other interesting research questions. When analyzed from an employee perspective, this research might provide important insights into many human resource management issues.

4. Is the nature of the multi-business firm sufficiently distinct from that of a single business firm to warrant differences in the form of the mission statements? Pearce and David (1987) identified differences in the composition and intent of corporate and business-unit mission statements as a critical area for future mission statement research. The computerized content analysis program described in this article has been used to analyze the mission statements of firms with differing levels of diversification. Statistical analysis of the frequency of inclusion of the mission statement components revealed significant differences existed by level of diversification (Morris, in press). The computerized content analysis scheme will also be used in future research to examine differences in the content of mission statements for firms headquartered in different countries similar to the work of Brabet and Klemm (1994) comparing mission statements of French and British firms.

These four research areas just scratch the surface in terms of the many different ways content analysis could be effectively utilized to explore interesting management research issues. Wolfe, Gephart and Johnson (1993) provide an excellent discussion of the application of computerized content analysis methodologies to research in the areas of corporate social responsibility, industrial accidents, corporate values and organizational effectiveness. Researchers considering use of content analysis methodologies in their research must also consider the following general concerns about the utilization of content analysis as a tool for understanding organizations:

1. Uncertainty of the authorship of the text. Although signed by the CEO of the firm, most letters to shareholders are written by individuals other than the president. Mission statements are often the result of a collaborative effort of many individuals within the firm. Because mission statements and annual reports are closely reviewed by top managers (Goodman, 1980; Petzinger, 1982), it is believed that they reflect management’s perspective. Furthermore, chief executive officers are held fiduciarily responsible for annual report accuracy, thus the letters to shareholders may be attributed to top managers as a group (Salancik & Meindl, 1984). However, care must be taken in attributing messages contained in the text to specific individuals when authorship is unknown.

2. Uncertainty of the link between professed and intended positions. Documents prepared for public audience may contain statements for public relations purposes (professed position) that may differ substantially from the firms intended position. To counter this problem, Hawkins and Hawkins (1986) suggest use of Form 10-K, the annual report filed with the Securities and Exchange Commission, instead of annual reports. As Form 10-K is written more straightforwardly, researchers may be better able to discern the firm’s intended position if it is believed to be different from that professed in the annual report. Confirmation of a firm’s position using other less public sources such as internal communications would also reduce uncertainty.

3. Uncertainty of the link between intended and realized positions. Researchers must take care in interpreting the results of content analysis of public documents in that the firm’s realized or implemented position may differ substantially from the one stated. Other measures should be utilized to support conclusions about the actual strategies followed by the firm.

4. Uncertainty of the link between frequency of word appearance and management concern with the topic. Content analysis research typically presumes a direct relationship between the number of times a topic appears in a document and the intensity of the author’s concern for the topic. Weber (1990) points out that equal counting of each occurrence of a word/phrase in the same category in a given document is a practical simplification that may mask the author’s real concern for the topic. For example, contextual constraints on text length (such as the number of pages in an annual report or a time limit for a speech) may limit the author to one issue or category and preclude attention to other topics. Also, the assertion that zero mention of a topic or category reflects a lack of attention or concern for the topic may be wrong. Topics may be excluded in public reports to avoid drawing attention to a potential weakness or to avoid disclosing future actions of the firm to competitors. In these cases, the lack of mention of the topic may be as much of interest as every occurrence.

Researchers aware of these general concerns and the ways in which these issues may detract from the validity of the conclusions based on content analysis will make more informed choices about the appropriateness of the research methodology and the design of the research study. For some topics, computerized content analysis may provide researchers with a methodology that is more stable, reliable and comparable (Weber, 1990) than human coded methodologies. Additionally, computerized content analysis may permit researchers to extensively analyze large samples of data sources that have been neglected in the past.

Acknowledgment: This research was partially funded by a grant from the University Committee on Research, University of Nebraska at Omaha.

Notes

1. For an excellent review of these and a number of text management software programs and their application to qualitative research methodologies such as content analysis, see Tesch (1990) or special issues of Qualitative Sociology (Conrad & Reinharz, 1984; Tesch, 1991).

2. Computerized content analysis techniques require “costs” that may make their use too expensive for analysis of small data sets. In addition to software, the costs of computerized content analysis would include the (1) cost of a personal computer with sufficient memory, disk storage space and speed to process large volumes of data quickly; (2) the time and effort required to learn to operate the text management software; (3) the time and effort required to create the keywords/phrases to classify the text; and (4) the time and effort required to test, revise and validate the keywords.

References

Adams, A.L. (1979). Planning search strategies for maximum retrieval from bibliographic databases. Online Review, 3.

Agar, M. (1986). Speaking of ethnography. Beverly Hills: Sage.

Bowman, E.H. (1973). Corporate social responsibility and the investor. Journal of Contemporary Business, (Winter): 21-43.

Brabet, J. & Klemm, M. (1994). Sharing the vision: Company mission statements in Britain and France. Long Range Planning, 27(1): 84-94.

Bradley, P.H. & Baird, J.E. Jr. (1977) Management and communication style: A correlation analysis. Central States Speech Journal, 28: 194-203.

Cochran, D.S. & David, F.R. (1986). Communication effectiveness of organizational mission statements. Journal of Applied Communication Research, 14(2): 108-118.

Conrad, P. & Reinharz, S. (eds.). (1984). Qualitative Sociology, 7 (1,2), Special issue: Computers and qualitative data.

D’Aveni, R.A. & MacMillan, I.C. (1990). Crisis and content of managerial communications: A study of the focus of attention of top managers in surviving and failing firms. Administrative Science Quarterly, 35: 634-657.

Davenport, C. (1989). America’s most admired corporations. Fortune, January 30: 68-94.

David, F.R. (1989a). Strategic management, 2nd ed. New York: MacMillan.

—–. (1989b). How companies define their mission. Long Range Planning, 22(1): 90-97.

—–. (1991). Strategic management, 3rd ed. New York: MacMillan.

Deffner, G. (1986). Microcomputers as aids in Gottschalk-Gleser rating. Psychiatry Research, 18: 151-159.

Gephart, R.P. & Wolfe, R.A. (1989). Qualitative data analysis: Three microcomputer-supported approaches. Pp. 382-386 in F. Hoy (ed.), Academy of Management best paper proceedings. Washington, DC.

Goodman, R. (1971). Annual reports serving a dual marketing function–Report as survey. Public Relations Quarterly, 36(Summer): 21-24.

Gottschalk, L.A. & Gleser, G.C. (1969). The measurement of psychological states through the content analysis of verbal behavior. Berkely: University of California Press.

Grey, A., Kaplan, D. & Lasswell, H.D. (1965). Recording and content units–Four ways of coding editorial content. In H.D. Lasswell, N. Leites & Associates (eds.), Language of politics. Cambridge: MIT Press.

Hawkins, D.F. & Hawkins, B.A. (1986). The effectiveness of the annual report as a communication vehicle: A digest of the relevant literature. Morristown, NJ: Financial Executives Research Foundation.

Heath, R.L. & Phelps, G. (1984). Annual reports II: Readability of reports vs. business press. Public Relations Review, (Summer): 56-62.

Hockey, S. & Martin, J. (1987). The Oxford Concordance program version 2. Literary and Linguistic Computing, 2: 125-131.

Information Dimensions. (1993). ZyINDEX [computer program and manual]. Buffalo Grove, IL: ZyLAB Division, Information Dimensions.

Ireland, R.D. & Hitt, M.A. (1992). Mission statements: Importance, challenge, and recommendations for development. Business Horizons, (May-June): 34-42.

Kelly, E.F. & Stone, P.J. (1975). Computer recognition of English word senses. Amsterdam: North Holland.

Krippendorff, K. (1980). Content analysis: An introduction to its methodology. Beverly Hills, CA: Sage.

Lebar, M.A. (1982). A general semantics analysis of selected sections of the 10-K, the annual report to shareholders, and the financial press release. The Accounting Review, (January): 176-189.

Means, T.L. (1981). Readability: An evaluative criterion of stockholder reaction to annual reports. Journal of Business Communication, (Winter): 25-33.

Morris, R.J. (in press). Mission statement content and firm diversification: A contingency approach. Journal of Business Strategies.

O’Neill Software. (1977). The Text Collector, [computer program and manual]. San Francisco: O’Neill Software.

Pearce, J.A.II. (1982). The company mission as a strategic tool. Sloan Management Review, 23(3): 15-24.

Pearce, J.A.II & David, F. (1987). Corporate mission statements: The bottom line. Academy of Management Executive, 1(2): 109-115.

Pfaffenberger, B. (1988). Microcomputer applications in qualitative research. Newbury Park, CA: Sage.

Perez, E. (1990). Managing text. Data Based Advisor, 8(6): 83-93.

Petzinger, T., Jr. (1982). Double talk grips business reports as firms try to

COPYRIGHT 1994 JAI Press, Inc.

COPYRIGHT 2004 Gale Group