Development, Reliability, and Validity of the Measurable Assessment in Recreation for Resident-Centered Care (MARRCC)
Boothman, Sienna
This paper examines the Measurable Assessment in Recreation for Resident-Centered Care (MARRCC)-a new standardized computer-based assessment developed specifically for therapeutic recreation professionals working within Long Term Care (LTC). The conceptual background and development of the MARRCC and its validation within a sample population of residents from 11 LTC facilities will be outlined. Inter-rater reliabilities from .97 to .99 and intra-rater reliabilities from .95 to .99 are reported. Validity correlation with the Multidimensional Observation Scale for Elderly Subjects (MOSES) was statistically significant and supports the validity of the MARRCC. Results support the reliability and validity of the MARRCC as a clinical assessment for therapeutic recreation professionals working within LTC settings.
KEY WORDS: Therapeutic Recreation, Assessment, Reliability, Validity
In an era of increased accountability and demand for outcome-based programming, therapeutic recreation (TR) professionals working within Long Term Care (LTC) must have an empirically-tested clinical assessment (Dunn, Sneegas, & Carruthers, 1991; Stumbo, 2001; Witt, Connolly, & Compton, 1980). Such an assessment must be based upon a sound conceptual and theoretical foundation. It must undergo rigorous testing for both reliability and validity, and include comprehensive manuals with clear directions for consistent implementation, scoring, analysis, and interpretation of assessment results (Dunn, 1989; Peterson & Stumbo, 2000; Stumbo, 2002). Furthermore, it should be capable of assessing the general health status and the physical, cognitive, and psycho-social needs of residents as related to their ability to engage in optimally satisfying leisure interests and activities (Austin & Crawford, 1996; Howe, 1989; Peterson & Stumbo, 2000; Sneegas, 1989; Stumbo, 1996). Finally, it must be capable of establishing resident clinical direction as well as measuring change over time secondary to therapeutic intervention (Peterson & Stumbo, 2000; Shank, Kinney, & Coyle, 1993; Stumbo, 1996). The Measurable Assessment in Recreation for Resident-Centered Care (MARRCC) was designed to meet these needs.
The purpose of the MARRCC is to provide TR professionals with a standardized assessment of a resident’s functional level in each of the physical, cognitive, social, and emotional domains as related to recreation participation. The four domains are incorporated into the MARRCC for three reasons. First, the assessment of each of the four domains enables the TR professional to develop an objective, assessment-based understanding of the resident- one that includes both the resident’s strengths and limitations. second, an assessment of a resident’s functioning level in each domain provides an indicator of that resident’s ability to actively engage in leisure pursuits. Third, it enables the recreation professional to more accurately address the needs of the entire person rather than primarily focusing on an illness or disabling condition, thereby maintaining the foundation that TR is a holistic approach to wellness (Howe-Murphy & Charboneau, 1987; Peterson & Gunn, 1984; Peterson & Stumbo, 2000). Thus, the MARRCC enables the TR professional to place residents into appropriate recreation programs or groups based upon each resident’s assessed functional levels, needs, and interests.
Conceptualization
The underlying conceptual framework for the MARRCC may be found within the leisure ability model (Peterson & Gunn, 1984; Peterson & Stumbo, 2000) and the model of selective optimization and compensation (Baltes & Baltes, 1990). The leisure ability model operationalizes the delivery of TR practice and consists of three program categories: Functional Intervention, Leisure Education, and Recreation Participation. Functional Intervention focuses on enhancing a client/patient’s ability to participate in leisure experiences by addressing conditions relative to that person’s physical, cognitive, social, or emotional functional levels. Leisure Education focuses on the development and acquisition of skills, attitudes, and knowledge related to leisure participation and leisure lifestyle development. Recreation Participation services provide the opportunity for clients/patients to engage in structured group recreation experiences for enjoyment or self-expression (Peterson & Gunn, 1984; Peterson & Stumbo, 2000). The three elements of the leisure ability model offer a continuum of care that individuals may access based on assessed needs and anticipated outcomes.
The model of selective optimization and compensation (Baltes & Baltes, 1990) is a developmental theory of successful aging and adaptation. It is based on the premise that successful aging is a process that involves three interacting components: selection, optimization, and compensation. Selection refers to the restriction in the amount and variety of one’s life pursuits due to a decrease in functional abilities. This decrease in functioning is caused largely by impairment of the individual’s ability to adapt to environmental or intrinsic demands. The process of selection implies that an individual will choose to participate in life pursuits in which the environmental demands match personal motivations and remaining resources, skills, and abilities (Baltes & Baltes, 1990). Optimization refers to the propensity of an individual to capitalize on remaining personal strengths and abilities as well as environmental resources while engaging in leisure and life pursuits. Thus, even though the number, and perhaps variety, of life pursuits are decreased due to the aging process, the potential for the individual to experience a sense of satisfaction and control while engaging in these pursuits is maintained at the highest level possible (Baltes & Baltes, 1990). Compensation refers to the use of one or any combination of psychological cues, social/environmental adaptation, and technological adaptive devices to compensate for those functional abilities and behavioral capacities that are lost due to the aging process. These adaptations enable the individual to continue to participate in chosen life pursuits with a level of independence and mastery as similar as possible to that which was experienced prior to the onset of a decrease in functioning (Baltes & Baltes, 1990). The model of selective optimization and compensation, therefore, describes a lifelong process within which individuals pursue their own well-being. This is accomplished through ongoing assessment of one’s own abilities and limitations, maximization of one’s own abilities and resources, and the development and utilization of creative strategies for adapting to changing personal, task, and resource challenges. This model is also consistent with established theories of aging including role theory (Atchley, 1977; Kaplan, 1979; Kelly, 1983), continuity theory (Atchley, 1977; Kaplan, 1979; Neugarten, Havighurst, & Tobin, 1968) and activity theory (Havighurst & Albrecht, 1953; Havighurst, Neugarten, & Tobin, 1968; Lemon, Bengston, & Peterson, 1972; Longino & Kart, 1982; Neugarten, Havighurst, & Tobin, 1961).
Both the leisure ability model (Peterson & Gunn, 1984) and the model of selective optimization and compensation (Baltes & Baltes, 1990) lend support to the development of an assessment of resident/client/patient physical, cognitive, and psycho-social functioning. Within the leisure ability model, individuals with a disability or illness are assessed to determine their functional abilities and limitations, as well as their leisure-related knowledge, attitude, awareness and abilities. Specific disabilities that would limit or interfere with satisfying leisure involvement become areas for possible program intervention. Outcome-oriented programs are then designed that seek to ameliorate these disabilities and limitations (Peterson & Gunn, 1984; Stumbo & Peterson, 1998). Similarly, the selection component of the selective optimization and compensation model focuses on the domains of functioning that match environmental demands with the individual’s motivations, skills, and abilities. “We cannot predict what any given individual’s successful aging will look like until we know the domains of functioning and goals that that individual considers important, personally meaningful, and in which he or she feels competent” (Baltes & Carstensen, 1996, p. 399).
Development of the MARRCC
The MARRCC assessment package consists of an Initial Assessment, an Annual/ Change of Condition/Re-admit Assessment, a Quarterly Progress Note section, and a Care Plan Development section. Detailed manuals regarding the implementation of the assessment package accompany each section of the MARRCC. The manuals contain clearly written procedures and guidelines for scoring each item found within the four sections of the MARRCC as well as clinically valuable information for users regarding the analysis and interpretation of the assessment results.
Initial Assessment
The purpose of the Initial Assessment is to enable the TR professional to acquire information about each resident’s past and present leisure attitudes and interests, safety needs, and functional levels within each domain. The TR professional then uses this information to cluster-group residents into functional levels for each domain. Assessing each resident’s functional levels enables the TR professional to accurately distinguish between residents who require one or any combination of functional intervention, leisure education, or specialized programming for behavioral intervention and those whose needs can be adequately met through regularly-scheduled recreation participation programs.
Annual/Change of Condition/ Re-admit Assessment
The purpose of the Annual/Change of Condition/Re-admit Assessment is to enable the TR professional to track and monitor changes in a resident’s functional level and individual needs. Consequently, it also enables the TR professional to appropriately modify a resident’s recreation program in response to these changes.
Care Plan Development and Quarterly Progress Note Sections
The Care Plan Development section assists TR professionals in developing measurable, appropriate, and individualized care plan goals that are directly related to assessed needs. The Quarterly Progress Note section assists the TR professional in evaluating resident individualized care plans by helping to ensure that resident progress toward achievement of care plan goals is consistently monitored and documented.
Domain Scales
Domain scales were developed to assess a resident’s functional levels within the Initial Assessment and the Annual/Change of Condition/Re-admit Assessment of the MARRCC. Each of the MARRCC’s four domain scales-physical, cognitive, social, and emotional-is composed of 10 items with corresponding indicators. Each item is a statement defining either a strength or a limitation directly related to a particular domain. The indicators, which correspond to the items, provide definitive parameters for measurement. The domain scales are, therefore, capable of driving treatment in two ways. First, the domain scales enable the TR professional to place residents into appropriate recreation programs or groups based on functional levels. second, the domain scales provide the TR professional with a detailed list of strengths and limitations for each resident within each domain. This list should be utilized when developing individualized care plans, choosing appropriate interventions, and determining which residents require one or any combination of functional intervention, leisure education, or recreation participation programs.
As part of the initial development of the MARRCC Domain scales, an extensive literature review of instruments specific to TR (Buettner & Martin, 1995; Burlingame & Blaschko, 1997; Kloseck & Crilly, 1997; Olsson, 1990), general functioning level (Fillenbaum, 1988; Granger, Hamilton, Linacre, Heinemann, & Wright, 1993; Harvey & Jellinek, 1981), physical domain (Burlingame & Blaschko, 1997; Granger, Albrecht, & Hamilton, 1979; Linn, 1967), cognitive domain (Folstein, Folstein, & McHugh, 1975; Rosen, Mohs, & Davis, 1984), social domain (Burlingame & Blaschko, 1997; Linn, 1988), and emotional domain (Brink, 1982; Burlingame & Blaschko, 1997; Hays, Meador, Branch, & George, 2001; Neugartenet et al., 1961; Parker, Ellison, Kirby, & Short, 1975; Schnelle, Wood, Schnelle, & Simmons, 2001) was conducted. Items for the MARRCC Domain scales were then developed or modified from information identified within literature (Burlingame & Blaschko, 1997; Kane & Kane, 1981; McDowell & Newell, 1996) and pooled with items developed from information specifically derived from clinical experience. Therefore, the MARRCC Domain scales contain descriptions of functioning levels that (a) practicing TR professionals have identified as being particularly relevant in LTC, (b) are consistent with current clinical terminology, and (c) complement the content of existing assessments.
Each MARRCC Domain scale lists ten items divided in three sections. Section A consists of three items with corresponding indicators that point to high functionality; section B consists of four items with corresponding indicators that point to moderate functionality; and section C consists of three items with corresponding indicators that point to low functionality. An item is marked “true” if the resident meets all the minimum requirements of the indicator. An item is marked “false” if the resident does not completely meet any requirement of the indicator (see Appendix A).
Each section is then assigned a score based on how many items were marked “true” and on how many items were marked “false” for that section. Since the purpose of sections A and C is to identify residents who are beyond the realm of moderate functioning, the indicators in those sections are weighted slightly more than the indicators in section B. The scoring for the MARRCC Domains represents a resident’s score relative to a median rating of zero-the greater the positive number, the higher the functional ability; the greater the negative number, the lower the functional ability.
The parameters for defining high, moderate and low functioning were originally established based on the normal distribution. Subsequent analysis and trend data led to a minor revision of one scoring parameter. The revision increased the range of low functionality to include residents who are borderline low/moderate, and therefore have a higher likelihood of requiring treatment.
Pilot Phase
Content validity. A panel of experts was selected and asked to review the MARRCC and its accompanying manuals for content validity. The panel included four doctoral-level TR educators, two of whom developed and tested well-respected instruments in the field of TR, and one who developed practice guidelines for TR specific to dementia care. The panel also included one LTC placement specialist with over 15 years experience, and 10 recreation specialists each with 10 or more years experience in the field.
Upon completion of its review, the panel was asked to respond to five questions that pertained to the MARRCC in general (see Appendix B) and three questions that were specific to the MARRCC Domain scales (see Appendix C). Content validity results were highly supportive of the MARRCC indicating that it was well written, that items were appropriate and inclusive, and that items were representative of the domains.
Inter-rater reliability.
To evaluate the inter-rater reliability of the MARRCC, seven research assistants (resulting in 21 different pairings) were provided with the assessment and its manuals. Each assistant was then asked to assess the same three residents on the same day using the domain scales of the MARRCC. The research assistants received only the MARRCC and the MARRCC manuals to ensure that the utilization of the MARRCC during reliability testing would be consistent with its intended use in the field.
Percentage agreements were calculated by dividing the number of exact agreements by the number of exact agreements plus disagreements (Huck, Cormier, & Bounds, 1974). Percentage agreements between the assessors averaged 88%. Reliability analysis of the MARRCC domains was achieved utilizing a modified averaging technique. To determine if an alpha of scale reliability was appropriate, an additive scale (adding all items in each domain) was created and plotted against the domain scores calculated from the same data. Correlation between the two scores was very high, ranging between .98 and .99. Therefore, it was considered appropriate to calculate values for Alpha Reliability Coefficient utilizing the additive scale. The reliability results were exceptional: Physical (r = .99), Cognitive (r = .99), Social (r = .99), and Emotional (r = .97). The high inter-rater correlation indicates that recreation professionals can properly utilize and interpret the MARRCC data and score sheets with little or no assistance beyond the instruction provided in the MARRCC manuals.
Following inter-rater reliability analysis and expert panel review, it was found that two indicators in the physical domain scale and one in the emotional domain scale required some minor modification in order to increase clarity and decrease the likelihood of user error. These modifications were simple changes in the descriptive terms used to specify the parameters for the corresponding items. The cognitive and social domain scales did not require revision. After completion of the pilot phase in September 2000, the domain scales were tested for criterion-related validity and intra-rater reliability.
Method
Participants/Raters
Participants. Participants in the study included 66 residents from 11 Skilled Nursing Facilities (SNFs) located in California. To be included in the study, participants were required to have been residents in the facility for at least 30 days. To ensure that the sample was representative of the population with which the domain scales of the MARRCC would ultimately be utilized, no other restrictions were made with regard to participant selection. Participants were identified from the facility census listing using a table of random numbers. The final sample included 17 males and 43 females. Participants ranged in age from 48 to 103 years. The average age of those rated was 79.08 years, with a standard deviation of 11.46 years.
Raters. Raters in the study were 11 Recreation Service Directors (RSDs) from the same 11 SNFs as mentioned previously. Each of the RSDs was involved with direct resident care and was well acquainted with the residents being assessed. Because each of the selected SNFs offered rehabilitation and sub-acute and skilled long-term care, they were considered representative of SNFs in general.
Materials
Materials used in the study were (a) the MARRCC assessment package containing the four domain scales and manuals, and (b) the Multidimensional Observation Scale for Elderly Subjects (MOSES) (Helmes, Kaiman, & Short, 1987), a second assessment, along with written instruction specific to its use. The MOSES scale provides five separate scores each measuring a different level of functioning. The five levels of functioning assessed in the MOSES are Self-Care Functioning, Disoriented Behavior, Depressed/Anxious Mood, Irritable Behavior, and Withdrawn Behavior.
Design
Criterion-related validity. The MOSES (Helmes et al, 1987) provides a comprehensive, valid, and reliable assessment of resident physical, cognitive, and psycho-social functioning. Therefore, for the purpose of testing validity, the assessment scales of the MARRCC were compared to assessment scales of the MOSES. It was hypothesized that a positive correlation between the two measures regarding general functioning within each domain would be found resulting in acceptable levels of association.
Although the MOSES was developed to be used by individuals from the nursing profession and the items chosen to be assessed within the MOSES scales are not specific to recreation, parallels could still be drawn. Therefore, the MARRCC Cognitive Domain scale was compared to the Disoriented Behavior scale of the MOSES, the MARRCC Social Domain scale was compared to the Withdrawn Behavior scale of the MOSES, the MARRCC Emotional Domain scale was compared to the combined Depressed/Anxious Mood and Irritable Behavior scales of the MOSES, and finally, the MARRCC Physical Domain scale was compared to the Self-Care Functioning scale of the MOSES.
Prior to testing, the MOSES Self-Care Functioning scale was modified by removing items related to dressing, bathing, incontinence, and using the toilet. Removing these items was done for two reasons. First, the primary focus of the MARRCC Physical Domain is to assess the resident’s physical mobility and ability to physically participate in leisure activities of choice; it does not address issues related to incontinence, bathing, dressing, or grooming as does the MOSES. second, the resident’s ability to participate in activities of daily living may be determined more by the degree of the staffs willingness to enable residents to engage in self-care than by the actual physical functioning level of the resident.
Intm-rater reliability. Intra-rater reliability was supported using a test-retest method. Test-retest reliability was examined by assessing participants on two separate occasions, with a time period of three days between assessments. This procedure was utilized to test the consistency of the MARRCC over time. A three-day interval was allotted between the repeated measures to reduce the possibility of test effect due to participant recall of initial response. Also, the three-day interval was purposefully chosen to help reduce the possibility that exogenous influences such as maturation, significant change in health status, and ongoing therapeutic intervention, would be causal factors for changes in functioning levels among the participants. The data for the test-retest reliability was collected by the same RSDs who collected the initial data three days prior.
Procedure
The RSDs from 11 LTC facilities were asked to assess participants utilizing both the MOSES and the MARRCC. The RSDs received detailed written and verbal instructions regarding how to obtain a random sample of participants (utilizing a table of random numbers) as well as timelines for the completion of the assessments. The only training the raters received for the implementation of the MARRCC was the information provided within the MARRCC manuals, ensuring the raters would have the same training as those who would ultimately use the MARRCC in the field. Raters were also provided with written instructions for the implementation of the MOSES-a second assessment of functioning for testing criterion-related validity. At each facility, the RSD was asked to rate six residents utilizing both the MARRCC and the MOSES on the same day. The RSDs were then asked to rate the same six residents three days later using only the MARRCC. Criterion-related validity and intra-rater reliability testing was completed in September, 2002.
Results
Criterion-related Validity of the MARRCC
Currently there are no universally accepted benchmarks for interpreting the magnitude of correlation coefficients. Therefore, the following is an approximate guide for interpreting the strength of relationships between two variables based on the absolute value of the coefficient: A correlation of ±1.0 is considered perfect, a correlation of ±.80 is considered very strong, a correlation of ±.50 is considered moderate, a correlation of ±.20 is considered weak, and a ±.00 indicates no correlation (O’Rourke, Hatcher, & Stepanski, in press).
Pearson Product Moment Correlation Coefficients for the MARRCC and MOSES domain scores were calculated (N = 64). Results suggest that three of the MARRCC domains had at least moderate correlation with the MOSES domain scores: Physical: r = .68 (p ≤ .0001); Cognitive: r = .88 (p ≤ .0001); and Social: r = .62 (p ≤ .0001). The Emotional Domain of the MARRCC had a correlation with the MOSES subscale approaching the moderate level (r = .39 (p ≤ .0015). Inverse relationships are a logical function of the directional wording of the individual scales and are not significant in the analysis of validity. Reported levels of association provide support for criterion-related validity of the MARRCC.
Intra-rater Reliability of the MARRCC
Utilizing the Pearson Product Moment Correlation Coefficient, correlation coefficients were calculated for each rater (N = 59). Data from one site was not included in the results of the test-retest study because the RSD did not complete the post-test. Results were as follows: Physical (r = .97), Cognitive (r = .99), Social (r = .98), and Emotional (r = .95). Statistically significant values were found for each of the raters (p ≤ .0001) indicating that the assessment results obtained by the MARRCC remain relatively unchanged over time in the absence of clinical intervention or marked exogenous occurrences.
Discussion
The MARRCC has been found to demonstrate content validity, inter-rater reliability, and intra-rater reliability. Criterion-related validity supports a strong correlation between the cognitive domains of the MARRCC and the MOSES as well as acceptable correlation between their physical and social domains. While the emotional domain demonstrates a lower correlation than anticipated, this finding is consistent with other measures of emotional functioning. Reliability testing of the MARRCC found it to be consistent among raters and consistent over time in the absence of intervention. Given these findings, the MARRCC offers a well-developed, theory-based assessment for the institutionalized elderly that is capable of assessing resident functional levels in each of four domains (physical, cognitive, social, and emotional) as related to recreation participation. Criterion-related validity correlation for the MARRCC Physical, Cognitive, and Social Domain scales compares favorably with corresponding measures of the MOSES. However, the MARRCC Emotional Domain demonstrated a relatively weak level of association. When interpreting the values for the MARRCC Emotional Domain, several factors should be considered. First, although the MOSES does represent a reliable and valid geriatric assessment, the Depressed/ Anxious Mood scale within the MOSES demonstrated an initial inter-rater reliability of .58 (Helmes et al., 1987), which is significantly lower than the inter-rater reliability of its other scales (Self-Care .97, Disorientation .84, Irritability .72, and Withdrawal .75). second, emotion and mood-related scales generally tend to have lower reliability than scales that rate more objective behaviors such as cognitive and physical functioning (Helmes et al., 1987). This is further clarified by Minium and Clarke (1982), who stated that “Except for certain areas of research, the relationships studied by psychologists and other behavioral investigators seldom give correlations as high or higher than .71. Thus, the proportions of variance typically accounted for often fall well below .50” (p. 128). Finally, it is also possible that the low correlation between the MARRCC Emotional Domain scale and the Depressed/Anxious Mood and Irritable Behavior scales of the MOSES was a result of the manner in which the latter two scales of the MOSES were combined to facilitate this particular part of the study. It is anticipated that future research efforts will be conducted to further examine the criterion-related validity of the MARRCC, as well as to determine the extent to which the MARRCC is sensitive to change over time (predictive validity).
Inter-rater reliability for the MARRCC (ranging from a high of .99 for the Cognitive Domain to a low of .97 for the Emotional Domain) indicates that recreation professionals can properly utilize and interpret the MARRCC data and score sheets with little or no assistance beyond instruction provided in the MARRCC manuals. For 10-item scales this is very acceptable, especially for raters who are relatively inexperienced in using the scale. Intra-rater reliability for the MARRCC (ranging from a high of .99 for the Cognitive Domain to a low of .95 for the Emotional Domain) indicates that the MARRCC is consistent over time in the absence of intervention or marked exogenous occurrences.
From a clinical perspective, utilization of the MARRCC can assist TR professionals in making a transition from the traditional approach to recreation programming in LTC-repetitive use of long standing successful programs, use of programs that are open to all, and use of programs that are primarily suited to residents with higher cognitive functioning levels-to a resident-centered approach. A critical aspect of such a transition is the TR professional’s ability to accurately assess each resident’s capabilities and needs. An accurate assessment enables the TR professional to design and implement therapeutic interventions that are matched to the functional abilities and limitations of the residents they serve, thereby increasing the likelihood that the unique and specialized needs of the cognitively impaired and physically frail residents will be met.
References
Atchlcy, R. D. (1977). The social farces in later life. Belmont, CA: Wadsworth Publishing Company, Inc.
Austin, D. R., & Crawford, M. E. (Eds.). (1996). Therapeutic recreation: An introduction (2nd Ed.). Ncedham Heights, MA: Allyn & Bacon.
Baltes, M., & Baltes, P. B. (1990). Successful aging: Perspectives from the behavioral sciences. New York & England: Cambridge University Press.
Baltes, M., & Carstensen, L. (1996). Process of successful aging. Aging and Society, 16(4), 397-422.
Brink, T. L. (1982). Screening tests for geriatric depression. Clinical Gerontologist, 1, 37-43.
Buettner, L., & Martin, S. L. (1995). Therapeutic recreation in the nursing home. State College, PA: Venture Publishing, Inc.
Burlingamc, J., & BIaschko, T. (1997). Assessment tools for recreational therapy: Red book #1. Ravensdale, VA: Idyll Arbor, Inc.
Dunn, J. K. (1989). Guidelines for using publishcd assessment procedures. Therapeutic Recreation Journal, 23(2), 59-69.
Dunn, 5. K., Sneegas, J., & Carrulhers, C. A. (1991). Outcome measures: Monitoring patient progress. In B. Rilcy (Ed.), Quality management: Applications for therapeutic recreation, (pp. 107-116). State College, PA: Venture Publishing, Inc.
Fillenbaum, G. G. (1988). Multidimensional functional assessment of older adults: The Duke Older Americans Resources and Services Procedures. Hillsdale, NJ: Lawrence Erlbaum Associates.
Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975). “Mini-Mental State.” A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research, 12, 189-198.
Granger, C. V., Albrecht, G. L., & Hamilton, B. B. (1979). Outcome of comprehensive medical rehabilitation: Measurement by PULSES Profile and the Barthel Index. Archives of Physical Medicine and Rehabilitation, 60, 145-154.
Granger, C. V., Hamilton, B. B., Linacre, J. M., Heinemann, A. W., & Wright, B. D. (1993). Performance profiles of the Functional Independence Measure. American Journal of Physical Medicine and Rehabilitation, 72. 84-89.
Harvey, R. F., & Jellinek, H. M. (1981). Functional performance assessment: A program approach. Archives of Physical Medicine and Rehabilitation, 62, 456-461.
Havighurst, R. J., & Albrecht, R. (1953). Older people. New York: Longmans Green.
Havighurst, R. I., Neugarten, B. L., & Tobin, S. S. (1968). Disengagement and patterns of aging. In B. L. Neugarten (Ed.), Middle age and aging, (pp. 161-172). Chicago, IL: University of Chicago Press.
Hays, J. C., Meador, K. G., Branch, P. S., & George, L. K. (2001). The spiritual history scale in four dimensions (SHS-4): Validity and reliability. The Gerontologist, 41(2), 239-249.
Helmes, E., Kaiman, G. C., & Short, J. (1987). Standardization and validation of the multidimensional observation scale for elderly subjects (MOSES). Journal of Gerontology, 42(4), 395-405.
Howe, C. Z. (1989). Assessment instruments in therapeutic recreation: To what extent do they work? In D. Compton (Ed.), Issues in therapeutic recreation: A profession in transition, (pp. 205-221). Champaign, IL: Sagamore Publishing.
Howe-Murphy, R., & Charboneau, B. G. (1987). Therapeutic recreation intervention: An ecological perspective. Englewood Cliffs, NJ: Prentice Hall.
Huck, S. W., Cormier, W. H., & Bounds, W. G. (1974). Reading statistics and research. New York, NY: Harper & Row.
Kane, R., & Kane, R. (1981). Assessing the elderly: A practical guide to measurement. Lexington, MA: Lexinglon Books, D. C. Health and Company.
Kaplan, M. (1979). Leisure, lifestyle and life-span. Philadelphia, PA: W. B. Saunders Company.
Kelly, J. R. (1983). Leisure identities and interaction. London: George Alien & Unwin.
Kloseck, M., & Crilly, R. (1997). Leisure Competence Measure: An introduction to measuring outcomes in therapeutic recreation. London, Ont: Marchard, Inc.
Lemon, B. W., Bengston, V. L., & Peterson, J. A. (1972). An exploration of the activity theory of aging: Activity types and life satisfaction among in-movers to a retirement community. Journal of Gerontology, 27, 511-523.
Linn, M. W. (1967). A rapid disability rating scale. Journal of American Geriatric Society, 15, 211-214.
Linn, M. W. (1988). A critical review of scales used to evaluate social and interpersonal adjustment in the community. Psychopharmacology Bulletin, 24, 615-621.
Longino, C. F., & Kart, C. S. (1982). Explicating activity theory: A formal replication. Journal of Gerontology, 37, 713-722.
McDowell, L, & Newell, C. (1996). Measuring health: A guide to rating scales and (juestionnaires, (2nd Ed.). New York and Oxford: Oxford University Press.
Minium, E. W., & Clarke, R. B. (1982). Elements of statistical reasoning. New York: John Wiley and Sons.
Neugarten, B., Havighurst, R., & Tobin, S. (1961). The measurement of life satisfaction. Journal of Gerontology, 16, 134-143.
Neugarten, B., Havighurst, R., & Tobin, S. (1968). Personality and patterns of aging. In B. L. Neugarten (Ed.), Middle age and aging, (pp. 77-102). Chicago: University of Chicago.
Olsson, R. H. (1990). The Ohio leisure skills scales for normal functioning: A systems approach to clinical assessment. Expanding Horizons in Therapeutic Recreation, XIIl. Columbia, MO: University of Missouri.
O’Rourke, N., Hatcher, L., & Stepanski, E. J. (in press). A step-by-step approach to using SAS for unlvariate and multivariate statistics, (2nd Ed.). New York: John Wiley & Sons.
Parker, R. A., Ellison, C. H., Kirby, T. F., & Short, M. J. (1975). The comprehensive evaluation in recreation therapy scale: A tool for patient evaluation. Therapeutic Recreation Journal, 9(4), 143-152.
Peterson, C. A., & Gunn, S. L. (1984). Therapeutic recreation program design: Principles and procedures, (2nd Ed.). Englewood Cliffs, NJ: Prentice-Hall.
Peterson, C. A., & Stumbo, N. J. (2000). Therapeutic recreation program design: Principles and procedures, (3rd Ed.). Needham Heights, MA: AIIyn & Bacon.
Rosen, W. G., Mohs, R. C., & Davis, K. L. (1984). A new rating scale for Alzheimer’s disease. American Journal of Psychiatry, 141, 1356-1364.
Schnelle, J. F., Wood, S., Schnelle, E. R., & Simmons, S. F. (2001). Measurement sensitivity and the minimum data set depression quality indicator. The Gerontologist, 41(3), 401-405.
Shank, J. W., Kinney, W. B., & Coyle, C. (1993). Efficacy studies in therapeutic recreation research: The need, the state of the art, and future implications. In M. Malkin & C. Howe (Eds.), Research in therapeutic recreation: Concepts and methods, (pp. 301-328). State College, PA: Venture.
Sneegas, J. J. (1989). Can we really measure leisure behavior of special populations and individuals with disabilities? In D. Compton (Ed.), Issues in therapeutic recreation: A profession in transition, (pp. 223-236). Champaign, IL: Sagamore Publishing.
Stumbo, N. J. (1996). A proposed accountability model for therapeutic recreation services. Therapeutic Recreation Journal, 50(4), 246-259.
Stumbo, N. J. (2001). Revisited: Issues and concerns in therapeutic recreation. In N. J. Stumbo (Ed.), Professional issues in therapeutic recreation: On competence and outcomes, (pp. 215-236). Champaign, IL: Sagamore Publishing.
Stumbo, N. (2002). Client assessment in therapeutic recreation services. State College, PA: Venture Publishing, Inc.
Stumbo, N. J., & Peterson, C. A. (1998). The leisureability model. Therapeutic Recreation Journal, 32(2), 82-108.
Witt, P., Connolly, P., & Compton, D. (1980). Assessment: A plea for sophistication. Therapeutic Recreation Journal, 14(4), 3-8.
Sienna Boothman, BS, CTRS, Geriatric Healthcare Consultants, Twain Harte, California; Keith Savell, Ph.D., CTRS, Geriatric Health Consultants, Oakland, California.
Sienna Boothman is now at Heritage Village, Chilliwack, British Columbia, Canada.
The authors would like to gratefully acknowledge Ken Maynard and Jocelyn Nelson for their assistance with the technical writing of this article.
Correspondence concerning this article should be addressed to Sienna Boothman at S iennasMarkLLC @ Yahoo.com.
Individuals wishing to access more information regarding the MARRCC are encouraged to visit www.SiennasMarkLLC.com.
Copyright National Recreation and Park Association Fourth Quarter 2004
Provided by ProQuest Information and Learning Company. All rights Reserved
