Developing and Assessing Students’ Entrepreneurial Skills and Mind-Set*

Developing and Assessing Students’ Entrepreneurial Skills and Mind-Set*

Bilén, Sven G


A primary goal of The Pennsylvania State University’s new Engineering Entrepreneurship (E-SHIP) Minor is to build students’ life skills so they can succeed within innovative, product-focused, cross-disciplinary teams. The E-SHIP Minor is designed for undergraduate students majoring in engineering, business, or IST (Information Sciences and Technology) who aspire to be innovation leaders for new technology-based products and companies. This paper outlines five E-SHIP program components to meet this mission: the core courses for the minor, E-SHIP competitions in which students exhibit their products and ideas, the E-SHIP Event Series, student organizations to support out-of-classroom entrepreneurial interest, and team projects for local industry and Penn State researchers. Penn State’s engineering entrepreneurship program is reviewed, summarizing both quantitative and qualitative assessment data to date, previewing future assessment plans, and providing a summary of lessons learned during the development and implementation of this program.

Keywords: engineering entrepreneurship, entrepreneurship program assessment, entrepreneurial skills


The Pennsylvania State University’s (Penn State’s) Engineering Entrepreneurship (E-SHIP) Minor is housed within the College of Engineering and operates in close collaboration with both the Smeal College of Business and the School of Information Sciences and Technology (IST) [1]. The broad goals of the E-SHIP minor are to provide students with multiple exposures to what it means to have an entrepreneurial mindset and to facilitate the development of both the passion and the ambiguity-management skills needed for new product or venture creation. The expected outcomes of the minor include an increase in motivation; improvement of communication, leadership, and teamwork skills; development of problem solving and innovative thinking skills; and a better understanding of business and financial knowledge. Research into and assessment of students’ growth in these skills and aptitudes are critical for three reasons. First, the skills listed above can be mapped directly onto the ABET Engineering Criterion 3 [2]. Success in entrepreneurship education means success in achieving Criterion 3’s challenging goals. Second, as the population of young, bright engineers grows in developing countries-where their salaries are typically lower than U.S. engineering salaries by a factor of five or so-corporations must perceive the value of retaining engineering jobs in the U.S. Engineers with the skills developed in this program will be of high value as corporate innovators as well as technical leaders. In addition, improved entrepreneurial skills such as commercializing technology should lead to significant economic development benefits to corporations and the U.S. economy as a whole. Finally, the faculty and administrations in engineering programs across the U.S. are launching new courses in technology entrepreneurship, often in collaboration with other disciplines such as business, liberal arts, and science. Membership in the Entrepreneurship Division of ASEE has grown from less than 20 in 2000 to over 500 members in 2004. Such rapid growth in a new area of engineering education should be researched and assessed.

Penn State’s Engineering E-SHIP Minor is part of a growing movement of technology-focused entrepreneurship programs. Led by the early and well-known program innovators at Stanford University [3] and MIT [4], a broad spectrum of colleges and universities have developed strong undergraduate engineering entrepreneurship programs or courses, including Lehigh University [5], Rose-Hulman Institute of Technology [6], University of Maryland [7], University of Central Florida [8], Rowan University [9], Worchester Institute of Technology [10] and Tri-State University [11]. Smaller colleges and universities, such as the University of Kansas-Salina, are also creating innovative grass-roots engineering entrepreneurship programs, showing that fostering student interest in entrepreneurship before establishing courses is a potential consensus-building approach leading to a mindset change for students and faculty.

A common feature across these engineering entrepreneurship programs, including Penn State’s, is their cross-disciplinary and the desire to focus their curricula and learning on technology entrepreneurship rather than a more general entrepreneurship program that may be offered by a business school alone.

Penn State’s Engineering Entrepreneurship Minor and related programs were designed in early 2001 using four inputs:

* a review of technology-focused entrepreneurship curricula and programs across the U.S., such as the ones listed above;

* suggestions and insights from experienced local technology entrepreneurs and from entrepreneur panels at meetings such as the REE (Roundtable for Entrepreneurship Education) Conferences held at yearly at Stanford University [12] ;

* the need to make wise re-use of selected existing Penn State courses; and

* budget constraints, the new courses in the minor were developed with a grant from the GE Foundation but ongoing support will largely come from endowments and grants as well as some institutionalized support.

In summary, the structure of Penn State’s E-SHIP Minor combines best practices from other engineering entrepreneurship programs with some new course innovations and the realities of launching a new academic minor in tight financial times. For example, Penn State followed the footsteps of Rowan University with a student venture fund, underwritten by the National Collegiate Inventors and Innovators Alliance (NCIIA). The E-SHIP Minor was one of the first programs to require students to take a crossskills course, which is described in section II. (This cross-skills effort has become part of the entrepreneurship program at Tri-State University.) Another unique feature of the program is the strong working relationship between the Smeal College of Business and College of Engineering in offering the E-SHIP Minor. Business faculty members teach the Business Basics course, ensuring top quality instruction and a mix of faculty from across Penn State actively involved in the E-SHIP Minor.

It should be noted that the desired impact for the Penn State E-SHIP Minor is help ensure each E-SHIP student has a new vision of what types of jobs he or she can hold, and that creating a job (and a company), rather than just getting a job, is feasible. For example, students see that being hired by an innovative, growth-oriented company will likely be highly rewarding, allow personal growth, and encourage networking and learning-keys to establishing the groundwork for becoming an entrepreneur. Granted, having a student or student team start a successful company while at Penn State is a sign of program success-and is exciting and welcome-it is not a specific metric by which the program is measured. We note that some students decide that becoming an entrepreneur is not for them, also a valid outcome of the program. We also note that the broader impacts of the E-SHIP Minor for the graduates will likely be seen years after graduation.

This paper outlines student opportunities in Penn State’s E-SHIP Minor and program, as well as the assessment program designed to measure their progress in becoming more entrepreneurial.


A. Core Courses and Faculty Characteristics

Starting in 2001, Penn State was able to develop and offer a core four-course sequence for engineering entrepreneurship with underwriting from the GE Fund. Each core course stresses problem-based learning. The E-SHIP Minor is open to all Penn State students interested in technology-focused entrepreneurship, resulting in classes that are a mix of majors, including students from engineering, business, liberal arts, and science. Not all students taking these courses are in the E-SHIP Minor; many take a course as an elective to explore technology-focused entrepreneurship. The cross-disciplinary goal for the E-SHIP Minor is supported by the two-path course flow shown in Figure 1, which affords the development of cross-disciplinary entrepreneurial skills. Engineering and IST students take ENGR 411, “Business Basics” (business finance, marketing, and business law and intellectual property). Business students take QMM 492, “Introduction to Engineering Design Principles.” These courses provide skills development that most students do not get in their respective majors, but that are critical to cross-disciplinary team success. Students are strongly advised to take the core courses in the order shown in Figure 1 because ENTR 430 is viewed as the capstone course in the minor, using the skills and knowledge gained in the previous core courses. Entry to the minor is requested after taking ENGR 310, and some flexibility in the order of the courses is provided so that students can fit the courses into their often crowded schedules.

A fundamental goal of the E-SHIP Minor from its beginning has been to conduct the core courses in a problem-based learning format to the maximum extent possible. Table 1 shows the student skills development focus for the core courses.

As part of the E-SHIP Minor, the students are expected to lead as well as participate in teams. Almost all work is performed within a team. With respect to communications skills, the students are exposed many times to techniques for improving their presentations and are critiqued almost continually (weekly or biweekly). To develop students’ presentation skills, the minor has hosted occasionally during class time a presentation advisor that works with the students. The materials employed during this workshop are used in all E-SHIP classes. In addition to these materials, we use in one of the courses (ENGR 407) a collection of video-taped business pitches that were provided by Stanford University. The students watch the video and an instructor stops periodically to draw out of the students what were good and poor points about the presentation. This helps the students be prepared and avoid some common pitfalls as they prepare for their presentations to experienced entrepreneurs and business-oriented people. Penn State’s E-SHIP students are compelled to make effective presentations on a routine basis. It is a constant of the curriculum.

With respect to ABET Engineering Criterion 3 for Program Outcomes and Assessment, Table 2 maps the criterion’s outcomes to the outcomes of the E-SHIP Minor. Criterion 3 states that accredited engineering programs must demonstrate that their graduates have an ability, understanding, or knowledge in 11 areas. The numbers in the E-SHIP Minor column are a measure of how well the E-SHIP Minor courses satisfy the subcriteria. The scale used is: 1, no support for meeting this subcriterion; 2, some support for meeting this subcriterion; and 3, very strong support to have students excel at this subcriterion. Table 2 shows that for 10 of the 11 subcriteria, Penn State’s E-SHIP Minor provides broad support for meeting these learning goals. A mapping like this can be a valuable tool for institutions to determine if a technology-focused entrepreneurship program or minor can help meet ABET engineering accreditation criteria.

The faculty members in the E-SHIP Minor possess a unique range of skills and expertise and act as instructors, coaches, and mentors in the courses. Seventy-five percent of the tenured and tenure-track engineering professors (two tenured and two tenure-track) who teach E-SHIP Minor core courses either have patents or have experience in technology-based venture creation. All of the other non-tenure track faculty (three from business and three from engineering) either have high-tech start-up experience, work in technology transfer, or are doing research involving next-generation technologies. Finally, a local technology entrepreneur co-teaches with engineering faculty in the Entrepreneurial Leadership and Technology-Based Entrepreneurship courses.

B. E-SHIP Competitions

An E-SHIP Competition is held at the end of each semester. Early in the development of the E-SHIP program, the faculty and program leaders made a conscious decision to steer the E-SHIP Competition away from being solely a business plan competition to more of a new venture competition. E-SHIP teams from three core courses (ENGR 310, ENGR 407, and ENTR 430) compete as part of the College of Engineering’s Product Design Showcase. Hence, students are part of a product solution team a minimum of three times as they complete the E-SHIP Minor. Deliverables required for each team at the E-SHIP Competition include:

* prototype of product/process/solution;

* five-minute “elevator pitch” to convince reviewer of the product’s value proposition;

* brochure describing product’s market niche; and

* business overview (condensed product or business plan).

Judges are drawn from local industry, small start-up technology companies, venture capitalists, and business development staff. It should be noted that the judges as asked to focus on the educational aspects of how the teams developed the product solutions, rather than simply the product’s likelihood of commercial success. Table 3 summarizes the number of teams that participated in the E-SHIP Competition at the end of each semester. Due to large numbers of student participants, a pre-competition was held in each core course beginning Fall 2002 to select the top teams to go on to the E-SHIP Competition Finals. As Table 3 shows, the E-SHIP Competition continues to evolve as we collect feedback from the judges, students, and faculty on ways to improve the learning experience. The number of teams in the E-SHIP Competition compares to approximately 50 teams per semester in the industry-sponsored capstone projects within EE, ME, and IE.

C. E-SHIP Events Series

The E-SHIP Events Series is composed of four events each semester that are designed to draw together students, faculty, and local entrepreneurs with meetings on E-SHIP topics of interest. Students in the first-year seminars, core courses, and Stage II Team courses are required to attend at least two of the four events each semester. Many students attend all four events, but because of other non-class commitments (e.g., employment, team meetings, student groups), attendance at all events is difficult. These meetings are never conducted as lectures. Instead, they include such diverse approaches as:

* Documentary Film, with panel comprised of local dot-corn entrepreneurs who reviewed the film, led a discussion, and fielded audience questions.

* Venture Capital Basics, with a panel including a venture capitalist and entrepreneurs who hold opposing views on the use of VC funds for company growth based on personal experience.

* Panel on Corporate Financial Scandals, including Penn State faculty in business law, corporate ethics, and a former Enron employee.

* “Life as a Technology Entrepreneur,” with two entrepreneurs relaying their start-up experiences, including how to balance life with work

* E-SHIP Collegiate Bowl, with a moderator providing questions related to entrepreneurship and current business events to two teams, one of students and the other faculty where each person was provided a “beeper” or horn to signal that they had the answer and points scored in different rounds.

D. Support for Student E-SHIP Interest

After the pilot year of the E-SHIP Program, faculty and students identified the need for a “Venture Fund” to provide support for more realistic prototyping of their technology product concepts. In addition, we saw that implementation of this type of venture fund is a component of other successful entrepreneurship programs [9]. In May 2002, the NCIIA awarded Penn State the E-SHIP Venture Fund and Competitions Grant. This award provides funds for E-SHIP student teams to extend their ideas through the purchase of prototype materials and work, such as:

* sheet metal, wood, plastic, etc.;

* electronic products (such as evaluation boards for RF, infrared, data collection);

* cables, connectors;

* development software;

* prototyping costs within Penn State (e.g., material for rapid prototyping machines).

Funds are disbursed on a sliding scale. First-Year Seminar teams receive an average of $50 per four-person team. E-SHIP Minor Core Course teams receive an average of $125 per four-person team. Stage II Teams (which are generally award-winning teams from the E-SHIP Minor Core Courses) receive an average of $500 per four-person team. At the completion of each project, materials are handled in one of three ways:

* materials stay with the student team if further product/business development is planned either as a Stage II Team (in ENGR 496), or the team continues work with the Penn State’s SIFE (Students in Free Enterprise) student group;

* materials, especially electronic equipment, are returned to the E-SHIP Program to become E-SHIP Minor prototype material “stock”; or

* prototypes from disbanded teams are either kept as example prototypes, or recycled/discarded.

The process for obtaining any Venture Funds is competitive. Each student team must first submit a Venture Fund Request specifying the use of money, estimate of cost based on research, timeline for prototyping, and description of final prototype’s use. Funds are released by evaluating if innovation is high, product need exists, and the design can lead to entrepreneurial opportunities such as possible launch of a new company or development of entrepreneurial skills. Students are advised that the Venture Funds are not simply play money, but rather a unique resource. To underscore this advice, the students are made aware that if the prototyping process produces little or no results, either the team’s funding will be pulled (via refund of the cash amount), or the course grade reduced by a predefined (and agreed to) amount.

In addition to the funds available to the students, there are significant additional resources available if needed, including fabrication facilities (the Learning Factory and Center for Engineering Design and Entrepreneurship-CEDE), meeting space (through CEDE), and computational resources (through CEDE and university/departmental computer labs).

E. Team Projects and Interaction with Local Industry and University Researchers

A second NCIIA grant was awarded in May 2003 to support the development and pilot teaching of a new course entitled “Market-Pull Technology Transfer,” cross-listed as a business and engineering course (BA/ENGR 497E). Business and engineering faculty co-taught the course in Fall 2003 and Spring 2004, with undergraduate teams of engineering, business, and pre-med students. Teams focus on a technology kernel from within Penn State and provide guidance to the Technology Transfer Office on best approaches for commercialization; intellectual property protection and leverage; and licensing and option agreements. Student teams also provide suggestions on ways to improve the technology transfer process through improved invention screening; educating faculty on government (e.g., FDA) procedures; and providing ways to improve the non-confidential teasers for the inventions. The goal is to institutionalize the course to be of benefit to E-SHIP students, research faculty, the Technology Transfer Office, and corporations.

With respect to other industrial involvement with the program, the E-SHIP Minor from its inception has included inputs from large and small companies: people from industry have served as judges in the E-SHIP Competitions each semester, entrepreneurs have been the core of the E-SHIP events series, and presentations are made each semester to the Leonhard Center Advisory Board (with corresponding suggestions as to the direction of the E-SHIP Minor). To tighten the guidance and input from entrepreneurs, a new E-SHIP Program Advisory Board was established in Fall 2004, comprised of 15 successful entrepreneurs.


A. Program Goals and Objectives

As part of the original grant proposal, Penn State agreed to create an extensive assessment plan including both formative and summative elements. The minor was assessed formatively in order to determine what changes needed to be made. In addition, summative assessment was performed in order to determine whether the minor was meeting intended programmatic goals [13]. In order to assess the program, the desired areas of student growth, as displayed in Table 1, were considered and used in the development of four assessment questions, which follow:

1. How does the minor affect students’ motivation and self-efficacy? (Motivation/Needfor Achievement)

2. Are students more successful in tackling ambiguous problems and thinking innovatively? (Innovation)

3. Are students more likely to see the connections to aspects of problems outside those related to their individual discipline, especially relating to business and finance? (Business Planning Skills, Customer Orientation)

4. Do students exhibit other necessary skills to become an entrepreneur? (Teamwork, Communication Skills, Leadership, Risk-Taking)

B. Methods of Assessment

In order to evaluate the minor, a mixed-methods approach was utilized consisting of both qualitative and quantitative types of data collection [14]. Using mixed-methods designs allowed for a more pragmatic approach to answering the evaluation questions. As Eeydens et al. state, “[Q]uite often the assessment of engineering education has both…the need for detailed information concerning a subset of individuals and for generalization across the population” [15, p. 70]. The qualitative information collected provided much rich and detailed information regarding the potential benefits of the minor. However, the quantitative surveys allowed for a larger sample size of students to be collected with the expectation that the results would be generalizable to the larger population. In addition, qualitative data was found to be extremely valuable in providing information on some of the constructs that proved difficult to measure. Using the terminology of Cresswell et al., a concurrent triangulated design was utilized, administering quantitative survey methods to all respondents and then collecting qualitative information from a smaller subsample of these respondents [16].

The primary tool for collecting information was an online instrument that contained questions concerning demographic information as well as various rating scales including the Engineering Self-Efficacy scale [17] and the Leadership Attitudes and Beliefs Inventory [18]. Responses were linked electronically with a college database, which provided additional demographic information such as GPAs and SAT scores. This instrument was administered via a Web-based format at the beginning and end of each semester to students enrolled in the core courses, with a response rate of 55 percent. Completion of the survey was voluntary and was not performed during class time. In addition, the survey was administered at the end of a semester to a comparison group of students who were not enrolled in any of the core courses. Other measures administered to participating students included pre- and post-tests of content knowledge and in-class rating scales of self-perceived characteristics.

To supplement the quantitative data from the above instruments, a total of twelve focus groups comprised of students enrolled in the core courses were held during the Spring 2003 and 2004 semesters [19]. The purpose of these focus groups was to collect students’ perceptions to be used for course improvement purposes as well as an indication of whether or not the program’s goals were being met. Because of the lengthy nature of the qualitative data, only selected results have been included in this paper. Additional findings of the focus groups are discussed in Okudan and Rzasa [20].

The data analyses for the qualitative and quantitative data were conducted independently. The focus group data were analyzed using a grounded theory approach [21]. Initial themes were hypothesized based on the evaluation questions. Additional themes were added and revised as necessary through open coding of the documents. The results of the qualitative data were directly compared with the results of the quantitative data to see if the statistical trends could be supported by the emergent qualitative themes [16].

C. Demographic Information

The demographic data obtained through the online instrument are displayed in Table 4.

Several statistical comparisons were performed to allow us to make inferences as to how representative is the responding sample. First, E-SHIP students were compared with the College of Engineering population. Participants in the E-SHIP courses tended to have higher math and verbal SAT scores than the general population (math: t = 3.736, d[function of] = 372, p = 0.000; verbal: t = 3.010, d[function of] = 372, p = 0.003). The actual mean difference was 15 points in math and 13.52 points in verbal. Entrepreneurship students also were likely to be younger than the college population (mean difference = 0.93, t = 11.416, d[function of]= 429, p = 0.000). However, given the range of ages in the larger college population (17-65), it is possible that the ages of certain nontraditional students may be serving as outlying data points. This could potentially affect the results of the t-test. Finally, the E-SHIP courses drew proportionately more females than the college as a whole (χ^sup 2^ = 9.660, d[function of] =1, p = 0.002). Although we have no current definitive explanation as to why this is the case, we have noted a similar larger proportion of females in Penn State’s Engineering Leadership Development Minor. Future research will seek to understand this phenomenon.

In comparing the respondent group to the non-respondents who had participated in E-SHIP courses, they were found to be statistically similar in semester standing, verbal and math SAT scores, and GPA. Non-respondents tended to be older as a group than respondents (t = 3.223, d[function of] = 428, p = 0.001). Differences based on gender were noted, but not statistically significant (χ^sup 2^ = 3.086, d[function of] = 1, p = 0.079).

There were no statistically significant differences between the control group (n = 32) and the respondents in verbal and math SAT scores, gender, or semester standing. The respondents tended to be about one year older than the control group (t = 2.698, d[function of] = 239, p = 0.007).

D. Assessment Results for Attributes and Tendencies

Using both qualitative and quantitative data, the evaluation plan attempted to measure four questions relating to the program’s goals. Each of these is discussed below.

1) How does the minor affect students’motivation and self-efficacy? Three aspects of motivation were investigated through a combination of qualitative and quantitative methods. First, are students more motivated in the academic environment? Second, do students feel more motivation to become entrepreneurs in the future? Third, do students have a higher self-efficacy or confidence in their ability to become entrepreneurs?

Data related to the effects of the minor on students’ academic motivation were primarily collected through the focus group information. Students discussed the E-SHIP Minor as being a welcome change of pace from their traditional engineering classes. This change of pace, brought on by the interactive, problem-based learning environment, was found to be motivating for some students. One student in particular expressed that the E-SHIP Minor had renewed her interest in engineering. A very common theme in the focus groups was the sentiment that the courses offer a break from the traditional, more technical classes. In addition to improved motivation in the academic setting, several students noted that they became motivated to be entrepreneurs in the future. The speakers at E-SHIP sponsored events were also often noted as being inspiring as students could see “how successful they were with their own idea and how far they’ve gotten.”

Another aspect of motivation required in individuals wanting to become entrepreneurs is self-confidence and a feeling of empowerment. Students need to believe that it is possible to become an entrepreneur. Both the qualitative and quantitative information helped to answer this evaluation question. The online instrument included an Entrepreneurial Self-Efficacy scale-a 22-item Likert-type scale-designed to measure respondents’ confidence in their own abilities as entrepreneurs [17], which showed a noticeable trend towards a higher mean for the late respondents, with early respondents having a mean of 3.687 (s.d. = 0.4761) and later respondents showing a mean of 3.870 (s.d. = 0.3954). While this difference approached statistical significance (t = 1.724, d[function of] = 197, p = 0.086), the small size of the difference mitigates its usefulness. A small group of students (n = 17) consented to a second administration of the online instrument. A paired t-test was used to compare the mean scores for this group. Although there was a slight gain in mean score (pre-test mean = 3.899, post-test mean = 4.000), the small sample size prevented the attainment of statistical significance (t = 0.709, d[function of] = 9, p = 0.496).

While there were statistical trends that the students in the minor had a higher self-efficacy, no definitive statements could be made regarding this difficult-to-measure characteristic. Therefore, the notion of self-efficacy and empowerment was further explored with the focus groups. Many students overwhelmingly emphasized the effects of the minor on their self-confidence to become an entrepreneur. The focus group from the ENGR 407 course seemed to have the most students who reported feeling empowered. A particularly poignant statement by a student was the following:

“Here’s the typical scenario. I hear my classmates saying, ‘It’ll be really cool to work for a certain company.’ And my thought is, ‘hmmm, maybe I’ll hire you someday.'”

The constructs of motivation and self-efficacy are somewhat difficult to measure quantitatively. Not surprisingly, although the survey data show trends that E-SHIP students have a higher self-efficacy for entrepreneurship, no definitive claims can be made. However, the qualitative information seems to support that many of the participating students have higher motivation both in academic settings and for a future entrepreneurial career.

2) Are students more successful in tackling ambiguous problems and thinking innovatively? The ability to tackle ambiguous problems and to think innovatively is another important aspect of an entrepreneur. Once again, these characteristics were measured through both quantitative and qualitative methods. On a self-report in ENGR 407, students were asked to rate their level of creativity at the beginning and the end of the course on a scale of 1 to 5, with 5 being highest. A paired t-test found a significant difference between the scores (pre-test mean = 2.786, post-test mean = 3.5, t = 3.68, p = 0.003). Although a similar survey in the Stage II course did not find a significant difference for creativity (pre-test mean = 2.18, post-test mean = 2.64, t = 1.838, p = 0.096), the students’ perception of their ability to generate ideas was significantly different for the pre- and post-test (pre-test mean = 2.20, post-test mean = 3.00, t = 3.207, p = 0.011).

This issue was also explored during the focus groups. Once again, students in the ENGR 407 class expressed that the E-SHIP Minor helped to improve their ability to think creatively. Many students often found themselves in an unfamiliar classroom situation where they were expected and encouraged to “think outside the box” and “step out of their comfort zones.” The students often described the course as a lesson in creativity where they learned what it feels like to fail-and that failing wasn’t the end of the world. One student quantified the impact to his creativity as being “in spades.” He described the instructor as someone who will “force creativity out of you if he has to crush you into a meat grinder to find it.” These students described interesting self-designed projects ranging from creating a pinata shaped liked the professor’s head to holding Jello® wrestling matches, all with the intention of recognizing the possibility of making money in creative and innovative ways.

Although the ENGR 407 course seemed to have a strong impact on the creative skills of the students, other courses seemed to have less of an effect. This difference is perhaps the result of the emphasis that ENGR 407 places on creative thinking. Thus, it is not surprising that other classes, which have different course objectives, have less of an impact.

Throughout the E-SHIP courses, students expressed a perceived improvement in problem-solving skills. However, even with this self-perceived improvement in problem-solving skills, many students expressed a resistance to the unstructured, open nature of the courses. Several of the assignments in the courses are somewhat vague and open-ended. For example, in the ENGR407 classes, students are asked to create a portfolio that best characterizes them. This assignment, which came along with very limited directions, often frustrated the students. Many times when students were asked how the class they attended could be improved, they mentioned they would like to have more guidance.

3) Are students more likely to see the connections to aspects of problems outside their major disciplines, especially relating to business and finance? At this point in time, the primary source of information concerning content knowledge in the E-SHIP minor is through focus group data. Although tests of content are administered in some of the courses by the individual professors, this information has not been analyzed for the evaluation of the minor. Judges’ comments regarding student teams at the Product Showcase have suggested that students are learning necessary skills and techniques to create a business plan and “elevator pitch.” A qualitative review of these comments revealed that most teams were impressive, although some could still use improvement on various skills. For example, one judge wrote about a particular student team as having “an in-depth business plan.” However, this judge did note that some student teams “were more serious than others.”

From the focus group data, students in the more business-oriented courses report a definitive increase in the amount of knowledge they have attained relating to business and finance, although this is less apparent from students enrolled in the ENGR 407 course. The focus groups for this class revealed that students have mixed feelings on what they are learning. Some students noted the material learned in ENGR 407 was not quantifiable. Rather, the students think that they are learning more about life experiences. As one student described his learning relating to an interesting group project:

“I feel like if I had to write down something I learned that I wouldn’t be able to do it. But some experience, some situation that I’m going to find myself in a month from now, a year from now, ten years from now, it’s going to go back to what did we do [on a class project]?'”

In addition to the factual and experiential knowledge gained in the minor, students discussed the opportunity to work with students from other disciplines as being important for their later careers. Students found that they were exposed to individuals from other educational backgrounds in the E-SHIP courses. These included students from business, information sciences and technology, as well as other engineering majors.

One complaint that students had, however, was the limited number of business students who enrolled in the courses. Often, only one or two enrolled students were business majors. This observation came up repeatedly in almost all of the focus groups. Recently, however, we have seen a shift toward higher numbers of business students, which we attribute to the cross-listing with business of a new course called “Introduction to Entrepreneurship.” We will continue to examine enrollment trends.

4) Do students exhibit other necessary skills to become an entrepreneur? Leadership, communication, and teamwork skills are stressed throughout the curriculum, as students are expected to complete many presentations and team-based projects, culminating with the Product Showcase. Students’ perceptions of these skills were assessed through both surveys and the focus groups.

To measure the impact of the minor on leadership abilities, the Leadership Attitudes and Beliefs Scale completed the online suite of instruments [15]. Consisting of 28 statements concerning organizational leadership, this scale was found to have good reliability with α = 0.7998. Results for the Leadership Practices Inventory were collected from the E-SHIP sample, the control group, and a small post-test sample (n = 13). A comparison of the entrepreneurship students to the control group showed no statistical significance (t = 1.075, d[function of] = 230, p = 0.284). The control group (mean = 3.758) actually scored slightly higher than the E-SHIP sample (mean = 3.688). This is most likely due to the difficulty in isolating the development of attitudes towards leadership to a particular course experience. No significant differences were found when comparing early-semester to late-semester responders, or when comparing pre- and post-tests for entrepreneurship students.

The students’ perception of leadership revealed through the focus groups provided mixed results. Some students thought that the ENGR 310 class did help with encouraging leadership. While this class seemed to influence the perception of leadership skills positively, the open-ended structure of the ENGR 407 course seemed to deter students from taking a leadership position in many instances. More data collection will need to be performed in order to better understand the impact of the E-SHIP Minor on leadership skills.

For the impact of the E-SHIP Minor on communication skills, students were asked to rate their presentation skills at the beginning and the end of the course on a scale of 1 to 5, with 5 being highest. A paired t-test found a significant difference between the scores (pre-test mean = 2.14, post-test mean = 2.89, t = 5.51, p = 0.000). The same survey was given in the Stage II course. A paired t-test also found a significant difference (pre-test mean = 2.55, post-test mean = 3.36, t = 4.5, p = 0.001).

The focus group information confirmed that students did perceive an improvement in their presentation skills from the E-SHIP classes. The Product Showcase provided an additional opportunity for students to display their presentation skills by pitching their product or service to a panel of local judges from the community. The students felt that this added pressure could indeed be beneficial for the development of their presentation skills.

The impact of the E-SHIP Minor on teamwork skills was examined through both surveys and the focus groups. On a self-report in ENGR407, students were asked to rate their teamwork at the beginning and the end of the course on a scale of 1 to 5. A paired t-test did not find a significant difference between the scores (pre-test mean = 2.92, post-test mean = 3.357, t = 1.71, p = 0.111). The same survey in the Stage II course also did not find a significant difference (pre-test mean = 3.09, post-test mean = 3.45, t = 1.789, p = 0.104.)

Judging from the focus group information, some students felt that the courses did allow their teamwork skills to improve. Many participants noted that because there were students from a variety of majors in the courses, they had the opportunity to work with people from different academic backgrounds. However, some students noted that many of their other non-entrepreneurship courses provided opportunities for group work and that the entrepreneurship courses were not unique in that respect.

F. Thoughts and Future Directions Regarding Assessment

Observed scores in all measures showed trends in the expected direction in pre- and post-tests, despite failing to rise to the level of statistical significance in many instances. In comparisons with a control group (consisting of similar students who had not taken an E-SHIP course), the E-SHIP student was likely to score slightly higher on Self-Efficacy but not on the Leadership Aptitudes and Beliefs Inventory, as noted above. Again, these findings did not meet the requirements of statistical significance, which means that they cannot stand alone and declare that actual changes in these attributes have taken place over the course of a semester or throughout the minor experience. In order to provide support for these identified trends, we have provided qualitative data collected through focus groups. This data provides evidence that the developmental trends may be an accurate reflection of the effects of both the course and the minor.

The respondent rate of the E-SHIP students was consistent with similar efforts. It was more difficult than expected to obtain a fair-sized comparison group and to manage the timing of test administration. However, there could be potential differences between the students from the E-SHIP Minor who responded to the survey and the comparison group of students. The students in minor self-select to enroll in the E-SHIP courses. These students potentially already have an interest in entrepreneurship. They potentially also could have different characteristics, such as stronger leadership abilities or creativity.

Due to budgetary constraints, only the online instrument was administered to a very small comparison set of students. The comparison group was not administered the surveys relating to communication, teamwork, and creative-thinking skills. A better understanding of these constructs might be obtained by administering surveys to both the E-SHIP and a similar group of non-participating students. While there were no significant differences in SAT scores, gender, or semester standing, we did not control for the grades of the students. The best type of data collection technique would be to administer the surveys to a larger random sample, which could potentially provide information on the pre-existing differences between E-SHIP students and the remaining engineering student population.

An additional concern of the assessment included the potential self-selection bias of the respondents versus non-respondents. Because the respondents voluntarily choose to participate in both the survey and the focus groups, the participants could in fact have different characteristics than non-participants. Perhaps those students who participated were more motivated or were high-achievers. While the SAT scores are similar for respondents versus non-respondents on the survey, a future course of action might be to compare final course grades in the E-SHIP courses.

Additionally, administering the surveys online potentially decreased the response rate. In-person, in-class administration of the instruments would probably have increased participation and perhaps yielded enough statistical power to gain significance, but the use of the online version was deemed preferable as being less intrusive. Also, while an increased response rate could have been achieved by administering the surveys in person or by giving extra credit for participation, a potential concern was that students could be enrolled in more than of the E-SHIP courses during a particular semester. Therefore, students could be asked to take the survey in more than one class. If extra credit had been offered, it would be difficult to set up criteria for which course the points would be allocated.

As the trends all appear to be in the hypothesized direction, it seems likely that an improved administration of the instruments will result in statistically significant findings. Qualitative data do provide support to the relatively weak quantitative findings. Students were consistently positive toward the courses, while honestly expressing some reservations. Given these limitations to the quantitative data, which must be considered, it still does seem that participation in the E-SHIP courses does have a positive effect on the students’ development in the several identified areas.

In many cases, the focus group information seemed to provide a clearer picture of the effects of the E-SHIP minor. One potential concern with our data, however, is that we do not have proportions for students who agreed with each individual question. In other words, we do not have an exact percentage of students who believed that the minor was beneficial in each area. This is due to the nature of focus groups. Not every student answers each individual question, which makes calculating proportions difficult. A future analysis will include collecting frequency data for each emergent theme.

Another weakness of the current assessment plan is the over-reliance on self-report data. Although other types of data have been collected-such as online portfolios, examples of business plans, and open-ended tests of course content-analyzing this type of data has proven to be challenging. This part of the assessment plan requires further development in the form of valid rubrics and training of raters.

An additional future action is to explore the findings related to gender. The E-SHIP program at Penn State has been shown to draw proportionately more females than are enrolled in the college as a whole. At this point in time, we cannot determine the reason why the program draws a high proportion of females. However, other programs at Penn State, such as an Engineering Leadership Minor, have reported similar findings. Future investigation into this phenomenon could perhaps yield interesting results for other institutions.

Some of the effects of the E-SHIP program may not be immediately evident. The hope is that students who participate in the courses will come away with knowledge and skills that may influence their future career decisions. In order to best evaluate the effects of the ESHIP program, assessment must occur after students enter the workforce. With the impending graduation of the first cohort of students, extended assessment of alumni is being developed.


Penn State’s E-SHIP Minor has achieved the milestone of institutionalization and now has multiple new milestones to reach. Based on two years of GE Fund program time and three semesters of E-SHIP course offerings, the following are our principal lessons learned for new engineering or technology E-SHIP programs:

1. Implement assessment as early as possible. Having data from very early in the E-SHIP program on student growth in E-SHIP skills areas is a powerful tool for guiding the E-SHIP program, grant writing, and requesting funding from within the institution.

2. Encourage students that they can define a new great product with huge potential to meet a product need. Set the standard high for quality of new product and venture ideas, and the students will respond (albeit after initial doubt).

3. Provide E-SHIP students multiple experiences in diverse teams, high-pressure presentations, and tough questioning by entrepreneurs. Handling these situations with success and confidence separates the engineering entrepreneur from the “standard” engineering student.

4. Look for “starter technologies” within your own institution, and have E-SHIP student teams work with the researcher(s) to develop new products, companies, and strategic alliances. This opportunity is unique to the university environment. Use it to your program’s advantage. It is a win-win situation for all stakeholders (researcher, student, institution, and E-SHIP program).


The authors and the Penn State College of Engineering recognize and appreciate the support of the GE Foundation and the National Collegiate Inventors and Innovators Alliance for inter-disciplinary technology entrepreneurship education. The assessment section of this paper draws heavily from the final report on the Problem-Based Learning in Entrepreneurship project that was funded through the GE Learning Excellence Fund (Summer 2003).

* This article is an expansion of a manuscript presented at the 2003 National Collegiate Inventors and Innovators Alliance Conference.


[1] The Pennsylvania State University Engineering Entrepreneurship Education (E3),

[2] Accreditation Board for Engineering and Technology (ABET) 2000, Criterion 3: Program Outcomes and Assessment,

[3] Miller, S.J., Doshi, R, Milroy, J.C., and Yock, P.G., “Early Experiences in Cross-Disciplinary Education in Biomedical Technology Innovation at Stanford University,” Journal of Engineering Education, Vol. 90, No. 4, 2001, pp. 585-588.

[4] Banzaert, A., Lokuge, P.D.H., Smith, A., and Susnowitz, S., “The MIT IDEAS Competition: Innovations for Public Service,” 7th Annual Meeting of the National Collegiate Inventor and Innovators Alliance, Boston, Mass., March 2003.

[5] Ochs, J.B., Watkins, T.A., and Boothe, B.W., “Creating a Truly Multidisciplinary Entrepreneurial Educational Environment,” Journal of Engineering Education, Vol. 90, No. 4,2001, pp. 577-583.

[6] Berry, F., Moore, D., and Mason, T., “Continuous Improvement in Entrepreneurship and Engineering Design,” 7th Annual Meeting of the National Collegiate Inventor and Innovators Alliance, Boston, Mass., March 2003.

[7] Thornton, K, Djamshidi, S., and Barbe, D., “VentureAccelerator Program: Accelerating Fledging Technology Start-ups at University of Maryland,” 8th Annual Meeting of the National Collegiate Inventor and Innovators Alliance, San Jose, Calif, March 2004.

[8] D’Cruz, C., and O’Neal, T., “Turning Engineers into Entrepreneurs,” 7th Annual Meeting of the National Collegiate Inventor and Innovators Alliance, Boston, Mass., March 2003.

[9] Marchese, A.J., Schmalzel, J.L., Mandayam, S.A., and Chen, J.C., “A Venture Capital Fund for Undergraduate Engineering Students at Rowan University,” Journal of Engineering Education, Vol. 90, No. 4, 2001, pp. 589-596.

[10] Banks, M.C., “Experience with an Entrepreneurship Minor for Engineering,” Teaching Entrepreneurship to Engineering Students Conference, Monterey, Calif, January, 2003.

[11] Finley, D.R., Enneking, T.J., and Tichenor, D.M., “Developing an Entrepreneurial Framework at Tri-State University,” 8th Annual Meeting of the National Collegiate Inventor and Innovators Alliance, San Jose, Calif, March 2004.

[12] Many of these programs were overviewed at the Roundtable on Entrepreneurship Education held at Stanford University on Oct 22-24, 2003. Posters maybe downloaded from

[13] Popham, W.J., Educational Evaluation, Needham Heights, Mass.: Allyn and Bacon, 1993.

[14] Teddlie, C., and Tashakkori, A., “Major Issues and Controversies in the Use of Mixed Methods in the Social and Behavioral Sciences,” Handbook of Mixed Methods in Social and Behavioral Research, Eds. Tashakkori and Teddlie, Thousand Oaks, Calif.: Sage Publications, 2003.

[15] Leydens, J.A., Moskal, B.M., and Pavelich, M.J., “Qualitative Methods Used in the Assessment of Engineering Education,” Journal of Engineering Education, Vol. 93, No. 1, 2004, pp. 65-72.

[16] Cresswell, J., Piano Clark, V.L., Gutmann, M.L., and Hanson, W.E., “Advanced Mixed Methods Research Designs,” Handbook of Mixed Methods In Social and Behavioral Research, Eds. Tashakkori and Teddlie, Thousand Oaks, Calif.: Sage Publications, 2003.

[17] Chen, C.C., Greene, P.G., and Crick, A., “Does Entrepreneurial Self-efficacy Distinguish Entrepreneurs from Managers?,” Journal of Business Venturing, Vol. 13,1998, pp. 295-316.

[18] Wielkiewicz, R.M., “The Leadership Attitudes and Beliefs Scale; An Instrument for Evaluating College Students’ Thinking About Leadership and Organizations,” Journal of College Student Development, Vol. 41, No. 3, May/June 2000, pp. 335-347.

[19] Rzasa, S.E., Wise, J.C., and Kisenwether, E.G., “Evaluation of Entrepreneurial Endeavors in the Classroom: The Student Perspective,” 8th Annual Meeting of the National Collegiate Inventor and Innovators Alliance, San Jose, Calif., March 2004.

[20] Okudan, G.E., and Rzasa, S.E., “Teaching Entrepreneurial Leadership: A Report on Work in Progress,” 2004 Annual Meeting of Frontiers in Education, Savannah, Ga., October 2004.

[21] Strauss, A., and Corbin, J., Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory, Thousand Oaks, Calif: Sage Publications, 1998.


Engineering Design Program

The Pennsylvania State University


Engineering Entrepreneurship Program

The Pennsylvania State University


Schreyer Institute for Teaching Excellence

The Pennsylvania State University


Engineering Instructional Services

The Pennsylvania State University


Sven G. Bilén received the B.S. degree from The Pennsylvania State University in 1991 and the M.S.E. and Ph.D. degrees from the University of Michigan in 1993 and 1998, respectively. In January 2000, he joined Penn State as an assistant professor in Engineering Design and Electrical Engineering. Dr. Bilén is the Design Curriculum Coordinator in the Engineering Design Program within SEDTAPP and as such is responsible for developing, defining, funding, and coordinating the industry-sponsored design projects used in all sections (approx. 1000 students per year) of ED&G 100: Introduction to Engineering Design. Dr. Bilén teaches in the Engineering Entrepreneurship Minor (ENTR430: Entrepreneurship and New Product Development). He is a Co-PI for the PRESTIGE (Preparing Engineering Students to Work in the Global Economy) consortium. He has mentored several undergraduate students and teams interested in entrepreneurship. Dr. Bilén is member of IEEE, AIAA, AGU, ASEE, and Sigma Xi.

Address: School of Engineering Design, Technology and Professional Programs, 213N Hammond Building, University Park, Pennsylvania, 16802; telephone: (814) 863-1526; fax: (814) 863-7229; e-mail:

Elizabeth C. Kisenwether holds a B.S. degree in Electrical Engineering from Penn State (1979), and M.S.E.E. degrees from Massachusetts Institute of Technology (1981) and The Johns Hopkins University (1988). She worked in industry for 11 years with a large defense contractor, and then co-founded and worked for five years with a high-tech startup. Since joining Penn State in 1999, Prof. Kisenwether has taught design-focused courses in her home department as well as the Mechanical, Electrical, and Civil and Environmental Engineering Departments. Currently she is the program lead on two NCIIA grants (2002-2005) and involved in a Kauffman Foundation grant with Srneal College of Business developing a new Introduction to Entrepreneurship course. Prof. Kisenwhether also is President and founder of KidTech, Inc, a non-profit engineering outreach company developing hands-on design and problem-based learning kits and activities for K-12 youth ( and is the Chair of the Entrepreneurship Division of ASEE for 2004-2005.

Address: School of Engineering Design, Technology and Professional Programs, 213D Hammond Building, University Park, Pennsylvania, 16802; telephone: (814) 863-1531; fax: (814) 863-7229; e-mail:

Sarah E. Rzasa is the Teaching and Learning Assessment Specialist at the Schreyer Institute for Teaching Excellence at Penn State. She received her B.A. in Psychology from the University of Connecticut and her M.S. in Educational Psychology specializing in Tests and Measurement. She previously served as a research assistant for Engineering Instructional Services and the Leonhard Center for the Enhancement of Engineering Education. She is currently a doctoral candidate in the department of Educational Psychology.

Address: Schreyer Institute for Teaching and Learning, 301 Rider II Building, University Park, Pennsylvania, 16802; telephone: (814) 863-9094; e-mail:

John C. Wise is the director of Engineering Instructional Services at Penn State’s College of Engineering. In this capacity, he provides assistance to faculty members and teaching assistants in the areas of teaching, learning, and instructional technology. He also provides educational assessment support for the College of Engineering. He received his B.A. in Liberal Arts from The University of the State of New York and his M.S. and Ph.D. in Instructional Systems from Penn State.

Address: Engineering Instructional Services, 201 Hammond Building, University Park, Pennsylvania, 16802; telephone: (814) 865-4016; fax: (814) 865-4021; e-mail:

Copyright American Society for Engineering Education Apr 2005

Provided by ProQuest Information and Learning Company. All rights Reserved