Understanding performance evaluation

In tourism, performance evaluation involves objectively assessing projects completed to date according to a predefined plan, whether this relates to a policy, a program or a strategy. Such an evaluation relies on clearly defined goals, pertinent indicators and a number of information sources that are too often incomplete.

Various information sources

Databases of strategic information are produced by various sources. For example, Statistics Canada supplies a wide range of tourism-related statistics. Tourisme Québec conducts studies and research on specific products, and certain tourist regions have developed performance indicators for accommodations, attractions, activities, events, business travel, and so on. However, the information obtained can be several years out of date – this is often true of statistics – making it of little use for strategic purposes. Furthermore, since methodology and data collection can differ from one source to the next, data are often difficult to compare.

An important strategic tool

To clearly understand tourist behaviour on a continuous basis, and measure monthly and annual changes in various tourism sectors, it is vital that information be obtained in as close to ?real time? as possible to encourage effective decision-making. Therefore, data must be produced rapidly and – more importantly – be integrated into a marketing plan or development process that enables managers to better evaluate their performance in terms of their own objectives.

An ongoing cycle

The evaluation process begins when goals are set for the plan, strategy or program. This step is crucial because it creates a foundation for developing evaluation mechanisms and selecting performance indicators. Of course, the goals set must reflect the concerns of the entire industry. In addition, well-defined goals share some basic characteristics: they are specific, measurable, attainable (given the resources available), realistic and timely.

The next step is to define pertinent indicators to measure the progress made toward the goals and supply information for decision-making. These indicators should be:

• Useful. Are the indicators truly appropriate for the decision-making process? Are they relevant to a wide range of users? Can they be used to assess the actual changes brought about by the action taken? For example, a study of the complaints registered at various tourist information offices could certainly be used to evaluate visitor satisfaction;
• Reliable. Are the information sources recognized and reliable? Is the research methodology valid? Are the findings accurate? In Canada, the primary source of reputable statistical data is still Statistics Canada, even if the information is sometimes incomplete;
• Timely. What is the most opportune time to evaluate performance? For example, before reprising a promotional campaign to attract off-season visitors, one must ensure the preceding campaign successfully achieved the goals set in the marketing plan, by measuring the number of requests for information, the number of packages sold, and so on;
• Comparable. Will the indicators supply data comparable to data obtained from similar companies or organizations? It is easier to compare results when data are collected using standard classifications. For example, when most members of the industry use specific age groups for the socio-demographic breakdown of their customers, it is best to adopt the same groupings to make comparisons easier. Obviously, this means one must previously research the primary indicators and information sources used.

In addition, when defining indicators, it is essential to identify certain parameters like:

• Frequency. How often is information required? Monthly? Quarterly? Annually?;
• Participation. Who is involved in the process (data suppliers and users)? Who will be in charge of collecting the data? Is co-operation needed from partners? Establishing a partnership among stakeholders is key to the success of the evaluation process;
• Cost. What financial and human resources are available to perform evaluation-related tasks? Are these resources sufficient to carry out all the steps of the process? Is there any chance costs could be shared among various partners?;
• Information sources. What is the best way to obtain this information? Are any studies, statistics or information sources currently available? Should new surveys or studies be planned? How should one fill in missing information?;
• Expectations. Do partners or financial backers have specific expectations of the evaluation process?

Once objectives and indicators have both been clearly defined according to the previously identified characteristics and parameters, the evaluation itself can produce an honest assessment of the projects already carried out, whether they refer to marketing activities, economic impact, product development, human resource management, or information services.

In fact, all sectors that require enlightened action and decision-making should be evaluated. Then, the information should be disseminated to all partners involved in the plan, strategy or policy.

Sources:
– British Tourist Authority. «A Strategy for the Sustainable Growth of Tourism to Britain», September 2001
– Tourism New Zealand. «Statement of Service Performance, Annual Report 2003».