Atlantic Canada Opportunities Agency
Symbol of the Government of Canada

Evaluation of the Community Futures Program in Atlantic Canada Final Report  

Atlantic Canada Opportunities Agency

Finance and Corporate Services

Evaluation Unit

June 2009

3.3        Program Impact – Short Term

The findings of the evaluation of the CF Program in Atlantic Canada on the issue of short-term program impacts are presented in this section of the report.  The evaluation questions considered in addressing short-term impacts were as follows:

  • To what extent has the CF Program provided appropriate information, referrals and counselling to clients?
  • To what extent has the CF Program improved business knowledge and skills of clients?
  • To what extent has the CF Program created new business start-ups or strengthened existing businesses?
  • To what extent is the CF Program serving the needs of Official Languages Minority Communities (OLMCs)?
  • To what extent is the CF Program serving the needs of Aboriginal Communities (AC)an ACOA target group?

3.3.1     Appropriateness of Information, Referral and Counselling Services to Clients

Findings:
According to E-reports data, over the five-year period covered by the evaluation, almost 40,000 clients received counselling services from the CBDCs. 

Client and CBDC survey respondents indicated that services most often accessed (not including loans) are business counselling and business information, training courses and seminars, and referral services. On the client survey, clients were asked to identify, from a list of options, services that they had accessed from their CBDCs.  Similarly, CBDC survey respondents were asked to indicate, from the same list of options, which services they provided. The list of services provided and accessed included all services provided under the national CF Program. As discussed previously, the mandate of the CBDCs in Atlantic Canada is less broad than the CF Program in other parts of the country and, therefore, some of the services provided under the national CF Program may not be offered by CBDCs in Atlantic Canada. Figure 5, below, illustrates the percentage of CBDC survey respondents and clients who indicated they sometimes or often offered/used the CBDC services available in Atlantic Canada.

As indicated by Figure 5, almost all CBDC survey respondents indicated they sometimes or oftenprovided these services.  Clients were less likely to indicate they had sometimes or oftenused these services;  however, this may be due to the fact that, as outlined in the methodology section of this report, clients invited to respond to the client survey were almost entirely CBDC loan clients, whereas the services listed included other services in addition to loan services (business financing).

Figure 5:       Survey Responses re: Provision of/Access to CBDC Services

See Header

 

Other services provided by the national CF Program include the development of infrastructure to support economic development; promotion of the community for tourism investment; organizing other partners to address telecommunications issues and promote the use of the information highway; generating awareness and action on sustainable development; and assistance with the development of community strategic plans.  In spite of the fact that these activities are not included in the mandate of Atlantic Canada’s CBDCs, some CBDCs did indicate that they sometimes or often provide these services,  as did some clients.  Table 21 illustrates the percentage of clients and CBDCs who indicated that they used / provided these services sometimes or often.

 

Table 21.       Survey Responses Re: Use/Provision of Services

 

Percentage of clients and CBDCs who used / provided listed services    
  Sometimes or Often  
  Clients CBDCs
Development of infrastructure to support economic development 20.4 37.3
Promotion of the community for tourism investment 17.3 61.7

Organizing other partners to address telecommunication issues and promote use of information highway

14.4 38.3
Generate awareness and action on sustainable development 20.6 61.7
Assistance with the development of community strategic plans 17.8 56.7

Client survey respondents indicated they were highly satisfied with the services provided by the CBDCs in Atlantic Canada.  The percentage of clients who indicated they were satisfied (i.e. either very satisfied or somewhat satisfied) with these services were as follows:  provision of business information (90.9%), business counselling (87.6%), referral services (84.0%), and training courses and seminars (83.4%).  Clients generally indicated a high level of satisfaction with the other services provided by CBDCs (as listed in the table above).

Conclusion:  Appropriate information, referrals and counselling services are being offered and clients have a high level of satisfaction with these services.

3.3.2     Impact on Business Knowledge and Skills of Clients

Findings:
Both client and CBDC survey respondents were asked to what extent the CBDCs have been successful in improving clients’ business skills and knowledge.  All CBDC survey respondents (staff and board members) felt they had been successful in improving the business skills and knowledge of clients (57.4% said very successful, 42.6% said somewhat successful).  While the CBDCs were slightly more positive in this respect than were clients, this is consistent with what was reported by clients, who indicated that CBDCs had been very successful (35.4%) and somewhat successful (45.6%) in improving their business skills and knowledge.   However, while it is encouraging that so many clients reported the CBDCs had been successful in improving their business skills and knowledge, it should be noted that there remained a small proportion of clients that disagreed (4.0% indicated not very successful and 4.5% indicated not at all successful).

Clients were also asked to indicate to what extent they were using the information obtained from the CBDC to undertake specific activities.  While many clients indicated they are using the information obtained, it appears to be to a limited extent.  Few clients are using information for human resource management, to access other funds, or for project management (Figure 6 below). 

CBDC services appear to be most useful for clients to create a funding application (36.0% said to some extent and 21.9% said to a great extent).  Clients are also using information for financial management (27.5% said to some extent and 10.7% said to a great extent) and to market their business (24.4% to some extent and 8.1% said to a great extent).

Over half of all client survey respondents (52.8%) also indicated their CBDCs had helped them develop business plans. 

Responses from interviewees supported the survey response findings.  All but one interviewee (26 of 27) agreed that the CF Program had contributed to the business knowledge and skills of its clients.  Most interviewees (22 of 27) stated that the training services (i.e. workshops and seminars) were key to developing knowledge and skills of clients.  As well, almost half (13 of 27) referenced counselling or one-on-one coaching.
 

Figure 6:       Client Use of Information Obtained From CBDC

See Header

Conclusion:  The CBDCs have improved clients’ business skills and knowledge, and clients are using the information/advice provided by the CBDCs, although they are not specifically using the information for human resource management, for project management, or to access other funds.

3.3.3     Impact on New and Existing Businesses

Findings:
Previous evaluation studies have noted that the CF Program nationally and the CBDCs in Atlantic Canada have created new business start-ups and strengthened existing businesses. According to the previous national CF evaluation, 42% of CF loan recipients obtained funds to start a business, whereas 58% were already operating a business. One in five of these existing businesses would have shut down if it had not received a loan, while 15% would have ceased some operations.[51]

Based on E-reports data, during the five-year evaluation period (April 2003 to March 2008), the CBDCs approved 5,914 loan applications, or an average of 1,183 per year.  Just over one-quarter of these loans were loans to new businesses (1,676 loans to new businesses and 3,873 loans to existing businesses). Table 22 shows the breakdown by province of total applications approved during the five-year evaluation period, the average number of applications approved per year, and the average number of applications approved per CBDC per year.

Table 22.       Loans Approved by Region 

Region
Total approved applications
Average per year
Average per CBDC per year
PEI 267 53.4 17.8
NL 1,437 287.4 19.2
NB 1,936 387.2 38.7
NS 1,933 386.6 38.7
CB 341 68.2 22.7
Total 5,914 1,182.8 28.8

As shown in Table 22, New Brunswick and Nova Scotia (except Cape Breton) give out almost twice as many loans per CBDC as do the other regions.

Survey respondents (both clients and CBDCs) and interviewees agreed that the CBDCs have been successful at creating and maintaining businesses in communities.  Over three-quarters of client survey respondents (77.5%) indicated that the CBDCs had been successful at creating businesses in their communities, and almost as many indicated that the CBDCs had been successful in maintaining businesses in their communities (76.4%).  All CBDC survey respondents agreed that the CBDCs had been successful at creating and maintaining businesses in communities. 

All interviewees (38 of 38) indicated that the CF Program has had an impact on the creation and/or the expansion and strengthening of SMEs.  Many interviewees (17 of 27 interviewees) noted the volume of CF assistance provided to businesses (i.e. loans and counselling) or the number of businesses that have been assisted.  Some interviewees (7 of 27) also noted that traditional lenders are less and less present in rural communities, and a few (5 of 38) noted that the CBDCs provide the only lending option for many businesses.

The majority of clients surveyed stated that they would not have been able to start, maintain or expand their businesses without CBDC support.  A large percentage of those who received loans to start a business (62.6%) indicated this was very or somewhat unlikely; compared with 59.7% of those who received loans for an existing business.  The remainder indicated they would have been somewhat likely or very likely to have been able to start, maintain or expand their business without CBDC support. 

The analysis for this evaluation question also included an examination of leveraging.  Over 20% of clients indicated they were able to leverage other funds as a result of their CBDC loan (22.5% of start-ups and 21.4% of exiting businesses).  For both start-ups and existing businesses, funds leveraged as a result of CBDC loans came primarily from financial institutions (51.4% for start-ups, 61.1% for existing businesses).  According to E-reports data, over the five-year evaluation period, a total of $191,605,168 was leveraged by clients, with an average of over $31 million per year.  Client survey responses supported this finding.  Three-quarters of clients surveyed indicated that their local CBDC had been somewhat successful (39.4%) or very successful (36.3%) at improving the ability of businesses to access capital from traditional sources. 

As a final indicator of the extent to which the CF Program created new business start-ups or strengthened existing businesses, the evaluation looked at the survival rate of businesses assisted by the CF Program.  Over 90% of client survey respondents indicated that their businesses were still in existence.  Of those that were not (30), 40% had ceased operations in 2007 or 2008.  Of those that were not still in business, 25% had been in business five or more years.  Table 23 illustrates the length of time that now-closed businesses had been in existence.

Table 23.   Life Span of Closed Businesses 

# of years
Frequency
Percentage
Cumulative Percent
0.00 2 5.6% 5.6%
1.00 3 9.3% 14.8%
2.00 7 25.9% 40.7%
3.00 4 14.8% 55.6%
4.00 2 5.6% 61.1%
5.00 4 13.0% 74.1%
6.00 2 7.4% 81.5%
7.00 2 5.6% 87.0%
8.00 1 3.7% 90.7%
9.00 1 3.7% 94.4%
16.00 1 1.9% 96.3%
24.00 1 3.7% 100.0%
Total 27 100.0%
Businesses that were closed were also examined across client attributes of age, gender, language, and whether or not the client was Aboriginal, part of a visible minority group, or disabled. There are no significant trends in these attributes that can be identified as underlying reasons for business closures. See Appendix A for further details of closed businesses across these attributes. 

Conclusion:  CBDCs have created new business start-ups and strengthened existing businesses.

3.3.4     Serving the Needs of Official Language Minority Communities

Findings:
CF Program documentation illustrates a commitment to serving Official Language Minority Communities (OLMCs). The RMAF developed for the CF Program states that “… CFOs which operate in areas where a significant Official Language Minority Community (OLMC) exists, must comply with official languages requirements to provide service of comparable quality in both official languages to the public.  CFOs must:

  • make announcements to the public concerning the activity, project or program in both official languages;
  • actively offer services to members of the public in both official languages;
  • make available in both official languages any documents for the general public relating to the activities, projects or programs;
  • encourage members of both official language communities to participate in the activities, projects or programs; and
  • organize activities, projects or programs, when appropriate, in such a manner as to meet the needs of the two linguistic communities.”[52]

In addition, the revised business plan for the pan-Canadian CF Program makes a commitment to “work toward ensuring all major documents/communications are prepared and distributed in both official languages.”

There are 11 CBDCs located in OLMCs in Atlantic Canada; eight in New Brunswick, one in Prince Edward Island, and two in Nova Scotia.[53]  CBDCs in other areas also indicated that they make efforts to provide service in both official languages.

Of clients who had accessed services provided by the CBDCs, over 99% indicated they were able to access the services in the language of their choice.  Only three respondents indicated they could not access services in the language of their choice,  and of these three, only one lived in an OLMC.  Interviewees (33 of 35) indicated that the CBDCs provide services in both official languages where required.  The two other respondents indicated that they were either not sure of the language of service or noted the lack of language capacity in the area, but indicated that if someone ever asked for service in a specific language (French), the CBDC would be able to obtain it with assistance from others.  Interviewees provided various examples of actions taken by the CBDCs to ensure this capacity exists.  For example, some interviewees (11 of 35) noted that the CBDCs have bilingual staff on site.  A few others (6 of 35) indicated that a back-up plan was in place to offer service in French when needed.  This was particularly noteworthy since five of the six interviewees who noted this were from Newfoundland and Labrador, where there are no OLMCs.  These interviewees indicated they could make referrals to, or get assistance from others, such as CBDCs with bilingual staff, board members or REDO staff.  A couple of interviewees (2 of 35) noted that documents and publications are available in both official languages. 

The most common challenge noted by the CBDCs to providing bilingual services was difficulties in hiring qualified, bilingual staff to fill positions.  Four of 19 CBDC interviewees noted this challenge, although none of these four worked in a CBDC in an OLMC.  Three other senior managers or program managers also noted this challenge.

Conclusion:  CBDCs are serving the needs of Official Language Minority Communities.

3.3.5     Serving the Needs of Aboriginal Communities

Findings:
Accessing loan capital can be particularly challenging for Aboriginal SMEs.  The Ulnooweg Development Group Inc. (Ulnooweg) supports Aboriginal entrepreneurs in Atlantic Canada.   Ulnooweg has been providing loans and business services to Aboriginal entrepreneurs since 1986.  According to the Ulnooweg website, “Ulnooweg’s lending services bring financial support for businesses who may not be eligible for loans though other lending institutions.”[54]  The Ulnooweg Development Group originally received $6 million in investment funds from the CF Program to provide support to Aboriginal SMEs.  The $6 million is used, as with the CF Program funds, to provide repayable loans to Aboriginal SMEs.   The Ulnooweg Development Group provides counselling to Aboriginal SMEs across the Atlantic Provinces as do the CBDCs, through the CF Program. 

According to the Impact Assessment of Community Futures in Canada study of March 2006, in Atlantic Canada, the CBDCs work with Aboriginal clients on an individual basis or refer them to the Ulnooweg Development GroupInc., which supports Aboriginal entrepreneurs much like a CBDC.[55]  The extent to which the CBDCs and Ulnooweg work together is unclear.  According to a representative from the Atlantic Association of CBDCs, individual CBDCs have relationships with Ulnooweg, but there is no formal agreement between Ulnooweg and the Atlantic Association.  According to this representative, there are examples of CBDCs and Ulnooweg working collaboratively, on initiatives such as referrals, associate membership of Ulnooweg into the Atlantic Association, purchasing agreements, and other collaborative work.  However, according to interview comments by a representative of ACOA’s Aboriginal Affairs staff, while Aboriginal SMEs are well served through the Ulnooweg Development Group, there are opportunities for more collaboration and co-operation between Ulnooweg and the CBDCs.  This interviewee suggested that while Ulnooweg would like to partner with the CBDCs, and has approached the Atlantic Association in an effort to develop a relationship, a relationship has not yet developed.  It was unclear why this has not happened, although the interviewee added that it may be due to the fact that loans to Aboriginal SMEs are sometimes viewed as higher risk and are, therefore, less attractive to the CBDCs.  The interviewee also reported that Ulnooweg would like to share the risk with the CBDCs where possible.  The scope of the evaluation did not allow for more investigation into the relationship between Ulnooweg and the CBDCs, therefore,  the extent to which they work together is not clear.

CBDC survey respondents were asked to provide information about the percentage of Aboriginals within their communities and among their clients.  Just over 60% of respondents answered these two questions.  Of those, 55% indicated that the percentage of their clients who are Aboriginal is equal to or greater than the percentage of Aboriginals in their communities.  The other 45% stated that Aboriginal clients represented a smaller percentage of their clients than was the proportion of Aboriginals in their communities. 

Statistics Canada labour market data indicates that the percentage of the population in Atlantic Canada that is Aboriginal has increased from 2.7% in 1996 to 3.7% in 2006.  The percentage of the population that is Aboriginal in each of the Atlantic Provinces is shown in the table below.

Table 24.       Aboriginal Populations in Atlantic Canada 

 
1996
2001
2006
Atlantic Canada 2.7% 3.2% 3.7%
Newfoundland and Labrador 2.6% 3.7% 4.7%
Prince Edward Island 0.7% 1.0% 1.3%
Nova Scotia 1.4% 1.9% 2.7%
New Brunswick 1.4% 2.4% 2.5%
A small number (n=8) of client survey respondents identified themselves as Aboriginal. This represents 2.8% of respondents, which is slightly less than the 3.7% Aboriginal population for Atlantic Canada identified by Statistics Canada 2006 census data.

Conclusion:  The Ulnooweg Development Group Inc.originally provided investment funds through the CF Program, and is serving the needs of Aboriginals in Atlantic Canada.  However, some information suggests there is opportunity for more partnership or collaboration between the Ulnooweg Development Group and the CBDC

3.4        Program Impact – Intermediate

The evaluation’s findings on the issue of intermediate-term program impacts of the CF Program are presented in this section of the report. There was one evaluation question relating to program intermediate impacts:

  • To what extent has the CF Program: supported community economic development; assisted communities to develop and diversify their economies; and strengthened community capacity?

3.4.1     Intermediate Impact of the CF Program

Findings:

Community Economic Development:
Foundation documents for the CF Program state that CFOs receive funding to, among other activities, support community-based projects and special initiatives by collaborating with other partners in the public sector and civil society to implement strategic community projects or deliver special initiatives targeted to communities.  However, community economic development projects are not part of the mandate of the CBDCs.  In spite of this, CBDC survey respondents indicated that most CBDCs participate in Community Economic Development (CED) projects.  When asked how many CED projects their CBDC would be involved in during a typical year, only a few respondents said ‘none’ (three CBDC staff members, and one CBDC board member).  A large number of CBDC board members (11) and several CBDC staff members (four) also answered ‘don’t know’.  Of the remainder who answered this question, most indicated between one and six projects (51.8% of CBDC staff members, and 33.3% of CBDC board members), with the median response being four.  For each CED project in which they were involved, the CBDCs contributed an average of 11 hours. 

Stakeholder interviews suggested that CED is supported in Atlantic Canada primarily through the loans and counselling activities of the CBDCs, by creating and expanding businesses, and building skills and knowledge in the businesses supported. The CBDCs have created new business start-ups and strengthened existing businesses. Since the inception of the CF Program, the CBDCs have disbursed 20,571 loans.  Based on E-reports data, over the five-year period of the evaluation, the CBDCs have supported the creation of 5,452.3 full-time equivalent (FTE)[56] jobs, and the maintenance of 10,496.0 FTE jobs.  (For a breakdown of FTE jobs created and maintained by province, see Appendix B.)  Stakeholders argue that it is through loans and the other services provided by the CBDCs (e.g.  counselling) that jobs are created and maintained, and businesses strengthened, all of which supports community economic development by creating, maintaining and strengthening businesses.

Economic Diversification

Three-quarters of interviewees agreed that the CF Program has helped communities to diversify their economies (24 of 32); a few interviewees (five) were unsure, and a few (three) disagreed. 

Information from E-reports provides a breakdown in the investment by sector over the five-year evaluation period, although a breakdown by number of loans is not available.  Table 25 indicates the percentage of total investments in each sector over the evaluation period.

Based on E-reports data, results indicate that the CBDCs are investing in SMEs across almost all sectors.  Only the mining and aquaculture sectors had no investment in some provinces during this evaluation period.  This is supported by survey responses from clients.  Table 26 shows the breakdown of client survey respondents by sector.  (Note: the sector choices in the client survey did not match those of E-reports, so direct comparison is not possible.)

Table 25.        Percentage of Investments by Sector, by Region 

PEI
NL
NB
NS
CB
ATLANTIC
Agricultural 3.2% 0.7% 3.1% 3.3% 1.1% 2.5%
Aquaculture -- -- 2.1% 1.9% -- 1.4%
Fish Harvesting 0.3% 13.1% 3.8% 1.5% 1.6% 4.8%
Forestry -- 2.3% 3.2% 1.1% 0.6% 2.1%
Mining -- -- -- 0.2% -- 0.1%
Construction 4.9% 6.5% 4.0% 7.7% 3.5% 5.6%
Tourism 19.5% 8.0% 3.7% 9.4% 13.4% 7.5%
Transportation and Storage -- 3.8% 7.8% 6.2% 9.1% 6.3%
Wholesale and Retail Trade 15.4% 34.1% 23.7% 22.9% 9.5% 24.4%
Manufacturing 15.2% 5.5% 8.7% 7.6% 7.7% 7.8%
Other Industries[57] 41.4% 26.0% 39.8% 38.2%  53.6% 37.5%
Total 100.0% 100.0% 100.0% 100.0% 100.0%

Table 26.       CBDC Clients by Sector 

Sector:
#
%
Agriculture, Forestry, Fishing and Hunting 63 16.8
Utilities 2 0.4
Construction 28 7.4
Manufacturing 24 6.5
Wholesale Trade 10 2.6
Retail Trade 67 18.0
Transportation and Warehousing 12 3.2
Information Technology and Cultural Industries 11 2.8
Finance and Insurance 2 0.4
Real Estate and Rental and Leasing 7 1.7
Professional, Scientific and Technical Services 16 4.2
Management of Companies and Enterprises 2 0.4
Educational Services 5 1.2
Health Care and Social Assistance 12 3.1
Arts, Entertainment and Recreation 21 5.6
Accommodation and Food Services 34 9.0
Other Services (except Public Administration) 57 15.2
Other 5 1.3
Total 372 100.0

Strengthened Community Capacity

Industry Canada/FedNor defines capacity building as a ”broad and complex set of activities that will, if successful, enable people of diverse backgrounds in communities to take a more active role in shaping their economies and determining their collective futures.  It enables communities to set their priorities, identify and develop their own capabilities and resources and make the best investments.” [58]

Almost all CBDC survey respondents agreed that their CBDCs have helped to build capacity for community economic development:  just under 90% of respondents (89.8%) indicated to some extent (40.7%) or to a great extent (49.2%).  Respondents who did not agree indicated they did not know.

Both interviewees and survey respondents indicated that the CBDCs had been successful at strengthening the capacity of the community.  Almost all interviewees (34 of 36) agreed that the CF Program has strengthened community capacity.  This was corroborated by both CBDC and client survey respondents.  Almost all CBDC survey respondents (96.7%) indicated that their CBDC had been successful at strengthening the capacity of their communities:  38.3% said somewhat successful and 58.3% said very successful.  Similarly, client survey respondents indicated that their CBDCs had been somewhat successful (42.2%) or very successful (29.0%) at strengthening the capacity of their communities. 
Developing or strengthening community capacity involves building the skills of individuals to play a role in this.  Information from the evaluation shows that the CBDCs are improving the business skills and knowledge of clients. As well, there is some evidence that clients have developed partnerships as a result of their involvement with their CBDCs. This too, impacts on community capacity.  Fifteen percent of clients indicated they had developed partnerships as a result of their involvement with a CBDC.  Of these, most indicated they had developed between one and three partnerships (83.3%).  Financial partnerships represented 38.1% of those mentioned (n=18).  Other partnerships included those for exchanging information, services or references.  Clients who indicated they had developed partnerships most often referred to other SMEs, associations, banks or the BDC. 

Interviewees also believe that the efforts of CBDCs have led to the development of partnerships (32 of 33 interviewees).  This evaluation did not define ”partnerships” for interviewees, so this  report cannot provide a definition of what is or is not included.  However, interviewees most often referenced other organizations with whom they work either formally or informally to achieve objectives.  Those most often cited were REDOs or regional development agencies, chambers of commerce, various levels of government, financial institutions, and the BDC.  As discussed in section 3.1 (Relevance), there is a formalized partnership between the CBDCs and the BDC. 

Conclusion:  Community economic development (CED) projects and the development of community strategic plans are not part of the mandate of CBDCs in Atlantic Canada.  However, stakeholders still believe that the CBDCs have been successful in supporting CED, assisting communities to develop and diversify their economies, and strengthening community capacity, through their loan activities and business services.

3.5         Program Impact – Long Term

The findings of the evaluation of the CF Program in Atlantic Canada on the issue of long-term program impacts are presented in this section of the report.  The evaluation questions considered in addressing long-term impacts were as follows:

  • To what extent has the CF Program contributed to long-term goals e.g. economic growth and stability, diversification and development of local rural communities, sustainable communities, and survival of business assisted by the CBDCs?Has the CF Program produced unintended positive and/or negative outcomes?

3.5.1     Contribution to Long-Term Goals

Findings:
There has been a slight change in the labour force participation rates of CBDC-served regions (although minimal in PEI).  CBDC-served regions were fairly consistent with the provincial figures, as illustrated in the table below; therefore, there is no evidence that the CF Program has had an influence on participation rates.

Table 27.       Change in Participation Rates (from 1996 to 2006) in CBDC Regions versus Atlantic Provinces 

 
Provincial
CBDC served regions only
NL 2.6% 2.6%
PEI -0.1% 0.1%
NS 1.9% 2.0%
NB 1.5% 1.1%

There have been more significant changes in the rates of unemployment.  Overall, unemployment has decreased in Canada from 10.1% in 1996 to 6.6% in 2006.  Similarly, in Atlantic Canada, the unemployment rate has dropped from 16.6% to 11.5% during the same time period.  When looking at the change in the unemployment rate in CBDC-served regions compared with provincial averages, it is apparent that unemployment rates in CBDC-served regions had more significant decreases than the provincial averages, as illustrated by table 28. 

Table 28.       Change in Unemployment Rates (from 1996 to 2006) in CBDC Regions versus Atlantic Provinces 

Provincial
CBDC served regions only
NL -3.3% -6.4%
PEI -0.6% -2.3%
NS -2.4% -4.8%
NB -3.0% -5.5%

Average earnings have been increasing both in CBDC-served regions and in the Atlantic Provinces. Table 29 illustrates the changes in average earnings from 1996 to 2006. As illustrated, with the exception of Newfoundland and Labrador, the increase in average earnings in CBDC-served regions has been slightly lower than the increase in the provincial averages.  

 Table 29.       Change in Average Earnings (from 1996 to 2006) in CBDC Regions versus Atlantic Provinces 

% change in average earnings 1996 to 2006
  Provincial CBDC served regions only
  Male Female Male Female
Newfoundland and Labrador 9,465% 6,842% 10,138% 7,596%
Prince Edward Island 6,705% 7,971% 6,038% 7,966%
Nova Scotia 9,895% 7,856% 9,475% 7,448%
New Brunswick 8,142% 7,623% 8,096% 7,152%

Some indicators of long-term success for economic development programs are: change in employment rate, change in employment distribution, and change in annual average household income.  While these measures are all highly relevant, changes to these economic indicators cannot be solely or directly attributed to the CF Program.  There are always many other factors (e.g. other investments made, economic conditions) that also have an impact on these broad measures. The impact of the CF Program relative to all of the other community influences is difficult to assess.  While the changes to unemployment rates seem to indicate that CBDC-served regions are improving more rapidly than are the provinces as a whole, changes to the participation and the average earnings rates are not significantly different from provincial averages. 

Using data collected through the evaluation, combined with existing data from E-reports, ACOA staff conducted an economic impact analysis of the CF Program in Atlantic Canada. This analysis used econometric models developed by The Conference Board of Canada for each Atlantic province to estimate the economic impact.  While relevant highlights will be discussed here, the full analysis, including hypotheses and methodology, can be examined as Appendix C of this report.

The economic impact analysis estimated the economic benefits of the CBDC’s CF Program in Atlantic Canada.  According to this analysis, “through CBDCs’ direct support to business, the direct creation of jobs can be utilized to estimate the indirect and induced impact on the economy of the four Atlantic Provinces.”[59]

Based on this economic impact analysis, the real gross domestic product (GDP) was $150 million higher (in constant 1997 dollars) in 2007 than it would have been in the absence of the CF Program’s expenditures in direct support of business.  Furthermore, over the five-year evaluation period, CF Program support to businesses produced increases of over $2.35 in GDP for every dollar of expenditure (including loans and operating expenditures).  This same study indicates that total employment in Atlantic Canada was over 3,300 higher (including direct, indirect, and induced employment gains) in 2007 than it would have been without the CBDCs’ direct support to businesses over the five-year evaluation period.  Note that these employment numbers are gross estimates and do not take into consideration net job gains or losses as a result of economic conditions.  It is important to keep in mind that the CF Program introduced a structure for the creation of and support for community-based development/adjustment initiatives in non-metropolitan areas of significant economic stress across Canada[60] and, as a result, job losses may be more severe than in other areas.  Finally, based on the economic impact analysis, personal income in Atlantic Canada was $141.6 million higher (in constant 1997 dollars) in 2007 than it would have been in the absence of the CF Programs’ expenditures in direct support of business.

Survey respondents agree that the CBDCs are contributing to the long-term objectives of the CF Program.  Table 30 illustrates the extent to which client and CBDC survey respondents agree with statements that the CBDCs are contributing to the long-term objectives.
 
Table 30.       Survey Respondent Views on Contribution to Long-term Goals 

% of respondents who agree or strongly agree that the CBDCs contribute to: CBDC survey respondents Client survey respondents
Economic growth and stability 100.0% 82.4%
Diversification and development of local rural communities 100.0% 80.8%
Sustainability of community 98.0% 80.3%
Survival of local businesses 100.0% 76.7%
Interviewees agreed that the CBDCs are contributing to these long-term objectives.  Almost all interviewees agreed that the CBDCs contribute to all of the objectives:
  • economic growth and stability (35 of 37 interviewees);
  • diversification and development of local rural communities (24 of 32 interviewees); sustainability of communities (23 of 24 interviewees); and
  • survival of local businesses (26 of 30 interviewees).

Conclusion:  Measuring the impact of the CF Program on long-term goals presents a number of challenges due to the indirect influence most programs have on indicators of interest (e.g. economic growth, stability). However, an ACOA economic impact analysis suggests that the CF Program has contributed to economic growth in Atlantic Provinces.  As well, stakeholders agree that the CF Program is contributing to its long-term goals.

3.5.2     Unintended Impacts

Findings:
Although two-thirds of interviewees indicated that there were unintended impacts from the program, examples provided were unexpected rather than unintended, and often were not impacts of the program, but rather characteristics or attributes of the program.  In other words, while interviewees may have indicated that the program had an impact that surprised them (unexpected impact), generally the impact they spoke of was not unplanned (unintended impact), rather it was the magnitude of impact that was unexpected.  For example, interviewees identified the following as unintended positive and negative impacts of the program.

  • The longevity of the CF Program. It has stood the test of time. The program is a success that has been around for decades.
  • The good reputation of the CF Program.
  • The diversity and dedication of board members.
  • The high survival rate of businesses that have been assisted by the CF Program.
  • Some CBDCs are too risk-averse/rigid with regards to their lending practices.

The longevity of the program, the reputation of the program, and the diversity and dedication of board members are all characteristics of the program that interviewees feel have exceeded expectations. The high survival rate of businesses is reported as somewhat of a surprise, but is not an unintended impact, since it was always the intention of the program to help businesses survive.

Conclusion:  The current evaluation did not reveal any specific unintended impacts of The CF Program in Atlantic Canada (positive or negative).
 

3.6        Cost-Effectiveness and Alternatives

The findings of the evaluation of the Community Futures Program in Atlantic Canada on the issue of cost-effectiveness/alternatives are presented in this section of the report. The evaluation questions which were considered in addressing the issue of cost-effectiveness were as follows:

  • To what extent is the CF Program cost-effective?
  • Are there other more cost-effective/efficient approaches or alternatives to be considered that would achieve CF Program objectives?

3.6.1     Cost-Effectiveness

Findings:
To assess the cost-effectiveness of a program, generally, the costs for program operations[61] are compared to program outcomes to calculate a cost-per-unit outcome.  For the CF Program in Atlantic Canada, it was not possible to assess the extent to which ACOA is cost-effective in administering the program (i.e. costs for operation versus outcomes).  This is due to the fact that ACOA does not separate expenditures for the operation of the CF Program from those of the other programs it delivers.  Expenditures for the CF Program are included in overall spending by ACOA.  Only operating cost allocations are available. 

In the absence of ACOA’s operating costs for the CF Program, the evaluation examined the cost-effectiveness of the program based upon the operating dollars provided to the CBDCs, and reviewed the activities of the CBDCs in relation to their outputs and outcomes.

During the five-year evaluation period, the CBDCs and CF associations received a total of almost $65.5 million for the CF Program.  Of this amount, $45.7 million was used for the operation of the CBDCs; therefore, the average per CBDC is $220,347 per year for operations (2003-2004 to 2007-2008), with the amount of funding increasing over a four-year period to 2006-2007, before falling slightly in 2007-2008.   Table 31, below, presents the operating costs per year.

Table 31.       Operating Costs for the CF Program in Atlantic Canada 

  2003-2004 2004-2005 2005-2006 2006-2007 2007-2008
Total CBDC-only operating costs $6,835,112 $7,972,612 $8,667,019 $11,139,373 $11,064,825
Total Association’s operating costs $1,253,724 $1,408,401 $2,747,415 $3,250,481 $2,639,415
Total CBDCs’ and Associations’ operating costs $8,088,836 $9,381,013 $11,414,434 $14,389,853 $13,704,240
Average operating cost per CBDC (not including Association’s costs) $158,956 $189,824 $211,391 $271,692 $269,874
Using the dollar value of the operating costs provided to the CBDCs (row one in the above table) and the outcome information available in E-reports (i.e. jobs created/maintained and businesses started/maintained), it was possible to calculate the cost of generating these outcomes. The average cost per FTE job created was $8,378; the average cost per FTE job maintained was $4,352. The average cost per start-up business assisted was $27,255; and the average cost per existing business assisted was $11,794. 

Table 32.      Operating Cost per Job Created/Maintained and Business Started/Maintained

  Year
  2003-2004 2004-2005 2005-2006 2006-2007 2007-2008 Overall Average
Cost per FTE job created $ 5,803.83 $ 6,567.93 $ 8,061.74 $ 12,501.91 $ 10,107.80 $ 8,377.87
Cost per FTE job maintained $ 3,696.15 $ 3,582.59 $ 3,985.53 $ 5,129.65 $ 5,332.01 $ 4,352.04
Cost per business started $ 17,799.77 $ 21,664.71 $ 25,566.43 $ 37,380.44 $ 38,553.40 $ 27,254.74
Cost per business maintained $ 8,853.77 $ 9,916.18 $ 10,595.38 $ 15,601.36 $ 14,463.82 $ 11,794.20

Operating costs have increased generally over the five-year period and, in turn, costs per job created, job maintained, start-ups assisted and existing businesses assisted have also increased. 

It is important to note that these figures were calculated using the total operating dollars provided to the CBDCs.  Since it is not possible to split the operating costs associated with loans to start-up businesses versus those associated with loans to existing businesses, total costs were used to calculate both outcomes.  Given that 100% of the operating funds are not being expended on each of these outcomes, these costs/job figures are higher than actual.

Perhaps a more useful measure is the level of investment funds required to create one job.  Looking at investment funds only, an average of $14,253 in investment was required to create or maintain one job.  Although the investment per job created or maintained has increased somewhat over the five-year period, it has remained relatively stable over the last three years, as illustrated in Table 33. 

Table 33.       Investment per Job 

  2003-2004 2004-2005 2005-2006 2006-2007 2007-2008 5-year average
PEI  $ 9,552   $ 7,084   $ 11,004   $ 10,368   $ 11,710   $ 9,944 
NL  $ 20,669   $ 16,075   $ 17,064   $ 21,334   $ 23,428   $ 19,714 
NB  $ 12,877   $ 12,067   $ 12,990   $ 12,460   $ 12,877   $ 12,654 
NS  $ 11,948   $ 12,079   $ 17,925   $ 16,072   $ 15,395   $ 14,684 
CB  $ 13,478   $ 10,105   $ 11,033   $ 14,113   $ 13,377   $ 12,421 
ATLANTIC  $ 13,756   $ 12,385   $ 14,841   $ 15,092   $ 15,193   $ 14,253 

As illustrated in the table above, investment per job created varies significantly across the provinces, from a low cost of $9,944 in Prince Edward Island to a high of $19,714 in Newfoundland and Labrador.  It is also important to remember that this investment is paid back to the CBDC by the client.  It would appear, therefore,  to be a worthwhile investment.

As another indicator of cost-effectiveness, the evaluation looked at funds leveraged from other sources.  Over the five-year period, a total of $191.6 million was leveraged, amounting to $0.84 per dollar invested.  As illustrated in Table 34, the leveraged funds per dollar invested varies significantly across the provinces, from a low of $0.57 in mainland Nova Scotia to a high of $1.12 in Prince Edward Island. Unfortunately, there are no reliable benchmarks to determine what level of fund leveraging is ideal. These figures on their own are relatively difficult to interpret and must be considered only as a proxy measure for cost-effectiveness.

Table 34.       Leveraged Funds per Dollar Invested 

  2003-2004 2004-2005 2005-2006 2006-2007 2007-2008 Average
PEI $2.17  $1.06  $0.70  $0.72  $0.80  $1.12 
NL $0.66  $1.25  $1.29  $0.77  $0.27  $0.83 
NB $1.06  $0.85  $1.03  $0.96  $1.27  $1.04 
NS $0.90  $0.61  $0.42  $0.55  $0.50  $0.57 
CB $0.80  $1.07  $0.67  $0.69  $0.88  $0.82 
Atlantic Canada $0.96  $0.92  $0.84  $0.76  $0.77  $0.84 

Cost-effectiveness of the CF Program is enhanced by the participation of volunteers in the administration of the program.  CBDC survey respondents reported an average of 68 hours in total time contributed per month by volunteer board members (median = 42.5 hours). 

Finally, CBDC survey respondents were asked how effective the current delivery model (i.e. the delivery of the CF Program through the CBDCs) is for delivering the CF Program.  All CBDC respondents felt it was somewhat effective (20.0%) or very effective (80.0%). 

Conclusion:  It is not possible to determine whether or not ACOA is cost-effective in its administration of the CF Program, since ACOA’s operating

Conclusion:  Although targets do not exist, nor have other programs been identified for comparative purposes, the CBDCs appear to be cost-effective in administering the CF Program, as evidenced by the relative stability and growth of investment funds required per job created or maintained, the level of funds leveraged for the program from other sources, and the benefit of volunteer hours in the delivery of the CF Program by the CBDCs.  However, there are large variations across regions in the ratios of investment per job created/maintained and leveraged funds per dollar disbursed.  
 

3.6.2     Alternatives

 
Findings:
Interviewees were also asked whether they believe that the current delivery model is the most appropriate and efficient delivery model to meet the objectives of the CF Program.  Almost all interviewees (30 of 32) believe that the current model is the most appropriate and efficient delivery model.  Several of these interviewees (11 of 32) stated that the strength of the program resides in the local involvement and local decision making  A few others (3 of 32) noted that a volunteer board of directors is an advantage to the program, since the people who serve on the boards truly want to make a difference. 

Client survey respondents were asked whether there was another approach that could be used to deliver the CF Program.  Over half (54.8%) said no, and another 19.7% said that they don’t know.  The 25.5% who said yes noted improvements that could be made to the program, as opposed to alternative delivery approaches.  Interviewees (27 of 34) also identified improvements that they believe would make the program more efficient.

A few interviewees (3 of 34) also suggested a need for closer alignment or more collaboration with the REDOs.

Survey respondents were also asked to identify the top three challenges in the design and delivery of the program. These were:

  • lack of new investment funds from ACOA (70.5%);
  • lack of operational funds for more staff members (54.1%); and
  • delays in knowing funding levels (i.e. not knowing at the start of a new fiscal year) (44.3%).

Finally, with regard to alternative delivery methods, the 2008 Senate report entitled Beyond Freefall: Halting Rural Poverty[62] found that:

          “The challenges of accessing credit in rural Canada have long been recognized at the federal level. Arguably the most successful program to help address this concern has been the Community Futures Program, a federally funded, but community-based and community-led initiative…”

          “The Community Futures Program success is due in no small part to the fact that it is locally run and suited to local conditions…”

Given the recognized benefits of having local involvement in the CBDCs, and in having decisions regarding CF loans made at the local level, any alternative delivery model would need to incorporate this local presence to be viewed as equally or more effective than the current model.
 
Conclusion:
  No more effective/efficient approach to the delivery of the CF Program has been identified; however, some stakeholders suggested improvements that could be made to the program

4.         OVERALL CONCLUSIONS AND RECOMMENDATIONS

This section provides the conclusions and recommendations of this evaluation of the CF Program in Atlantic Canada.

Relevance Conclusions

There is a continuing need for the CF Program, primarily because SMEs are an important contributor to local economies, and the CF Program is supporting businesses that would otherwise not have been able to start, survive or expand. As well, research supports the notion that there are challenges for SMEs in accessing financing from conventional financial institutions. Stakeholders agree that there is a continuing need for all program services offered through the CF Program in Atlantic Canada, with the need for business financing being the most frequently noted.

This evaluation found that federal government involvement is needed and appropriate, since the program addresses a national need, and consistency and national coordination of the program require the involvement of the federal government.  As well, it was recognized that financial support is required from the federal government and that if the program were devolved to the provinces, the level and quality of services provided could be negatively affected. 

In addition to providing funding for SMEs, the CF Program offers a range of business services (e.g. counselling, business information, referrals).  There are other programs and/or organizations that offer some services that are offered by the CF Program; however, no other programs offer the whole range of services offered by the CF Program.    The level and type of collaboration with these organizations varies from one CBDC to the next.

Finally, local CBDC objectives generally align with the national CF Program. The two main activities carried out by the CBDCs (i.e. business services and access to capital) are directly aligned to the objectives of the national CF Program. However, in Atlantic Canada, the CBDCs are not mandated to carry out community economic development projects and/or strategic community planning activities, both of which are part of the national program. While CBDC mandates in Atlantic Canada differ from those in other Canadian regions, theirs objectives and activities are responsive to the needs of SMEs in rural Atlantic Canada. 

The CF Program objectives are consistent with ACOA’s objectives.  ACOA has a mandate for economic development in Atlantic Canada, and the objectives of the CF Program (i.e. fostering economic stability, growth and job creation; helping to create diversified and competitive local rural economies and; helping to build sustainable communities) are clearly aligned with ACOA’s strategic outcomes, as articulated in the department’s program activity architecture (PAA) as: Competitive and sustainable Atlantic enterprises, with emphasis on SMEs; Dynamic and sustainable communities for Atlantic Canada; and Policies and programs that strengthen the Atlantic economy.
 
Design and Delivery Conclusions

According to stakeholders, the CBDC networks that are in place (i.e. national, provincial, regional and sub-regional) are working effectively.  The main benefits of the networks are the sharing of best practices among members, and opportunities to discuss and address issues of concern.

Program stakeholders identified factors, at the national, regional and community levels, that both impact and facilitate the achievement of program results.  Funding was the most frequently cited.  Stakeholders indicated there was a particular need for additional funding for operations to address challenges in attracting and retaining qualified staff.   The challenges most frequently noted by CBDCs were factors beyond the control of the program (i.e. the collapse of industries/downturns in the economy, and out-migration or depopulation of communities/aging populations).  The networks and the ACCBIF were both noted as factors positively impacting the achievement of program results.  As well, the fact that decision making is done at the local level was noted by interviewees.

One of the evaluation questions, from the national CF evaluation question framework, is the extent to which community stakeholders have been involved in the development of strategic plans.  Although CF Program founding documents indicate that CFOs engage in community strategic planning activities, in Atlantic Canada community strategic planning is part of the mandate of REDOs, and not of the CBDCs.  Therefore, the Atlantic Canada CF evaluation examined the involvement of the CBDCs in developing community strategic plans.  The evaluation indicated that the CBDCs have been involved, to some extent, in the development of community strategic plans.  Although the CBDCs do not lead community strategic planning exercises, activities carried out by the CBDCs are linked to the community strategic plans. 

Generally, investment funds are well managed, as shown by the level of loan activity and growth in investment funds.  The agreement documents between ACOA and each of the CBDCs describe reporting requirements around many of these measures. However, there remain some large variations between regions, particularly with respect to the percentage of investment funds in active loans.

While the overall Atlantic-level data on active loans indicates an average of 77.83%, there are a number of CBDCs that could be described as under-performing on this measure over the five-year period. During this period, there were 11 CBDCs that average less than 70% of their funds in active loans; 6 that averaged less than 60%; and 2 that average less than 50%. 

Therefore,

Recommendation #1:  ACOA work in consultation with the CBDCs to establish appropriate targets for the percentage of  investment funds used in active loans. Analysis of e-Reports data suggests that a 70% minimum percent of funds in active loans is a reasonable target.

With regard to other performance measures, the CBDCs gather, monitor and regularly report on performance data, including most data elements needed for evaluation purposes.  One exception is job creation statistics which, while reported by the CBDCs, are estimates only.  This was noted as a shortfall in the previous evaluation and a recommendation was made that ACOA should encourage CBDCs to track actual job creation and maintenance.  In its management response, ACOA indicated that “actual jobs created will be monitored by CBDCs at a common period.  Regions will undertake periodic surveys.”[63]

As well, there is some inconsistency across the CBDCs in how loan losses are tracked, in when and how loans are written off, and in the calculations of allowances for doubtful accounts.

Therefore,

Recommendation #2:  ACOA work with the CBDCs to establish a systematic method for following up with clients to obtain actual job creation and maintenance figures. ACOA provides support to the CBDCs to develop an accepted performance measure related to loan losses and ensure that the CBDCs regularly gather and report on these figures.

Finally, on the issue of performance measurement, the fact that ACOA does not track operating expenses separately from other programs makes it difficult to monitor spending for the program and to measure ACOA’s cost-effectiveness in the administration of the program. 

Therefore,

Recommendation #3:  ACOA take steps to put into place the capability to separately capture and report on Operations and Maintenance expenditures associated with the CF Program.

The CBDCs have implemented policies and practices for selecting and renewing board members. As well, they have established processes to ensure that accountability requirements are communicated to board members.  The CBDCs report publicly on their activities, although many of their communications activities are marketing related, as opposed to venues for communicating their activities or results.  The methods and regularity of external communications are diverse and vary from one CBDC to another.  As well, a review of CBDC websites  shows that annual reports are not readily available on many CBDC websites[64], although the Atlantic Association of CBDCs has indicated that it will be adding a hyperlinked icon to all CBDC Web pages to allow individual CBDCs and Association offices the ability to upload their annual reports to a highly visible spot on their respective websites.  Finally, the format of annual reports, and what is included in them, varies across the provinces.

Therefore,

Recommendation #4:  ACOA work with CBDCs to establish a more standardized method for communicating results to communities. ACOA and the Atlantic Association of CBDCs encourage the development of a consistent format for annual reports, for use by all CBDCs in Atlantic Canada.  The Atlantic Association of CBDCs continues its work to provide a visible location on all CBDC websites for annual reports, and all CBDCs are encouraged to place their annual reports on their websites.

The previous CF evaluation recommended that ACOA encourage the CBDCs to establish standards in the areas of hours of service, minimal counselling time, minimal number of meetings with clients, and turnaround time for information requests and applications.[65] The management response from ACOA indicated that the “CBDCs will be asked to develop individual service delivery standards.”[66] Service delivery standards have been developed and implemented to different degrees, across the CBDCs. Some standards identified in the previous evaluation have not been implemented at some CBDCs. In particular, standards for minimal counselling time and for minimal number of meetings with clients have not been developed and implemented at a majority of CBDCs.

Therefore,

Recommendation #5:  ACOA follow up with the CBDCs and encourage them to develop the service standards identified in the previous evaluation.

Program Impact Conclusions

The CF Program has been successful in achieving its outcomes.  Appropriate information, referrals and counselling services are being offered and clients have a high level of satisfaction with these services.   The CBDCs are also meeting the needs of OLMCs by providing advice, support and services in both official languages, where required.  The CBDCs either have bilingual staff on-hand to provide services in both official languages, or have a plan in place to offer service in French when needed.  The most common challenge, noted by the CBDCs, to providing bilingual services was difficulties in hiring qualified, bilingual staff to fill positions;  however, interviewees who noted this difficulty were not located in OLMCs. 

The Ulnooweg Development Group Inc., which was originally provided investment funds through the CF Program, is serving the needs of Aboriginals in Atlantic Canada.  Information from interviews suggests, however, there is opportunity for more partnership or collaboration between the Ulnooweg Development Group and the CBDCs.  Based on information from CBDC survey respondents, it seems that some CBDCs are better serving the needs of Aboriginal communities than are others

Therefore,

Recommendation #6:  ACOA encourage the Atlantic and provincial associations to collaborate with the Ulnooweg Development Group to identify opportunities for greater co-operation and collaboration between Ulnooweg and the CBDCs.

Through their loan portfolios and business services, the CBDCs have been successful in improving clients’ business skills and knowledge, creating new business start-ups, and maintaining and strengthening existing businesses.  The majority of clients surveyed stated that they would not have been able to start, maintain or expand their businesses without CBDC support.  As well, over 20% of clients indicated they were able to leverage other funds as a result of their CBDC loans. 

Community economic development projects and the development of community strategic plans are not part of the mandate of the CBDCs in Atlantic Canada.  However, stakeholders still believe that the CBDCs have been successful in supporting community economic development, assisting communities to develop and diversify their economies, and strengthening community capacity, through their loan activities and business services.

The extent to which the program has achieved its long-term goals (i.e. economic growth and stability, diversification and development of local rural communities, sustainable communities, and survival of business assisted by the CBDCs) is more difficult to measure given that the program has a limited degree of influence on indicators measuring achievement of long-term goals (e.g. change in employment rate, change in employment distribution, and change in annual average household income), as well as difficulties with attribution.  However, an ACOA economic impact analysis suggests that the CF Program has contributed to economic growth in the Atlantic Provinces.  Survey respondents and interviewees also believe that the CF Program is contributing to its long-term goals.

The CF Program in Atlantic Canada has not had any true unintended impacts.

Cost-Effectiveness and Alternatives Conclusions

As mentioned previously in the design and delivery section, it is not possible to determine whether ACOA is cost-effective in its administration of the CF Program, since ACOA’s operating costs for the program are not tracked separately from those of other ACOA programs.  The CBDCs appear to be cost-effective in administering the CF Program, although no other programs been identified for comparative purposes. 

CF Program investment dollars per job created or maintained varies significantly across the provinces, from a low cost of $9,944 in Prince Edward Island to a high of $19,714 in Newfoundland and Labrador.  Given that this investment is paid back to the CBDCs by the respective clients, it would appear to be a worthwhile investment.  As well, over the five-year period, a total of $191.6 million was leveraged, amounting to $0.84 per dollar invested. Ratios for funds leveraged per dollar disbursed also vary greatly across the regions. 

Therefore,

Recommendation #7:  ACOA undertake a study to identify the reasons for the variances across regions in investment dollars per job created or maintained, and in funds leveraged per dollar disbursed.  This will allow ACOA to determine whether there are best practices or strategies in place in some regions that could be transferred to other locations to allow the CBDCs to improve on these measures.

The current delivery model in place for the CF Program is viewed as the most effective model.  It integrates a number of components that are important to successful community economic development, including a grassroots approach, where decisions are made at a local level; the involvement of local community volunteers; and the focus on partnership development and co-operation between community organizations. A high-level costing analysis shows that administration of the CF Program through the CBDCs is more cost-effective than having ACOA deliver the program itself.

Appendix A:  BREAKDOWN OF CLOSED BUSINESSES

*Note:  Although 30 survey respondents indicated their businesses were no longer in existence, not all provided additional tombstone information, so total numbers in the tables below do not include all 30 responses.

Table A: 

Region
# of closed businesses
New Brunswick 3
Nova Scotia (except Cape Breton) 9
Cape Breton  3
Newfoundland and Labrador 4
Prince Edward Island 0
Total 19

Table B: 

Average Gross Annual Revenues
# of closed businesses
Less than $50,000 16
$50,000 to $99,999 6
$100,000 to $199,999 3
$200,000 to $299,999 3
$400,000 to $499,999 1
Don't know / refusal 1
Total 30

Table C: 

Gender
# of closed businesses
Male 19
Female 11
Total 30

Table D: 

Age Category
# of closed businesses
Under 35 11
35 to 50 14
Over 50 5
Total 30

Table E: 

Other Attributes
Yes
No
Aboriginal 1 24
Visible Minority 1 24
Person with a disability 2 23

Appendix B:  BREAKDOWN OF FTE JOBS CREATED/MAINTAINED
 
 
 Number of F/T Equivalent Jobs Created
  2003-2004 2004-2005 2005-2006 2006-2007 2007-2008 TOTAL
PEI 50.3  43.0  42.0  20.0  35.0  190.3
NL 229.8  334.5  177.5  202.0  171.9  1,115.7
NB 484.9  442.0  436.5  345.1  548.7  2,257.2
NS 331.8  333.9  352.1  249.5  274.6  1,541.8
CB 81.0  60.5  67.0  74.5  64.5  347.5
TOTAL 1,177.7  1,213.9  1,075.1  891.0  1,094.7  5,452.3 

 

 Number of F/T Equivalent Jobs Maintained
  2003-2004 2004-2005 2005-2006 2006-2007 2007-2008 TOTAL
PEI 183.0  226.0 152.5 133.5 146.0 841.0
NL 301.0 370.5 392.0 332.0 322.8 1,718.3
NB 751.0 961.6 878.0 936.7 867.2 4,394.6
NS 529.3  522.8 572.6 688.8 613.8 2,927.2
CB 85.0 144.5 179.5 80.5 125.5 615.0
TOTAL 1,849.3 2,225.4 2,174.6 2,171.6 2,075.2 10,496.0
Appendix CACOA ECONOMIC IMPACT ANALYSIS

Economic Impact Analysis

The economic impact analysis estimates the economic benefits of the CBDCs’ Community Futures (CF) Program in Atlantic Canada.  The results are obtained through investment expenditures from commercial projects as well as the direct creation of jobs from these projects.  These are called the direct impacts, as they can be attributed to specific projects.  Through the CBDCs’ direct support to business, the di rect creation of jobs can be utilized to estimate the indirect and induced impact on the economy of the four Atlantic Provinces.  In the analysis, the jobs are translated into a measure of value-added output by economic sector.  From the value-added output, the economic impact is estimated using econometric models developed by The Conference Board of Canada for each Atlantic province.  In this manner, direct, indirect and induced macro-economic effects, in particular the impact on gross domestic product (GDP), employment, wages and tax revenues, are estimated.

See Header

Gross Domestic Product

It is estimated that real GDP was $150 million higher (in constant 1997 dollars) in 2007 than it would have been in the absence of the CF Program’s expenditures in direct support of businesses.  (Real GDP is the inflation-adjusted value added of products and services produced in a given year in an economy.)

Furthermore, over the five-year period, CF program support to businesses for commercial projects  produced increases of over $2.35 in GDP gains for every dollar of expenditure[67].

Employment

Total employment in the Atlantic Provinces was over 3,300 higher (including direct, indirect and induced employment gains) in 2007 than it would have been without CBDCs’ direct support to businesses over the previous five years, through the CF Program.  The employment numbers shown are gross estimates and do not take into consideration the net job gains or losses as a result of economic conditions.

See Header

Hypotheses

For a description of the methodology and approach used in estimating the economic impact analysis of the CF Program on Atlantic Canada’s economy, refer to the Technical Appendix. It is important, however, to note several features and hypotheses used for these estimates. 

  1. Jobs actually created from successful projects are used to calculate benefits. Estimated jobs created at the firm level are tracked through the CBDCs’ Quarterly Reports.
  2. Jobs maintained, although they do produce real economic benefits, are not included because of difficulties in providing a reliable measure of economic impact.
  3. Capital expenditures, broken down into construction, and machinery and equipment, are used to estimate the economic impacts. It is important for expenditures to be segregated because the impact differs by category.
  4. Incremental projects are used in the estimation of the economic benefits. Incremental projects are projects that would not have occurred in the absence of the financial assistance made available by the CF Program.
  5. Benefits include direct employment gains by ACOA-assisted businesses, plus indirect and induced employment as estimated by the econometric models. The indirect impact is generated through the purchase of goods and services from suppliers by assisted firms.  Induced impacts occur as incomes associated with direct and indirect employment gains are spent and re-spent throughout the economy.

TECHNICAL APPENDIX ON METHODOLOGY AND APPROACH

Introduction

The following pages contain a section describing the methodology and approach used in estimating the economic impact of the CF Program on the Atlantic region’s economy. The approach used is a refinement of the methodology used by the ACOA in its 2003–2008 Five-Year Report to Parliament. 
Methodology for Estimating Economic Impact 

 

PROCESS
OUTPUT
CBDC Quarterly Reports database on investment patterns
  • Project level expenditures, total by province
  • Expenditures broken down by construction investment, and by machinery and equipment (refer to Note 1)
CBDC data on employment creation for commercial projects and surveys
  • Incremental direct actual jobs created
  • Incremental impact of projects on investment expenditures (refer to Note 2)
Statistics Canada data on relationships between jobs, wages and salaries and value added by industrial sector
  • Incremental direct value added by industrial sector, by province
  • Incremental value added by sector, by province, adjusted for project failure rate and overlap with other ACOA programs
Results from Government Consulting Services survey carried out on CBDC clients for the CF Evaluation
  • Total impact including indirect and induced effects on GDP, employment and tax revenues
The Conference Board of Canada’s Atlantic Provinces econometric models – investment and sector value-added adjusted by incrementality and failure rate

A logic model of the methodology presents, in graphic form, the process followed in the calculation of the economic impact, and identifies the source of data used and instruments of analysis. The model also identifies the output identifying measures used at each step of the process. Additional details of the methodology are presented in the pages following the logic model.

NOTES TO TABLE

  1. It is important that expenditures be segregated into construction investment and machinery and equipment purchases because the impact on the Atlantic economy differs by category. In the case of machinery and equipment spending, most of the equipment would be manufactured outside Atlantic Canada, resulting in minimal indirect and induced benefits.  Construction investment, on the other hand, has a greater impact as local labour and materials are employed.
  2. The Auditor General, in his report of November 1995, Chapter 18, defines an incremental project as one that would likely not have proceeded with the same scope, at the same time, and in the same location, without government assistance.[68]

Notes concerning the precision of the estimates

  1.    The Conference Board of Canada Models

    The Conference Board of Canada models are econometric models of the four Atlantic Provinces.  Each provincial model consists of 11 industry groups. The major advantage of the Conference Board models is that each province is modelled explicitly. Other available econometric models determine effects using a national model, and then calculate provincial activity by dividing the national activity into fixed shares for each province using historical information. This allocation approach is likely to attenuate the effects of programs such as those administered by the CBDCs, which alter the share of GDP accounted for by the Atlantic Provinces in any particular industry.  The estimates of the effects on the Atlantic region are likely to be more accurate using the Conference Board models.

    The Conference Board models were selected because of the importance of accurate regional impact modelling; however, the fact that it is not highly disaggregated by industrial sector (i.e. 11 industry groups as opposed to over 100 groups for some competing models, poses limitations on precision.

  2.    Incrementality

    When investment is made in a business, there is always a possibility that the project would have been carried out with or without the government’s help. If the investment would have been carried out anyway, then the only effect of the government’s involvement is to add to the wealth of the entrepreneur. For the CF Evaluation, incrementality was estimated by the use of post facto survey questions such as: “Would this investment have taken place without aid from CBDCs?” and “If this investment would have taken place anyway, would it have been delayed without assistance?” Some entrepreneurs will provide answers they think the questioner wants to hear, in order to get them out the door, or not to prejudice their chances of receiving further assistance. Other business people are fiercely independent and reluctant to admit that government support has played any role in their success. It is reasonable to assume that these two phenomena will balance each other.

    For the impact analysis of the CF program, in order to provide a robust evaluation of the impact of the CBDCs, and to remain conservative, the data was adjusted for incrementality. The incrementality factor has been estimated at 61.5%, obtained from the evaluation program survey.

  3.      Estimated versus Actual Created Jobs

    Through the tracking module, the CBDCs monitor the estimated employment that will be created by assisted projects. For the analysis, the estimated amount of jobs created need to be transformed into actual jobs created. For the CF Program, actual job creation has been estimated at 95.2% of expectations by means of an independent survey of recipient firms. This factor was used to discount estimated jobs created for the CBDCs' commercial projects in order to arrive at actual jobs. 

  4.      Success/Failure Rate and Overlap with ACOA Programming

    Successful clients are identified through the client survey and only jobs associated with successful projects are included in the count.  For the impact analysis, expected job creation counts are discounted by annual rates of 7.6% for unsuccessful projects supported by the CF Program. Finally, employment data are also adjusted for an overlap between the CBDCs and other ACOA programming.  As a result, the data are discounted by an annual rate of 14.8%for the CF Program.
     


Part II - Main Results and Achievements 
Performance Indicators

Results and Achievements

Methodology


Gross Employement

 

 

 

 

 

 

 

 

Total gross employment was over 3,300 higher in 2007 than it would have been without the CF Program.

 

 

 

 

 

 

An estimate of actual jobs created is calculated from a project-by-project count of job impact for those projects in direct support of business.  Estimated jobs are adjusted by a factor of 0.952 to arrive at actual net employment gains on a company basis.  These factors are derived from an independent survey of clients.  Actual jobs are adjusted for incrementality (0.615).  The Conference Board models produce a multiplier of about two, meaning that each job created directly creates one other job through spin-off spending.

Seasonal and part-time employment is converted to full-time (1,600 hours/year) and long-term (five-year duration) equivalents.

Using job numbers from above, the value added per employee, as determined by the model, is entered into the Conference Board models to estimate overall impact on GDP, total employment and taxes.

Impact on Atlantic GDP

 

 

 

 

 

 

 

 

 

 

 

 

 

Total Atlantic GDP was estimated to be $150 million higher in 2007 that it would have been without the CF Program.

For each dollar invested directly in businesses through the CF Program, there were $2.35 gains in Atlantic GDP.

 

 

 

 

 

 

 

 

Jobs actually created from projects are used to calculate benefits.  The calculation is deliberately conservative since the job numbers are adjusted for:

  • Incrementality factor (61.5%);
  • Failure rate (excludes jobs for commercial projects which failed, estimated at 7.6%);
  • Overlap withother ACOA program (14.8%).

Also, jobs maintained are excluded from the analysis.

The methodology is a refinement of the methodology used in ACOA’s Five-Year Report to Parliament for 2003-2008.

In reviewing the underlying assumptions of this earlier methodology, PriceWaterhouseCoopers concluded in 1998 that “ACOA’s process and assumptions are reasonable…we even suggest that some argumentation could be developed that would imply that the process and assumptions are conservative.”

The analysis includes personal income taxes on wages earned by employees and sales taxes on purchases made by the companies and their employees.

Impact does not include corporate taxes or estimates of savings to government on employment insurance or welfare payments.

See Header

Management Response and Action Plan

Footnotes

[51] Roll-Up of CFP Formative Evaluation Reports from ACOA, CED-Q, WD and FedNor :Revised Final Report, (September 2003), page 39.
[52] CF Program RMAF - Annex C, (May 2005), page 7.
[53] The Community Futures ACOA/CBDC Agreement requires that 5%  or more of the total population speak the minority language in a majority area to declare such an area an OLMC.
[54] Ulnooweg Development Group Inc : http://www.ulnooweg.ca/about.php
[55] Pan-Canadian Community Futures Network,  Impact Assessment of Community Futures in Canada, (March 2006), page 22.
[56] Full time equivalent (FTE) is the “ratio of total number of paid hours during a period (part time, full time, contracted) by the number of working hours in that period.”  http://www.businessdictionary.com/definition/full-time-equivalent-FTE.html
[57] The codes in the table represent the standard NAICS codes that are used across North America. The “Other industries” category cannot be further broken down, as the information is not available in the E-reports system. However, based on anecdotal information provided by the CBDCs, the “Other industries” category includes personal services like barber shops, estheticians, professional service providers (lawyers, accountants, management consultants, etc).
[58] For the purposes of this evaluation, FedNor developed this definition of community capacity from existing material.   No definition of community capacity was provided by ACOA.
[59] Economic impact analysis provided by Performance Measurement, Analysis and Reporting Unit, ACOA (October 29, 2008).
[60] Community Futures Program Pan-Canadian Impact Evaluation, National Terms of Reference, (July 2007), page 3.
[61] Program costs are defined as the cost to ACOA to administer the program, not including funds provided to the CBDCs (i.e. ACOA salaries, and operational and maintenance costs).
[62] Senate Canada,  Beyond Freefall: Halting Rural Poverty, Final Report of the Standing Senate Committee on Agriculture and Forestry, (June 2008), page 297 and page 301.
[63] ACOA, Policy and Programs Management Response – Evaluation of the Community Futures Program, (January 2004), page 4.
[64] Copies of annual reports for the Atlantic Association of CBDCs, New Brunswick Association of CBDCs, CBDC Long Range, and the Newfoundland and Labrador Association of CBDCs are available in the news item section of each organization’s respective websites.
[65] ACOA, Evaluation of the Community Futures Program as Delivered through the Atlantic Business Development Corporations, Final Report, (May 2003), page 35.
[66] ACOA, Policy and Programs Management Response to the Evaluation of the Community Futures Program, (January 30, 2004), page 4.
[67] Expenditures are the sum of program spending (loans to businesses) and operating expenditure by the CBDCs for the Community Futures Program.
[68] Report of the Auditor General to the House of Commons: “Chapter 18 Atlantic Canada Opportunities Agency – Economic Development”, (1995).