Redirecting...

Chapter 14

SECURITY AND HEALTH

14.7 APPROACHES TO EVALUATING CRIME PREVENTION / YOUTH PROGRAMMES

14.7.1Legislative and Policy Framework

ACTS

  • The South African Schools Act, No. 84 of 1996 [SASA]
  • Employment of Educators Act 76 of 1998 [EEA]

14.7.2Framework for the Development of School Policy on Approaches to Evaluating Crime Prevention / Youth Programmes

  1. Why evaluate?
    • There is generally a lack of information in South Africa on what kinds of programmes work in which situations and why. Evaluating programmes helps us to choose which one is best for our school.
    • Evaluations tell us whether our intervention is working and which aspects of it are working best, and where we need to improve. It is a way of finding out whether we have reached the goal that we set for ourselves.
    • Evaluating a programme helps to convince potential donors to give money to the programme. It also tells them how their money is being spent, and how effectively it is being spent.
    • Evaluations provide us with proof that our programme works!
    • Evaluations often highlight new problems that need investigating in the school. For example, a programme aimed to reduce drug use among learners may alert you to the fact that there are very low levels of parental supervision after school hours.
  2. Getting to grips with the language of evaluation
    Throughout the unit several terms will be used. These terms are explained below.

    Terms Definition
    Objectives Your goals. The results you want to achieve through your programme.
    Inputs The resources you need in order to carry out your programme. These may be educators, funds, resources etc.
    Outputs The products of the programme. For example, “six high impact educational workshops”, a “training course for educators resulting in six trained educators”.
    Impact/Outcome The result of the programme. For example “a 50% reduction in drug use in the school following an education workshop”.
    Monitoring Keeping track of the programme progress. Monitoring what you are investing in the project (such as time, money) and whether the programme is being carried out according to plan. This is the most basic form of evaluation.
    Evaluation Finding out whether the programme is achieving its goal – such as reducing high rates of absenteeism in the school.
    Confounding variables External factors (outside of your programme) that may shape your programme results. If you are running self-esteem workshops for learners, a potential confounding variable could be a similar project being run by the local church. When you measure the learner’s self-esteem you won’t know whether it is due to your programme or the church’s programme.
    Formative evaluation This is an evaluation of the process of implementing your programme i.e. not the results, but the way in which the programme was run. Was the programme implemented as planned? Were there factors that stood in the way of its implementation? What parts of the programme went smoothly?
    Outcome evaluation or summative evaluation This asks the questions: “What changes took place? What was the impact of our programme? Did it change things at school?” For example: Compared to before the programme was implemented, how much has drug use in the school decreased?
    Indicators An indicator is used to measure how successful the programme has been. For example, finding out how many learners have been absent from the attendance register would be an indicator of rates of absenteeism. If you want to know whether levels of attendance at school have increased, your baseline data could be the attendance records before the programme was introduced. These will be compared with the attendance records after the programme has been run.
    Base-line information This is information you need before the programme starts so that you can measure it against information at the end of the programme. For example if your programme aims to increase the awareness of HIV/AIDS, you will need to do a survey with learners before you begin the programme to record their opinions and knowledge. This information is called base-line information. After the programme is finished you will conduct a summative evaluation to see if learner attitudes and knowledge has changed.
  3. What questions does an evaluation aim to answer?
    • Did we implement all the steps in our programme plan? We all start out with a plan, at the end of the programme we need to ask ourselves, were any steps in our plan left out? Why were they left out?
    • Did we achieve our outcome? An outcome is the final result of the programme: did we achieve our result? If you had planned to reduce truancy, you would ask “have levels of truancy been reduced and by how much?”
    • How do we know that the results (such as reduced truancy) are because of our programme? Were there any other programmes or factors (confounding variables) that might have also contributed to reducing truancy? What are they?
    • What factors made the programme work well?
    • Was the programme successful because educators were very involved? Because parents and the school governing body prioritized the programme? Because the materials were very effective?
    • What factors made it difficult to implement the programme?
    • What obstacles did you face? Where should we place more of our efforts in the future?
    • Is the programme cost effective? This is difficult to measure but you need to ask: “was our programme cost-effective?” Cost may include several things such as money, staff time or equipment used such as paper, pens etc. If clear records are kept of how much is spent, the cost per child enrolled in the programme can be worked out. Similarly, the time invested per child can be established. For example if you introduce a drug education programme for 12 learners for one hour a week over a three-month period (i.e. you invest 12 hours), the cost in time would be one hour per child. This investment can then be weighed up against the success of the programme in reducing drug use among learners.
  4. Who should do the evaluation?
    • Internal versus external evaluations
      Sometimes a person inside the school who has been involved in the programme design does the evaluation. We support the idea of the evaluation being internal; educators do not feel that they are being judged or treated like guinea pigs and the evaluation is more useful because educators ask and answer the questions that are most important to them. There are several advantages and disadvantages of an internal evaluation:

      • Advantages of an internal evaluation
        • People from within the programme often have a better understanding of the programme and how it has worked.
        • It is more cost-effective. Many people leave out the evaluation stage of their programme because they don’t have the resources. This is a very bad idea because it means you are unsure whether your programme makes any difference at all.
      • Disadvantages of an internal evaluation
        • We live in a society where many factors impact on our behaviour, attitudes and lifestyle. It is often very difficult to isolate one programme and say, “the reason that bullying has decreased in our school is because of this one programme”. Ideally, programme implementers should be on the lookout for other factors that may also influence the current programme.
        • Sometimes the kinds of changes that we are wanting are very hard to measure. For example, if we implement a programme to improve the self-esteem of learners, how do we know that it works? Self-esteem is very difficult to measure because we can’t observe or count it. In many cases we have to rely on what learners tell us has changed for them.
        • The language of evaluation can at times be confusing. It is important to remember that the evaluation is for your benefit only and is used to improve your programme and perhaps secure funding for it. It is therefore not necessary for you to use or understand all the jargon that can go with evaluations.
  5. Key principles in evaluating youth programmes
    • There is no right way to do an evaluation. There are many different kinds of evaluations that are useful for different purposes, at different times in a programme’s history.
    • Ideally you should decide what you are going to evaluate and how you will evaluate it at the start of your programme. Therefore if you have to collect base-line information, you can do so at the start.
    • Most programmes in schools spring not from a need to answer a set of questions, but from the desire to act NOW. This action usually expands and improves over time and our evaluation methods should be able to accommodate this.
    • Programme evaluation should be collaboration from the beginning. This collaboration should be between all stakeholders involved in the programme. E.g. parents, learners, educators etc.
    • It is important for an evaluator to be someone who communicates well with the programme staff and is able to clearly explain what is needed for the evaluations, how it will proceed, the costs, benefits and the limitations. This should be planned at the beginning of the evaluation so that you do not collect all your information and then disagree about what it means. This is especially important if an outside evaluator is used.The main purpose of evaluation is to provide information that is useful to the programme and can meet the programme needs. This cannot be accomplished if the evaluation is simply handed over to an evaluation expert without the programme staff understanding what will be done and how it will be done.
    • An incremental approach to evaluation should be taken. This means that the results of the evaluation should be used to build up your programme. For example, if the evaluation of a programme to reduce gun carrying at school shows that children are predominantly bringing their parent’s guns to school, then the next phase of the project may include a parent education component. Also aspects of the programme that were initially thought to be important may, after evaluation, not be important and might be dropped from the next phase of the programme.
      Educators are reluctant to spend time doing evaluations. How can we change this so that educators support evaluations?
    • Make sure everyone “buys into” the value that evaluations add to programmes, research and funding.
    • A lack of resources should not prevent an evaluation from taking place. Rather, when resources are scarce, evaluations are essential so that you can spend resources in the place where they will be most effective.
    • If you make sure from the beginning that the evaluation is a consultative process and that all programme staff understand the value of the evaluation and how it will be done, they will be more supportive of the evaluation.
    • Make evaluations participatory and collaborative. All programme staff, parents, and community members will have ideas about how the programme has worked and how it should be changed. It is these ideas that are most useful for evaluations.
    • Design evaluations that are realistic and that do not place too much pressure on programme staff.
    • Ensure that programmes will benefit directly from the evaluation. Make sure that the questions you ask will inform the running of, and the content of, the programme.
    • Make sure that the questions you ask can be measured. For example it is not useful to ask whether children are happier. Rather ask how are they happier? Are they doing better at school? Have their relationships improved?
      Developing good indicators

      Indicators should be related to your objectives. If you aim to reduce racism in school, an indicator could be the amount of time learners spend with members of race groups other than their own before and after the programme.

      Indicators must be clear, specific and measurable.

      Indicators should describe the result and the degree to which the result has taken place. It is not enough to say that attendance in class has improved, how much has it improved? How many children that were truant before the programme are now attending regularly?

      Indicators can also include the aspects of the daily lives of the programme staff. They may include questions such as “how did you spend your time?” If during a self-esteem workshop most of the educators time was spent discussing problems of child abuse, then perhaps child abuse needs to be addressed as a separate aspect of the programme or instead of the existing one.

      All involved in the programme must agree on the indicators that will be used.

  6. Some ideas to get you started
    • How to evaluate a programme that is going to be implemented?
      Example: tackling bullying in school
      You will first plan your programme carefully
      For example:

      • Programme objectives
        Broad objectives of the programme:

        • To create a school environment where learners feel safe and can concentrate on their studies.
        • To teach learners to respect one another and to express aggression in healthy ways.
      • Specific objectives:
        • To reduce the levels of bullying in the school.
    • Designing an evaluation framework
      You now need to ask “what do we want to measure in this programme?”

      • Do you want to measure the process of how the programme was implemented and how well you followed the plan?
      • Did you want to evaluate the impact of the programme? What did it change at your school?
      • Do you want to evaluate the cost of the programme?Let’s say you want to measure all three aspects of your programme:
    • Measuring the process of how your programme was implemented:
      • Keep a record of which activities took place, when they took place, who implemented them and how long they took to implement. Record anything that happened during these activities e.g. low learner turn out, workshop raised new problems. You will use these records at the end of your programme to see if you followed the programme plan and to see whether activities changed along the way to make the programme more effective. Another useful tool is to get the project participants to keep diaries throughout the programme. In this way you might be able to track the process better.
      • What activities took place that were not included in the plan? What effect did these changes have on the programme?
    • Measuring the impact of your programme:
      • Your programme aims to reduce the levels of bullying. How will you do this? Develop a set of indicators that will help you measure the decline of bullying. Do you have any records on how many children were reported by monitors for bullying before you started the programme? Or how many learners complained of being bullied? At this point you may want to collect some of this base-line information by having educators count the number of incidents of bullying that they see in the school. Remember to define carefully what you mean by bullying!
      • You may set aside one week to collect base-line information. Each educator will be given a book in which they note every incident of bullying that happens in their class and what kind of bullying it is. At the same time, if possible, records will be collected to see how many reports of bullying there have been in the six months prior to the programme. Someone will need to summarise this information.
      • After the programme is finished, count the number of incidents of reported and counted bullying before and after the programme. Work out by what percent bullying has decreased. For the focus group discussions, take detailed notes and find out what the most common themes are. For example, do most learners feel that the programme helped them to feel safer? Did they feel that it lessened bullying in the classroom but not on the playground? Did they find the education workshops too short or too long?
      • As part of your strategy to reduce bullying you may implement a programme aimed at teaching monitors conflict management skills and learners how to manage anger. You will need to think about how both of these could be evaluated. How will you decide whether learners are managing their conflicts any better than they did in the past?
    • Measure the cost of the programme
      • Cost may include several things such as money, staff time or equipment used such as paper, pens, etc. If clear records are kept of how much is spent, the cost per child enrolled in the programme can be worked out. Similarly the time invested per child can be established. This amount will then be measured against the success of your programme. The calculation is done by taking the overall cost of the programme and dividing it by the number of learners involved in the programme.
      • (Overall cost of project/number of learners = cost per learner). For example, if the project cost R200 000 to complete and 40 learners were involved in the programme, then the cost per learner is 200 000 / 40 = R5000. The cost is R5000 per learner. You can then decide if this is too costly to take to other schools. You can also calculate the budget if you wanted to re-run the programme with 400 more learners.
        Timing your evaluation

        Do not evaluate when learners are writing exams or otherwise distracted. If there has been a serious incident of violence on the school grounds, this may impact on the levels of bullying in the school. For example, if there has been a shooting, levels of conflict may be very high. This needs to be taken into account when you evaluate your programme. The shooting would be an example of a confounding variable. Also consider for how long you want to conduct the evaluation. For example, one week after the programme bullying may have reduced by 50% but is this still true six months later? The length of your evaluation will depend on the resources and time you have available, and what indicators you use.

  7. Helpful Contact Numbers:

EasternCape

Sexual Abuse/Child Abuse HIV/AIDS Safe Schools/Life Skills Substance Abuse
ChildLine
Tel: 08 000 55 555
Tel: (043) 722 1382
adminec@childlinesa.org.za
AIDS Training Information and Counselling Centre (ATICC)
City Health Dept, 30 Beaconsfield Road, East London 5201
Tel: (043) 705 2969
atic@iafrica.com
Project for Conflict Resolution and Development
22 Hurd Street
Newton Park, Port Elizabeth
Tel: (041) 585 5688
www.facebook.com/pages/Project-For-Conflict-Resolution-Development/1189675111043863
SANCA
22 St Marks Road, East London
Tel: (043) 722 1210
info@sancacec.co.za
www.sancacec.co.za
Family and Marriage Society of South Africa (FAMSA)
62 Western Road,
Port Elizabeth
Tel: (041) 585 9393
pe@famsa.org.za
NACOSA
1st Floor, Office No 3, Frere Square, 58 Frere Road, East London
Tel: (043) 726 2146
ecadmin@nacosa.org.za
www.nacosa.org.za

FreeState

Sexual Abuse/Child Abuse HIV/AIDS Safe Schools/Life Skills Substance Abuse
Family and Marriage Society of South Africa (FAMSA)
10 Strauss St., Bloemfontein
Tel: (051) 525 2395
famsabfn@xpd.co.za
AIDS forum
Thaba Nchu, Selosesha
Tel: (051) 873 2233
The Centre for Citizenship Education & Conflict Resolution
Tel: (051) 448 8200
SANCA
15 Brompton Road, Naval View, Bloemfontein
Tel: (051) 4474111
aurorasentrum@xsinet.co.za
www.auroracentre.co.za
Childline
Tel: (051) 430 3311
reception@cwcl.org.za
www.childwelfarebfn.org.za
NACOSA
7 Brill Street, Bloemfontein
Tel: (051) 011 0587
fsadmin@nacosa.org.za
www.nacosa.org.za
NICRO
No 12 Tannery Road, Hamilton, Bloemfontein
Tel: (051) 435 5193
marita@nicro.co.za
www.nicro.org.za

Gauteng

Sexual Abuse/Child Abuse HIV/AIDS Safe Schools/Life Skills Substance Abuse
Family Life Centre (FAMSA)
15 Pascoe Avenue, Kempton Park
Tel: (011) 975 7106/7
national@famsa.org.za
www.famsa.org.za
AIDS Consortium
66 Sailor Malan Avenue, Omnipark Block 1, First Floor, Aeroton 2013, Johannesburg
Tel: (011) 403 0265
info@aidsconsortium.org.za
www.aidsconsortium.org.za
Centre of Violence and Reconciliation
33 Hoofd Street, Braampark Forum 5, 3rd Floor, Johannesburg
Tel: (011) 403 5650
info@csvr.org.za

www.csvr.org.za
SANCA (National
Directorate)
2 Whitney Road, Whitney Gardens, Lyndhurst, Johannesburg
Tel: 086 14 72622
Tel: (011) 892 3829
sancanational@telkomsa.net
www.sancanational.info
Childline
Tel: (011) 645 2000
admingauteng@childline.org.za
www.childlinegauteng.co.za
NACOSA
Iparioli Office Park A2, 1166 Park Street , Hatfield, Pretoria
Tel: (012 ) 940 2829
lindiwe@nacosa.org.za
www.nacosa.org.za
NICRO
Room 544 5th Floor Van Erkom Building 217, Pretorius Street Pretoria Central
Tel: (012) 326 8115
www.nicro.org.za

KwaZulu-Natal

Sexual Abuse/Child Abuse HIV/AIDS Safe Schools/Life Skills Substance Abuse
Childline
Tel: (031) 312 0904
reception@childlinekzn.org.za
www.childlinekzn.org.za
AIDS Foundation of South Africa (AFSA)
2nd Floor, Standard Bank Centre, 135 Musgrave Road, Durban
Tel: (031) 277 2700
info@aids.org.za
www.aids.org.za
Independent Projects Trust (IPT)
Crime Reduction in Schools Project (CRISP)
2702 Old Mutual Centre, 27th Floor, 303 West Street, Durban
Tel: (031) 260 2366
iptnet@wn.apc.org
www.ipt.co.za
SANCA
185 Vause Road, Berea, Durban
Tel: (031) 303 2274
sancapmb@mweb.co.za
www.sancadbn.co.za
Family and Marriage Society of South Africa (FAMSA)
30 Bulwer Road, Berea, Glenwood, Durban
Tel: (031) 202 8987
famsadbn@mweb.co.za
www.famsa.org.za
NACOSA
Unit 5 Pinewood Office Park, 18 Underwood Road, Pinetown
Tel: (031) 701 1039
kznadmin@nacosa.org.za
www.nacosa.org.za
NICRO
G 14 Wheeler House, 40 Linze Road, Greyville
Tel: (031) 309 8333/6/9
jobm@nicro.co.za
www.nicro.org.za

Limpopo

Sexual Abuse/Child Abuse HIV/AIDS Safe Schools/Life Skills Substance Abuse
Family and Marriage Society of South Africa (FAMSA)
No. 8 Second Avenue, Medipark, Tzaneen
Tel: (015) 307 4833
famsatzaneen@telkomsa.net
www.famsalimpopo-tzaneen.co.za
AIDS Training Information and Counselling Centre (ATICC)
Cnr Potgieter & Diaz Streets, Polokwane
Tel: (015) 290 2363
Youth Commission
154 Van Rensburg Street, Polokwane
Tel: (015) 291 3678Naledi Foundation
Office number 7
Mtititi Thusong Service Centre, Plange Village, Malamulele
Tel: 079 971 6532
info@naledifoundation.org
www.naledifoundation.org
SANCA
33 Kerk Street, Polokwane
Tel: (015) 295 3700
info@sancalimpopo.co.za
www.sancalimpopo.co.za
Childline
18 Hans van Rensburg Street (Unit 1), Polokwane
Tel: (087) 943 6539
mufhandur@childlinelim.org.za
www.childlinelim.org.za
AIDS Foundation of South Africa (AFSA)
Tel: 087 353 7142
NICRO
No 28 Jorrison Str (between Biccard & Voortrekker streets) Polokwane
Tel: (015) 297 7538
nthabi@nicro.co.za
www.nicro.org.za

Mpumalanga

Sexual Abuse/Child Abuse HIV/AIDS Safe Schools/Life Skills Substance Abuse
Greater Nelspruit Rape Intervention Project (GRIP)
17 Ehmke Street, Nelspruit/Mbombela
Tel: (013) 752 4404
Cell: 083 310 1321
info@grip.org.za
www.grip.org.za
AIDS Training Information & Counselling Centre (ATICC)
7 Bell Street, Nelspruit
Tel: (013) 759 2167
Manna for Youth
PO Box 4148, Witbank 1035
Tel: (013) 656 2793
SANCA
Help Centre
8 Hope St, Nelspruit
Tel: (013) 755 2710
Cel: 082 451 3226
admin@sancalowveld.co.za
www.sancalowveld.co.za
Family and Marriage Society of South Africa (FAMSA)
6 Bester Street, North Nelspruit
Tel: 061 960 1549
famsanelspruit@vodamail.co.za
www.famsa.org.za
AIDS Foundation of South Africa (AFSA)
Tel: 087 353 7140
www.aids.org.za
NICRO
20 Ferreira Street, Volante House, 1st and 2nd Floor, Nelspruit (Mbombela)
Tel: (013) 13 755 3540 / 013 755 3745
claudine@nicro.co.za
www.nicro.org.za
Childline
Tel: (013) 752 2770
Phumzile@childlinempu.org.za
www.childlinempu.org.za

NorthernCape

Sexual Abuse/Child Abuse HIV/AIDS Safe Schools/Life Skills Substance Abuse
Childline
6 York Street, Kimberley
Tel: (053)8325962
administration@childlinenc.org.za
AGANG Aids Service
272 Jackson Makodi Street, Unit 1, Pampierstad
Tel: (053) 996 1254
NICRO
9C, Roper Street, Kimberley
Tel : (053) 831 1715
kuki@nicro.co.za
www.nicro.org.za
SANCA
8 Knight Street, Kimberley
Tel: (053) 8325216
sancakimberley@telkomsa.net
Family and Marriage Society of South Africa (FAMSA)
4 Fenton St, De Beers Building, Kimberley
Tel: (053) 872 2644
zolekabula@yahoo.com

NorthWest

Sexual Abuse/Child Abuse HIV/AIDS Safe Schools/Life Skills Substance Abuse
Childline
31 Rietief Street, Potchefstroom
Tel: (018) 297 4411
directorclnw@gmail.com
NWU AIDS Office
NWU, 246 Administration Building
Tel: (018) 389 2001
Pan.mabille@nwu.ac.za
http://www.nwu.ac.za/content/nwu-mafikeng-campus-hiv-and-aids
NICRO
Room 544, 5th Floor Van Erkom Building, 217 Pretorius Street, Pretoria Central
Tel: (012) 326 8115
www.nicro.org.za
Alcoholics Anonymous (AA)
NG Kerk Oosterlig, Corner of Hugo and Lea Street, Waterkloof Glen, Pretoria
Tel: (012) 993-5827
office@npaa.org.za
www.aasouthafrica.org.za
Family and Marriage Society of South Africa (FAMSA)
3 Singer Road, Potchefstroom
Tel: 018 293 2272
potch@famsa.org.za
www.famsa.org.za
SANCA
c/o President Kruger & Bishop Desmond Tutu Streets, Klerksdorp
Tel: 018 462 4568
sanpark@lantic.net
www.sanpark.org.za

WesternCape

Sexual Abuse/Child Abuse HIV/AIDS Safe Schools/Life Skills Substance Abuse
Childline
No. 38 Fleming Road, Wynberg
Tel: (021) 762 8198
info@childlinewc.org
NACOSA
3rd Floor, East Tower Century Blvd, Cape Town
Tel: (021) 552 0804
info@nacosa.org.za
www.nacosa.org.za
Centre for Conflict Resolution
Coornhoop, 2 Dixton Road, Observatory 7925
Tel: (021) 689 1005
mailbox@ccr.org.za
www.ccr.org.za
AL-ANON and Alateen
616 Pearl House, Strand Street, Cape Town
Tel: (021) 418 0021
Family and Marriage Society of South Africa (FAMSA)
9 Bowden Road, Observatory
Tel: 021 447 7951
famsa@famsawc.org.za
www.famsawc.org.za
NICRO
1 Harrington Street, Cape Town
Tel: (021) 462 0017
info@nicro.co.za
www.nicro.org.za
SANCA
18 Karoo Street, Bellville
Tel: (021) 945 4080/1
sanca@sancawc.co.za
www.sancawc.co.za
Rape Crisis
23 Trill Road, Observatory, Cape Town
Tel: (021) 447 1467
info@rapecrisis.org.za
www.rapecrisis.org.za