Working Systematically

Site map |

Working systematically: Some ideas about data

What are data?

Data are information, including observations and perceptions as well as quantitative information derived from sources such as test scores. They can be collected, for example, from surveys, from tests, from roll books, from teacher judgements and from collected opinions or formal assessments.

Provisions related to privacy and the safeguarding of personal information should, of course, be carefully observed.

What are they for?

In this case — to give you a clear idea about how you're going, to provide an effective basis for discussion and analysis of action and its results, to record progress, and to get a clear picture of what has happened.

You are also likely to need them for reports, for publicity, for allocating resources and seeking additional resources.

What can you collect data about?

You can collect data about anything, and you can waste a lot of time doing so. As suggested earlier, it is best to concentrate on a limited number of important indicators which are well formulated and can be reliably and fairly easily evidenced.

These materials suggest two focal points:

literacy and numeracy skills, and
completion rates

Where the numbers of students are small be judicious in your analysis. Small numbers can skew data quite dramatically.

which are both underpinned by

participation rates.

Even given a level of transience, it is possible to collect very accurate data on attendance and completion rates. Where there is a high level of transience, you need to identify your students carefully.

With literacy and numeracy skills the picture may be less exact. It is important to be clear and specific. So you say, for example, that: ‘Using X measure [at least twice] our Y students progressed Z. This compares with [state-wide averages, national rates,’ like school’ rates]’. Rather than: ’Our Y students are X’.

Current tests in wide use are vastly improved from those from a decade or so ago. The easy criticisms that were heard then have generally been attended to and there have been very serious efforts to rectify matters like cultural bias. The ACER National Literacy Mapping Survey is a good example. (You can read about this in Masters and Forster (1997) Mapping Literacy Achievement: Results of the 1996 National School English Survey available from ACER, 19 Prospect Hill Road, Camberwell, VIC 3124.) Results won’t be perfect, but they will generally be a good guide to what is going on.

How do you collect data?

Decide what you want information about.

Don’t collect information just for the sake of doing so. Time is too short. Collect information which:

is central to the priority issues you are interested in
will tell you what you want to know
you can measure as reliably as possible.

Decide how you're going to get it.

Find a suitable performance indicator. A performance indicator is something, a focal point, which tells you what's happened and how well you've done. For example,

— 'The proportion of Indigenous students receiving intensive literacy support', or

— 'The extent to which students achieve attendance target set out for them', or

— 'The extent to which families believe their children's needs are being met'.

You might need an instrument. The method used to collect the data is a process, but at its heart is usually an instrument. Surveys are instruments; a form for summarising attendance data is an instrument; the Basic Skills Test (itself) is an instrument, a list of questions you ask one or more people is an instrument.

Establish a baseline.

A baseline describes your starting point, where you are now. For example:

— ‘At the end of 2004, 68 percent of your Year 4 students were working at Level X in various aspects of literacy’,or,
— ‘At the beginning of 2005 no Indigenous staff were employed’, or
— ‘In May 2003 four teachers had incorporated Aboriginal perspectives in their courses’.

After a suitable period of time do just what you did before.

In order to make valid comparisons over time or across groups the same method or instrument should be used. This is basic to making valid and reliable comparisons.

For some examples of this process, look in THE WORKBOOK.

How do you analyse data?

In simple terms …

Make comparisons with other like data

Like data is the same information you are collecting, and collected by the same means.

You can make comparisons with how this group of your students performed compared with;

their performance previously

other students working in the same area at a different time (e.g. this year’s Year 9 literacy results compared with last year’s).

Be judicious in your analysis where the numbers of students are small. Small numbers can skew data quite dramatically.

Make comparisons with other like data from other sources.

You can make comparisons with:

the performance of the same individual or group of students, and/or

the performance of differing groups of students, like a school-wide group, a ‘like school’ group, a state/territory-wide group, or national results. (‘Like-school data’ is information collected using the same or equivalent instrument in a school like yours in terms, for example, of size, proportion of Indigenous students, makeup of the school population, and context.)

Think about the reasons for what you've found

This is the hard part. The relationships between outcomes and their causes are sometimes difficult to establish in education. You might need to investigate further and explore other sources of information. The best analyses come from using data from a range of sources. The case studies in these materials provide some good examples of this process.