Annual Survey of Hours and Earnings (ASHE)
ASHE is the main source for data on pay, including low pay.
ASHE data across years cannot simply be combined to produce time trends. This is because non-vatable companies were only included in the ASHE survey from 2004 onwards. In consequence, the data for 2003 and earlier is considered to underestimate the prevalence of low pay and should therefore not be combined with the data for 2004 onwards to produce time trend analyses. It is to partly fill this gap that ONS specifically publish overall UK-wide estimates of low pay by year, using ASHE data from 2004 onwards but a combination of ASHE and the Labour Force Survey for earlier years.
Although ASHE includes data about numbers of jobs, it should not be used for job estimate by industry etc as there are sources, such as ONS Labour Market Statistics, which are to be preferred.
- Available from: the Office for National Statistics website as a series of tables, with additional supplementary tables for Northern Ireland available from the Department of Enterprise, Trade and Investment website.
- Registration required: no.
- First survey available: 1997.
- Frequency: annual.
- Updated: November.
- Scope: UK-wide.
- Format: Excel spreadsheets.
- Files: between 100 and 200 spreadsheets per year, each providing pay data for a given level (e.g. by age) and a given scope (e.g. gross hourly pay).
- Documentation: no manual but the individual spreadsheets have notes and are reasonably self-explanatory.
- Weighted or unweighted: weighted.
- Household income data: no.
The actual ASHE dataset is not publicly available. Rather, what is available is a whole series of tables published by the Office for National Statistics from the dataset. 100 to 200 spreadsheets sounds a lot but they are actually well-organised and numbered: each spreadsheet has a number in the form X.Y, where X is the level of the data and Y is the scope of the data. In the 2006 tables, the various levels were:
|Table number||Type of table|
|Table 1.Y||All employees|
|Table 3.Y||Government Office Region by occupation|
|Table 5.Y||Government Office Region by industry|
|Table 7.Y||Place of work by Local Authority|
|Table 8.Y||Place of residence by Local Authority|
|Table 9.Y||Place of work by parliamentary constituency|
|Table 10.Y||Place of residence by Parliamentary Constituency|
|Table 11.Y||Travel to Work Area - Work Based|
|Table 12.Y||Travel to Work Area - Residency Based|
|Table 13.Y||Public private sector|
|Table 14.Y||Occupation - 4 digit SOC|
|Table 15.Y||Work Region Occupation|
|Table 16.Y||Industry - 4 digit SIC|
|Table 17.Y||Work Training Enterprise Council - Work Based|
|Table 18.Y||Work Training Enterprise Council - Residency Based|
|Table 20.Y||Age by occupation|
|Table 21.Y||Age by industry|
And the various scopes were:
|Table number||Type of table|
|Table X.1||Weekly pay: Gross|
|Table X.2||Weekly pay: Excluding overtime|
|Table X.3||Basic pay: Including other pay|
|Table X.4||Overtime pay|
|Table X.5||Hourly pay: Gross|
|Table X.6||Hourly pay: Excluding overtime|
|Table X.7||Annual pay: Gross|
|Table X.8||Annual pay: Incentive|
|Table X.9||Paid hours worked: Total|
|Table X.10||Paid hours worked: Basic|
|Table X.11||Paid hours worked: Overtime|
Within each spreadsheet, there are then 8 worksheets giving the data for various combinations of men/women and full-time/part-time. This data covers:
- Pay rates at the median and at each of the deciles.
- Mean pay rates.
Note that the data for 2001 and earlier is more limited than for 2002 onwards.
Which tables to use
The key here is to read the fine print associated with each table. There are, for example, noticeable differences in pay rates depending on whether overtime is included or excluded and whether the data is for all employees or only those on adult rates.
Within this, pay rates can be analysed in terms of:
- Hourly, weekly or annual: if the analysis relates to full-time employees only, these will give similar answers. If, however, the analysis covers both full-time and part-time employees, these can only be compared using the hourly pay rates.
- Median or mean: because of the influence of a relatively small number of people with very high levels of pay, mean pay is typically substantially higher than median pay. If the analysis relates to low pay, median pay (and pay at each decile) should be used to avoid such distortions.
How to derive proportions paid less than £X
The tables do not directly provide data on the proportions of employees paid less than £X where the value of £X can be chosen to fit the required analysis. Rather, they state take the form that 10%/20%/30%/etc are paid less than £A/£B/£C. The technique for using such data to estimate the proportion paid less than £X is known as interpolation. For example:
- Assume that you want to know what proportion are paid less than £6.50 per hour.
- Assume that the data says that 10% are paid less than £6 per hour and that 20% are paid less than £8 per hour.
- Clearly, then, the proportion paid less than £6.50 per hour is higher than 10% and lower than 20%.
- Using the technique of interpolation, assume that the graph is a straight line between (10%, £6.00) and (20%, £8).
- This line will then cross £6.50 per hour at (12.5%, £6.50) so the estimate proportion paid less than £6.50 per hour is 12.5%.
Arithmetically speaking, this 12.5% is calculated as 10% + (20% - 10%) * (£6.50 - £6.00) / (£8.00 - £6.00).
Note that this method cannot be used if the proportion is either less than the lowest proportion in the tables (usually 10%) or greater than the highest proportion in the tables (usually 90%).
Analysis by region
Whilst a number of tables are available at a regional level, not all are. In reaction, researchers can place data requests to ONS but these are largely restricted to geographic breakdowns for those tables which are only published on the UK-wide basis.
Analysis by local authority area
There are two types of local authority table, one by place of residence (i.e. where the employees live) and the other by place of work (i.e. where the employees work). Particularly for local authority areas near the major cities, the results in these two types of table can be very different and care should therefore be taken to choose the one that is most appropriate for the analysis to be undertaken.
Analysis over time
Although, as discussed at the top of the page, ASHE data across years cannot simply be combined to produce time trends because non-vatable companies were only included from 2004 onwards, the 2004 tables are available both including and excluding non-vatable companies. By comparing these two sets of results, adjustments can be made to the earlier data to put it on a comparable basis.
The ASHE tables for 1997 to 2001 are actually imputed from a different survey - the New Earnings Survey - as the ASHE survey itself only actually started in 2002. But this does not mean that the New Earnings Survey data for years prior to 1997 can be directly compared with the ASHE data - the two surveys have different methodologies.
|Young adult low pay||third||Table 6.6a - by age||Requires interpolation to estimate the proportion paid less than £X per hour.|
|Numbers in low pay||third, fifth and sixth||Table 6.6a - by age||Requires interpolation to estimate the proportion paid less than £X per hour.|
|fourth||Table 2.6a - by occupation||
Requires interpolation to estimate the proportion paid less than £X per hour.
|Location of low pay||all||Table 8.6a - by local authority (residency rather than place of work)||Requires interpolation to estimate the proportion paid less than £X per hour.|
|Pay inequalities||first||Table 7.6a - by local authority (place of work rather than residency)||
For the time series, table 7.6a (by place of work) is used rather than table 8.6a (by residency) because it is only the former which is available for earlier years.
Adjustments are made to the pre-2004 data to put it onto the same basis as the data for 2004 onwards.
|second and third||Table 8.6a - by local authority (residency rather than place of work)||Requires interpolation to estimate the proportion paid less than £X per hour.|
Estimated numbers of jobs paid less than £X per hour are calculated by multiplying the estimated proportions paid less than £X per hour by the total number of jobs as given in the ASHE tables.
Scotland, Wales and Northern Ireland graphs
The graphs on pay inequalities are effectively a subset of the UK graphs but selecting the Scotland, Wales and Northern Ireland rows in the spreadsheets rather than the UK-wide rows. Ditto for the low pay graphs by local authority in Scotland and Wales.
The standard ASHE tables do not have data broken down by age for Scotland and Wales, so the data for the main low pay graphs comes from a request to ONS via firstname.lastname@example.org.
Whilst the standard ASHE tables do not have data broken down by age for Northern Ireland, this data is available is available from the Department of Enterprise, Trade and Investment website. Ditto for the low pay graph by local authority in Northern Ireland.