Can an algorithm hurt? Polish experiences with profiling of the unemployed

By Jędrzej Niklas, March 2017

A CIHR fellow explains how an algorithm is used by Polish job centres to sort the unemployed into three profiles. He analyses how lack of transparency of the algorithm and safeguards against errors of the system creates tension with the human and social rights of the unemployed.

This article is based on a report Profiling the Unemployed in Poland: Social And Political Implications Of Algorithmic Decision Making conducted by the Panoptykon Foundation in Poland. 

Job centres in Poland use automated data processing and algorithmic decision-making to put the unemployed in three categories. Each job centre client gets an automatically assigned profile which determines what type of assistance the can get – a job placement, vocational training, apprenticeship or activation allowance.

The profiling of the assistance for the unemployed was first introduced in 2014, when the Ministry of Labour and Social Policy reformed the system of more than 340 job centres in Poland. Introduced in times marked by economic slowdown after the global financial crisis, the purported goal of the reform was to improve job centres efficiency. According to the government, this goal would be obtained by providing better and more personalised services to citizens.

Since it was first introduced as a proposal, this profiling mechanism has been criticised by Polish human rights institutions and social organisation. Experts have warned that this tool may violate the right to protection of personal data, social rights and can lead to discrimination. Two years later, even the Polish government admits that there are problems with the system.

How does profiling of the unemployed work?

All unemployed are divided into three profiles. Personal data is collected from the unemployed person at the job centre to determine which profile best fits them. The answers she gives, will have a fundamental importance for her fate – an algorithm will decide which type of unemployed assistance she will get.

The data is collected through a survey and an interview – a total of 24 data points. Eight data points are collected during the registration at the job centre – for example, age, gender, disability, knowledge of foreign languages or duration of unemployment. Another 15 data points are gathered during the computer-based interview. The questions are constructed in the way that suggests that they are open-ended in character. But in the reality the scope of the answers is closed. For example, the question “What is the main obstacle for returning to the job market” has 22 predetermined answers.

Surprisingly, being homeless, of different ethnic origin or convicted felon – all of which constitute a real obstacle to getting a job in Poland – are not included in these 22 options. Therefore, if the unemployed provides such an answer it will not be taken into account.

Based on the final score an algorithm decided which category should be given to the unemployed. The final calculation determines the scope of assistance which person can apply for. According to statistics profile I covers 2% of the unemployed in Poland, profile II – 65%, and profile III – 33%.

The first profile covers mainly the active and mobile individuals, who have appropriate professional qualifications and interpersonal skills. It is considered that they do not have any serious life problems that would make it impossible for them to find a job. Unemployed from this profile are offered 13 types of assistance – for example, funds to open a business, vouchers for trainings, refunds of costs of transportation or job search services.

On the other hand, profile II typically includes persons who have certain professional skills, but unfortunately are redundant on the labour market or worked for a very long time in only one company. They are seen as lacking ideas on how to solve their problems or self-presentation skills. 29 forms of assistance are reserved for profile II, like job placement or activation allowance.

Profile III comprises of individuals with serious life problems or those who do not want to cooperate with the employment office. In theory, they do qualify for one of 10 types of forms of assistance and special programmes. In practice, theses forms of support are too costly and difficult to organise, so job centres often simply do not offer them. According to the statistics 38% of labour offices cannot offer any kind of support to the persons assigned to the third profile.

Lack of Transparency

The process of profiling plays crucial role in shaping the situation of the unemployed. However, the logic behind the profiling and the algorithm itself are considered confidential. As a result, the unemployed person doesn’t know how certain individual features or life circumstances affect her chance of being assigned to a given category. What an unemployed person knows are only broadly describe categories of collected data (e.g. gender, disability) not the scoring rules. The rules used by the computer system and its operation is described only in the internal guidelines of the Ministry of Labour.

Following an intervention from the Panoptykon Foundation, the Ministry of Labour published the list of questions, which the unemployed are asked during the interview. An administrative court in Warsaw ruled that scoring rules are public information. This mean that the Ministry of Labour should either publish them or give justified circumstances to keep them in secret.

Lack of safeguards 

The profiling mechanism is a almost always fully automated – it is a computer that decides what type of assistance a person should get and not the job centre employee. The person entering the data has very little freedom in how the data is entered – for example when the job centre employee thinks that in a particular case the person should be categorized to a different profile, he or she can change the decision generated by the computer system, according to their own observations. Such an opportunity, however, is not grounded in legal regulations.  What is worth mentioning, this option is treated as exceptional and it is hardly ever used in practice – according to official statistics, only in 0.58% of all cases.

The unemployed person has a right only to ask for another determination of the profile. However, this may be done only if her life situation after the interview has changed. The law does not give a possibility to demand that the profile be changed or re-verified if the unemployed person herself thinks that she should be qualified to another profile or that an error was made in the course of its determination.

Discrimination and social sorting

As practice shows, a significant number of local job centres do not provide any assistance to the unemployed categorised in profile III. Accordingly, the very people who need job centre services the most are deprived of it. The whole system is designed in a way that instead of helping – at least in some situations – it blocks access to public services.

This may affect persons belonging to vulnerable groups. Determination of the profile is determined on the basis of such features as age, gender or disability. For example, women receive one point more than men because of their gender. There is an assumption that women still have worse chances to get job than men – so they need more help. But this one point more can bring them closer to the profile III, and in fact negatively affect access to any assistance form job centre.

At the same time profiling becomes a tool that not only ‘helps’ to divide type of assistance, but also to categorize people. Both the unemployed and the job centre staff developed an understanding of that profile III is designation for people in difficult situations who often cannot be helped (‘junk people’). This is party due to objective reasons, as job centres in Poland do not have enough public funding to deal with such difficult situations. However, it can also be argued that the very categorisation of people as a method of social sorting generates the perception that some people are better or worse than others. Because of the negative perception of Profile III and objective factors making this profile less attractive, it became a common practice among the unemployed – as well as among the frontline staff who were trying to help them – to manipulate the process of profiling in order to avoid being categorised as “III”.

Social rights and rule of law

Profiling and the lack of transparency have a significant impact on the practical enforcement of social rights of unemployed. Polish constitution and international agreements, like the European Social Charter, establish that every human has a right to work, which includes assistance during the period of unemployment. These standards also show that the criteria for granting, receiving or denying access to this assistance should be transparent and specified in legal acts. Previously, the rules and criteria for granting welfare aid were always codified in law. For example, only people of certain age or in specific life situation could request a specific type of support. Profiling changed this model in a radical way. Now it is no longer a piece of legislation, but the rules of IT system – and their creators – that decide who will receive assistance and who will not.

Mark Bovens and Stavros Zouridis show that this kind of automatic decision-making brings a fundamental change for the rule of law principle. From this perspective, the IT systems are much more difficult to understand by citizens, as well as the democratically elected authorities. Unfortunately, our democratic model has not yet developed good mechanism and procedures that allow the assessment new technologies from the perspective of the human rights. The protection of personal data might be an exception, but it has a limited impact as it protects only the data itself and not the effects of its processing.

Bovens and Zourdis postulate that such systems should be inspected by special committees or independent bodies. They also believe that citizens should receive full information about the system and the public should be given an insight to the “decision tree” used in these mechanism. Additionally, Danielle Citron and Frank Pasquale argue that publicly-used algorithms should be evaluated in case of discrimination risks. Assessment systems, like the one used for profiling of the unemployed in Poland, would only be able to operate once evaluated an independent auditors or appropriate institutions.

Next steps

The debate over the profiling of the unemployed in Poland continues. The criticism of the system was raised already as the proposal was first introduced as a bill – by the national Data Protection Authority, the Ombudsman, social organizations and trade unions. Now, even the Polish government admits that there are problems. The state secretary from the Ministry of Family, Labour and Social Policy announced plans to change the mechanism. However, no draft proposal was presented to the public yet.

Meanwhile the Ombudsman referred the case of profiling to the Constitutional Court. In his application, he focused on two issues. One is lack of possibility to change a given profile. According to the Ombudsman, this situation is a violation of the right to an effective remedy. Second issue is the formal basis for processing of personal data during the profiling process. The Ombudsman state that they should be regulated by the legal act adopted by the parliament and not by the regulation issued by the ministry. The case awaits consideration by the Constitutional Court.