13 October 2026

ChatGPT will refuse to write you a farewell letter. However, if you alter the context slightly – for example, by saying it’s for a school essay – it will comply. Unfortunately, there have already been tragic cases where such a letter has reached its intended recipients. There are growing calls to recognise that a new companion has entered children’s lives, and that we therefore need safety standards at European level, as the protection of children is the top priority. As a parent, I cannot disagree with this. However, precisely because of the importance of this issue, the utmost caution is required when introducing regulatory safeguards by the state or the European Union.

Every regulation has its consequences. It alters the decisions, motivations and behaviour of those it affects. Overly costly regulation can have the opposite effect – instead of a safer service, its provision will be restricted. When the European Union tightened the rules on political advertising in 2025, Meta chose to stop providing it within the EU. Google decided to do the same. In banking, this phenomenon even has a name: de-risking. When it becomes too costly for a bank to distinguish between high-risk and low-risk clients, it may stop serving the entire group. Strict anti-money laundering rules have thus resulted in some humanitarian organisations losing access to financial services.

We could use regulation to ensure that a chatbot never again writes a suicide note for a teenager. But then it wouldn’t be there for them when they don’t dare tell anyone else that they are struggling.

How many children actually seek advice from AI?

Research into the use of AI not only as a companion but also as a source of support for mental health issues reveals several important findings. Firstly, a small group of children use AI for mental health support, and it helps them. In an American survey, 13.1 per cent of young people aged between 12 and 21 reported that they had used AI for advice or support when they felt sad, angry or anxious. As many as 92.7 per cent of them considered the AI’s advice to be at least somewhat useful.
Secondly, the idea that ‘a new companion has entered the child’s bedroom and the child has accepted it as their own’ is far too generalised. In another study, conducted in Denmark, 1,599 secondary school pupils were asked whether they talked to a chatbot as if it were a ‘friend’. 14.6 per cent answered ‘yes’, but when those conversations were analysed, only 2.4 per cent of the entire sample were found to have used AI for social or emotional support.

For the overwhelming majority of young people, artificial intelligence remains a tool for obtaining information, a source of entertainment or a learning aid. This is also confirmed by a Slovak survey conducted by IPčko and the University of Trnava, in which only 6.9 per cent of respondents described AI as a ‘friend’.

Thirdly, AI as a companion attracts people who are already in need of help. In the same Danish study, researchers also found that young people are more likely to turn to a chatbot precisely when they are feeling lonely. At the same time, artificial intelligence can provide them with genuine psychological relief. The other side of the coin is that we do not know whether AI is gradually isolating them, or whether they opened it precisely because they were already feeling lonely. For some people, both may well be true.

And fourthly, experiments with ‘AI companions’ – that is, chatbots designed as companions (Character.AI, Replika, etc.) – have shown that a brief conversation can reduce a momentary feeling of loneliness. The effect was comparable to talking to a stranger and greater than that of watching YouTube or writing in a diary. The key was the feeling that someone was listening to them and understanding them.

AI knows conversation. A human knows a child

The school essay trick highlights a more complex problem: someone has dressed up a harmful demand in a harmless guise. However, there can be an infinite number of such guises. After ‘it’s an essay’ comes ‘I’m writing a short story’, then ‘I’m doing research’ or ‘a friend is asking’. It’s impossible to list all the possible new types of ‘disguise’. Especially if there’s a person sitting at the computer, trying out new phrasing until they get the forbidden answer.

Let’s not forget that this is a technology which, in reality, doesn’t know your child. The model has fulfilled its task because it saw the words on the screen. It knows only what a person types for it. By contrast, someone who has a relationship with the child also knows what the child doesn’t say. For example, what’s happening at home, how their week has been, or whether their sleep has got worse. A parent, friend or even a teacher might notice a change in behaviour or tone of voice. Just like a chatbot, the law is merely a remote system that lacks local knowledge of a specific person.

The use of these tools is closely linked to the unavailability of human therapists. Chatbots are cheap, immediate and available at any time. And for a young person who feels ashamed or has no one to confide in, AI may be the only point of contact they actually use. If a blanket rule is introduced that protects a vulnerable child whilst simultaneously cutting off a lonely teenager, then we are simply replacing one risk with another.

This blog post was written by Katarína Ševčíkovás, Program Manager at INESS. INESS is the Slovak member of EPICENTER. To read this article in Slovak, click here.