Following recommendation 3 of the Report of the Co-Chairs of the Special Panel, the proposed Regulation specifies, harmonises and extends safety by design rules for social networking services, video-sharing platforms services, online games, AI companions and general conversational chatbots accessible to minors. As noted by the Co-Chairs, extending and harmonising these rules ensure legal certainty and a high level of protection for minors across the digital environment ( ). Overall, these rules address inherent risks that certain features and functionalities pose to children and aim at ensuring a safer environment by design and by default.
Safety by design requirements for all services and systems in scope include a clear prohibition on addictive design features as well as requirements to provide minors with options to tune and control information suggested by recommender systems. In addition, providers of online social networking services and of video-sharing platform services that use recommender systems are required to use specific metrics capturing quality, safety and mental-health outcomes to evaluate their recommender systems. Stricter guardrails are also established to lower the risk of minors encountering harmful content or getting stuck in so-called ‘rabbit hole’ effects in which minors are exposed to prolonged chains of harmful content, including related to eating disorders, self-harm or suicide. Rules are set out to ensure that minors are provided with safe settings, including the disabling by default of the visibility of their profile to other users, of push notifications and geolocation, and provide minors with more control to manage these settings. To reduce risks of cyberbullying and grooming, contacts are limited with strangers and minors can easily block unwanted interactions. Additionally, requirements include prohibitions on spending maximisation, transparency and gambling techniques.
Online games must also implement safety by design obligations, prohibiting addictive designs that nudge children into continuing to play the game or returning to the game and requiring safe setting. Additionally, online games should implement safeguards to ensure that minors are not led to initiate contacts on other risky services.
Specifically, and in addition to these requirements, considering the risks posed by AI companions and general conversational chatbots and building on the DSA and the AI Act, such AI systems should fulfil additional obligations to ensure that AI companions and general conversational chatbots are safe for minors. In particular, providers of AI companions and general conversational chatbots are required to put in place strong child-safety protections, including designing systems so minors are not exposed to design features and systems behaviours that are likely to create emotional dependencies, and carrying out testing and post-market monitoring to identify and mitigate harms to minors’ safety, health, fundamental rights and well-being (except for systems provided by a micro or small enterprise) . Where such systems are deployed as a functionality of an online social networking services, of a video-sharing platform services, of an online game, the providers of those services should ensure that AI companions and general conversational chatbots are not automatically activated, that minors have the possibility to opt out of their used, where enabled, and are not encouraged to use them.
Additionally, software application stores are required to put in place an age rating system to establish the age-appropriateness of applications disseminated through their service. The providers of such services should ensure that minors are prevented from purchasing or downloading an app that is inappropriate for their age. In addition, where the provider becomes aware of applications which primarily consist of content that is subject to a minimum age under applicable Union law or national law in compliance with Union law, they shall prevent minors from downloading or purchasing such an application below that minimum age. A code of conduct would help create common methodologies and criteria for such age rating systems, building upon current industry practice. Codes of conduct are also foreseen in regard to online networking services, video-sharing platforms, online games, as well as AI companions and general conversational chatbots.
Under the proposed Regulation, requirements to ensure the agency of children online are also established. In line with Recommendation 2 from Chapter 4 of the Report of the Co-Chairs of the Special Panel, the proposed Regulation also strengthens reporting mechanisms. Such tools must be effective, accessible and child friendly enabling minors to get rapid support and appropriate help upon notifying. Parents are also supported through the easy-to-use, interoperable and age-appropriate parental control tools, thereby implementing Recommendation 7 of Chapter 4. Such tools for example will be mandatory for providers of AI companions and general conversations chatbots, software application stores and online games for children below the age of 13 years. In other words, those providers will be required to ensure that access to their services and systems referred below the age of 13 years is only enabled and controlled by means of the tools for guardians. Rules are established to ensure such tools enable parents to report content on behalf of their child and to manage their account settings as well as time-spent limits. To ensure these tools are effective, the proposed Regulation sets out a requirement for providers to encourage their uptake by parents. Those tools, as well as time limits, must respect children’s evolving capacities and the right to privacy. They are complementary to other safety by design requirements, they should not replace any other measures to ensure the safety of children.
