in , , , , , , , , , , , , , ,

Explanations of news personalisation across countries and media types

Written by Mariella Bastian, Mykola Makhortykh, Jaron Harambam, Max van Drunen


News outlets worldwide increasingly adopt user- and system-driven personalisation to individualise their news delivery. Yet, the technical implementation of news personalisation systems, in particular the one relying on algorithmic news recommenders (ANRs) and tailoring individual news suggestions with the help of user data, often remains opaque. In our article, we examine how news personalisation is used by quality and popular media in three countries with different media accountability infrastructures – Brazil, the Netherlands, and Russia – and investigate how information about personalisation usage is communicated to the news readers via privacy policies. Our findings point out that news personalisation systems are predominantly treated as black boxes that indicate a significant gap between practice and theory of algorithmic transparency, in particular in the non-EU context.

Citation & publishing information

Received: August 6, 2019 Reviewed: March 5, 2020 Published: October 13, 2020
Licence: Creative Commons Attribution 3.0 Germany
Funding: This work was supported, in part, by the European Research Council (grant 638514) and Netherlands Organisation for Scientific Research (grant 400.17.605).
Competing interests: The author has declared that no competing interests exist that have influenced the text.
Keywords: News personalisation, Privacy policies, Transparency, Accountability, Comparative journalism research
Bastian, M. & Makhortykh, M. & Harambam, J. & van Drunen, M. (2020). Explanations of news personalisation across countries and media types. Internet Policy Review, 9(4). DOI: 10.14763/2020.4.1504


Today, newsreaders worldwide increasingly consume news online, and this leads to profound changes in how the traditional media produces and disseminates news content (Mitchelstein & Boczkowski, 2010). The shift to digital distribution, together with the growing availability of data about audiences (including individual reading habits), enables new possibilities for making content selection and delivery more individualised for each reader. The options for personalised news distribution are many; they vary from customisable subscriptions to specific topics/authors (user-based personalisation) or individually tailored news suggestions generated via algorithmic news recommenders (ANRs) (system-driven personalisation). Unlike user-based personalisation, which is grounded on explicit user decisions (e.g., choosing a specific subscription mode or following a certain topic), system-driven personalisation relies on implicit data about user activity (e.g., what news stories are read or how much time is spent on a specific page) that are utilised to suggest content that the system views as interesting or relevant to the user.

The use of news personalisation, in particular in its system-driven form, is viewed as an important strategy for news outlets (Newman, 2018). The individualised news distribution allows news organisations to be responsive to consumers’ information needs while ensuring traffic, consumption, and revenue through user targeting and profiling (Karimi et al., 2018). However, the technical implementation of news personalisation systems, especially the ones relying on ANRs that automatically draw insights from personal user data to generate individually tailored content recommendations, often remains obscure to news consumers and practitioners alike (Diakopoulos & Koliska, 2017).

With regard to personalised systems of news delivery, this lack of transparency is not a trivial issue, considering the possible impact of news personalisation on individual consumers and society at large. Some scholars (Penney, 2017; Stoycheff, 2016) argue that personalisation can increase anxiety (or “chilling effects”) among news consumers who feel that their online behaviour is being monitored. Personalisation has also been argued to threaten the democratic role of the media by impacting the provision of non-discriminatory access to information and the readers’ right to receive information (Eskens et al., 2017; Zuiderveen et al., 2016). Filter bubbles (Pariser, 2011), which limit the variety of information that individuals receive from the mass media and can potentially polarise society, have been especially prominent in this discussion. Although empirical research (Dubois & Blank, 2018; Möller et al., 2016, 2018) has so far found little evidence supporting the existence of filter bubbles, the possibility that personalised information distribution can lead to information inequalities or amplify existing biases and thus undermine the integral functions of the media in democratic societies cannot be excluded (Helberger et al., 2019). Understanding the workings of personalisation systems – in particular the ones based on ANRs – is therefore of crucial importance, and the news media is responsible for explicating whether and how it uses personalisation technologies. This is, however, easier said than done in the case of system-driven personalisation, as explaining the workings of algorithmic systems is notoriously difficult (Pasquale, 2015).

In this paper, we ask how these challenges are tackled by contemporary mass media by exploring how the presentation of news personalisation varies between the digital outlets of the quality (or broadsheet) and popular (or tabloid) press in different countries. Similar to Hanusch (2013), we define the difference between the two types in terms of “content and form rather than publication size” (p. 499). The tabloid press tends to produce more sensationalist – or even scandalous – content, which often promotes iconoclastic views (Bastos, 2016), and combines this with emotional appeals (Örnebring & Jönsson, 2004). While doing so, it often aims to entertain, rather than inform or educate, the audience; this leads to a decrease in journalistic standards, which is a common source of criticism of the tabloid press (Chadwick et al., 2018; Esser, 1999). In contrast, the broadsheet press relies on more in-depth reporting and prioritises “hard news coverage, fact-checking, and research based on a timeline in which the story unfolds” (Bastos, 2016, p. 218); this is viewed as an integral condition for fulfilling the democratic functions of the media (Berry, 2009).

A number of studies have investigated the differences in the adoption of digital innovations by quality and popular media (see, for instance, Jönsson & Örnebring, 2011; Karlsson & Clerwall, 2012; Karlsson & Clerwall, 2013); however, to our knowledge, none has looked at the differences in using and communicating the use of personalisation between broadsheets and tabloids, especially from a comparative cross-country perspective. While the question of whether the type of outlet (i.e., popular or quality) influences the adoption of technological innovation remains an open one, we agree with Jönsson and Örnebring (2011), who argue that because of their more popular (or sometimes populist) nature, tabloids seem to be more inclined to adopt new technologies, including the ones related to the ways in which the readers interact with the content. Similarly, despite the current lack of comparative research on the adoption of personalisation systems across different countries, the existing studies on media innovation and its adoption suggest that these processes develop differently for specific media systems and media markets (Hanusch et al., 2019).

The above-mentioned factors motivated our decision to use a comparative approach to investigate how personalisation systems are adopted by different types of media outlets in different media systems. Using a sample of 12 newspapers from Brazil, the Netherlands, and Russia, we qualitatively examine which personalisation strategies are used and how they are communicated to their audiences. While doing so, we look at both user-driven and system-driven personalisation from the point of view of the user to determine whether the popular media outlets present the use of personalisation differently from the quality ones. As part of this comparison, we also look at the differences in communicating the use of personalisation between outlets coming from different media systems to investigate the degree to which such communication can be influenced by different contextual factors, such as the countries’ media accountability infrastructures.

To answer our main research question – that is, how the use of news personalisation is communicated by quality and popular media in different media systems – we start by identifying the news personalisation practices that are used by the news media. First, we examine the front-end features of their digital outlets to determine which (if any) forms of personalisation can be observed from a user perspective. Second, we explore how these outlets communicate their personalisation practices to users: more specifically, we focus on the presentation of personalisation through formal privacy policy documents. While doing so, we investigate how transparent and intelligible these communication procedures are and whether there are meaningful differences between outlets, depending on their types and the cultural contexts in which they operate.

Theoretical background

In recent decades, the news landscape has changed drastically due to the rise of digital technologies (Macnamara, 2010; Meyer, 2009; Van der Haak et al., 2012). A concomitant aspect of this computational turn in journalism (Coddington, 2015) is the possibility for news organisations to track what news people consume and how. Many newsrooms today use such audience metrics to inform various kinds of editorial decision-making and adapt their products if necessary (Anderson, 2011; Lee & Tandoc, 2017; Petre, 2015), but they are also used as input data for personalising news delivery. These personalisation technologies enable news organisations to customise their contents to the (assumed) interests of readers, and they are seen as one of the most promising innovations in the news industry (Newman, 2018).

The use of news personalisation, in particular system-driven personalisation based on ANRs, raises a variety of societal and academic concerns. The first is privacy: news organisations can now actively track people’s online reading behaviour, but what do they actually monitor, and for what purposes? ANRs need user data to do their work, so there is always a trade-off between privacy and personalisation (Li & Unger, 2012); however, it is often unclear what data news organisation collect to personalise news delivery and whether they “enrich” these with user data collected by third parties or even share these data with them. This form of surveillance, even if deployed for arguably benign goals, may cause so-called chilling effects – a form of self-censorship in the face of coercive threats (Penney, 2017; Stoycheff, 2016).

While the collection of personal data for personalising news content delivery does not necessarily trigger the same concerns for the users as government surveillance does, there is a possibility that corporate profiling also leads to the same effects, although the current lack of empirical research does not allow this assumption to be proved or disproved (Büchi et al., 2019). However, considering that personalised commercial offers are known to increase privacy concerns as participants become aware of their data being collected and used (Aguirre et al., 2016), we suggest that the deployment of personalisation by news media can result in the chilling effects caused by the newsreaders’ awareness that the outlet knows or is even predicting their information preferences. People may think twice about what they read, which limits their right to information (Balkin, 2009; Eskens et al., 2017). Moreover, what rights people have regarding the protection of their own personal data is in many contexts rather unclear. While the adoption of the General Data Protection Regulation (GDPR) in 2018 provided the European Union (EU) with a well-defined legal framework that sets a high bar for (algorithmic) transparency, the way in which its open norms apply to specific contexts, such as news personalisation, is far from straightforward and can even differ between EU member states (Erdos, 2016; Eskens, 2019). It therefore remains to be seen what obligations, such as the GDPR’s requirement to “provide meaningful information about the logic involved [in profiling]” (art. 13–15), mean for online newsreaders.

A second concern relates to the role of the media in democratic societies and the fear of increasing societal polarisation. The news media plays an important role here as it (should) create(s) collective realities and form(s) arenas for public debate, wherein a variety of sources, voices, and perspectives can be discussed (Hampton, 2010; Muhlmann, 2010; Starr, 2005). The diversity of media outlets and their contents is seen as a key requirement for performing this democratic role (Hardy, 2014; Helberger, 2011; Karppinen, 2013). It has been argued that recommender systems threaten the media’s role in democracy, as they are assumed to focus on satisfying user information preferences based on previous histories of interaction and can thereby isolate users from alternative opinions and new topics by creating echo chambers (Sunstein, 2017) and filter bubbles (Pariser, 2011). However, recent studies (Bruns, 2019; Dubois & Blank, 2018; Möller et al., 2018) question whether these effects exist, at least with regard to the general public, and point out that news personalisation can also promote diversity (Möller et al., 2016) and help the media realise its societal functions in democratic societies (Helberger, 2019).

Furthermore, research suggests that rather than valuing systems of personalised delivery that give readers more of the same content, there is an audience demand for information diversity (Bodó et al., 2019). The practical implementation of diversity through software design in the context of news personalisation is, however, a complicated task, as it can be operationalised in different ways, ranging from autonomy-focused perspectives, which aim to suggest content that allows readers to realise their own interests, to adversarial perspectives, which value suggestions that challenge users’ existing beliefs (Helberger et al., 2018). This complexity raises multiple questions, such as the following: What responsibility do media organisations take for the diversity of their personalisation systems? What kind of individual and collective effects or diversity do they speak about or promote? And do they speak about their individual and collective effects, or do they promote more diversity-centred ANR?

Third, the opacity of the algorithmic systems that undergird news personalisation, in particular its system-driven forms, is a concern shaping public and academic debates about individualised content delivery (Burrel, 2016; Pasquale, 2015; Stohl et al., 2016). Transparency has played a key role in the operationalisation of the media’s accountability to its audiences and the general public (McBride & Rosenstiel, 2013). However, newsreaders generally have little understanding of how personalisation systems work, why certain news is recommended to them, or how to intervene when needed (Bucher, 2017; Eslami et al., 2015; Fletcher & Nielsen, 2019). This lack of transparency is a wider societal problem: it prevents civil society from learning more about algorithmic systems and holding the organisations that implement them to account (van Dijck et al., 2018). It thus allows algorithms to distribute resources across populations with little public accountability (Diakopoulos, 2016) and fuels urgent calls for their regulation (Pasquale, 2015; Ziewitz, 2016).

While algorithmic systems are notoriously difficult to understand (Gillespie, 2014; Kitchin, 2017), there are efforts to make their implementation in the news sector more transparent to enable media accountability (Diakopoulos & Koliska, 2017; Hindman, 2017) or to increase the control options so that the newsreaders can use these systems more effectively (Harambam et al., 2018). It is difficult to predict whether these changes will make it to the market, but there is certainly a desire and need for more explanations amongst newsreaders (Harambam et al., 2019; ter Hoeve et al., 2017). In addition to facilitating action by individual newsreaders, transparency rights for users or the public can be used to power collective action and reduce power asymmetries (Ausloos & Dewitte, 2018). Several civil society groups (e.g., Algorithm Watch, Panoptykon Foundation, and Tactical Tech) focus on using algorithmic transparency as a means to enforce media accountability.

These concerns – privacy, diversity, and transparency – will guide us in the analysis of how media organisations communicate the use of news personalisation. In doing so, we challenge the tendency to take a Western-centric perspective on these concerns and call for a more contextualised view of them. James Whitman (2003) already noted a difference between some Western countries with regard to the meaning of privacy (specifically by looking at differences between the United States and Europe), and the urgent need for the de-Westernisation of journalism and communication research has been highlighted by numerous scholars (Hardy, 2008; McQuail, 2000; Wang, 2011). Empirical evidence has also shown significant differences in journalism cultures and media systems worldwide (Brüggemann et al., 2014; Hallin & Mancini, 2004; Hanitzsch et al., 2011; Willnat et al., 2013) and more specifically in terms of media accountability (Bastian, 2019; Fengler et al., 2014).

The latter concept of media accountability serves as the starting point of our study and is defined as “any non-State means of making media responsible towards the public” (Bertrand, 2000, p. 108). We emphasise the importance of accountability because it encompasses not only the journalistic work of media companies but also their ethics and societal role. The differences in media accountability infrastructures that exist at the local level (Bastian, 2019) contrast with privacy discussions that emphasise the potential universalising impact of European data protection law on other countries (i.e., the so-called “Brussels effect”; Bradford, 2012). Not only does the GDPR apply directly to non-EU organisations that monitor the behaviour of EU residents, but companies can also voluntarily apply European data protection law globally to avoid the technical, economic, and reputational difficulties of maintaining different privacy policies. At the same time, European data protection law can also influence non-EU legal systems indirectly by functioning as a model from which other countries can draw inspiration when creating their own national data protection legislation (Azzi, 2018; Bradford, 2020). To go beyond the usual focus on Western media systems, we selected three countries with different types of media accountability infrastructures: Russia, the Netherlands, and Brazil. We chose these countries to identify similarities and differences in the ways in which different media systems deal with privacy and transparency issues regarding news personalisation. In the following sections, we will shed more light on these issues – first by discussing our methodology and then by detailing our findings.


Sampling and data collection

To conduct this qualitative study, we selected a sample of 12 news outlets (six quality newspapers and six popular newspapers) from three countries: Brazil, the Netherlands, and Russia. Below, we will discuss the rationale for our choices while we made the selection – in particular our choice of the countries and specific outlets to compare.

Country selection

Our choice of countries was based on the assumption that local news industries are characterised by profound distinctions originating from a diverse set of culturally negotiated journalistic values and attitudes towards transparency, audience participation, and self-regulation, thus impacting the media accountability landscape. This assumption is based on comparative studies using concepts such as “journalism cultures” (Hanitzsch, 2011) or “media systems” (Hallin & Mancini, 2004) to trace the differences between journalistic organisations and routines at the local level. These differences are also known to influence the process of adopting media innovations as shown by multiple studies on media innovation and its adoption (see, for instance, Hanusch et al., 2019; Humprecht & Esser, 2018; Lehtisaari et al., 2018; Nozal Cantarero et al., 2020; Toepfl & Litvinenko, 2018) that suggest that these processes develop differently for specific media systems and media markets.

As media accountability is influenced by the relationship between the media on the one hand and the government and political sphere on the other (Bastian, 2019), the role of the state in the media sector already reveals important information about the media accountability infrastructure. Whereas in the Netherlands, both sectors are comparatively disconnected (Groenhart & Evers, 2017), the Brazilian media landscape is closely connected to the political sphere, depending on the outlet and the respective political actor in terms of either proximity or harsh opposition (Lima, 2011; Matos, 2009). Even more noticeably, the Russian state has an extensive influence on the media system, amongst others through the strong paternalistic tradition and intense instrumentalisation of the mass media by the ruling elites to secure political gains both domestically and internationally (see, for instance, Akhrarkhodjaeva, 2017; Rotaru, 2018; Vartanova, 2012).

Influenced by this role of the state and other characteristics of the respective media systems, media accountability infrastructures differ significantly: “the hybrid statist-commercialised nature of the Russian media system […] influences not only authentic journalism culture but accountability practices as well” (Vartanova & Lukina, 2017, p. 223). More concretely, transparency plays a role in the Russian case through the publication of self-regulatory documents and the trend to improve “the quality of public dialogue with users […] and [the] transparency of journalism subjects” (Vartanova & Lukina, p. 223).

In contrast to the Russian scenario, the Dutch one shows a different distribution of responsibility and influence among actors. Here, an increasing awareness of media accountability and transparency can be observed among the public, the political sector, and the media sector. Because of the lack of both governmental interference and active efforts by the public to hold journalists accountable, the media sector is characterised by higher media accountability (Groenhart & Evers, 2017).

Interestingly, privacy plays an important role in the Dutch media accountability sector because in addition to professional media guidelines on how to cover the political far right, its guidelines focus “mostly on [the] privacy protection of suspects and criminals” (Groenhart & Evers, 2017, p. 174). Transparency is a valued mechanism in the Dutch context, as evidenced by a variety of policy papers published by Dutch news organisations. Increasingly finding new ways to get in touch with the audience is a further characteristic of the Dutch media accountability landscape, though it is applied to differing degrees by different media organisations (Groenhart & Evers, 2017). Furthermore, the Netherlands is the only country in our sample where the GDPR requirements for transparency and accountability are immediately applicable.

These sharp differences between media organisations are visible in the Brazilian media landscape as well. Although a general trend towards more transparency can be observed in Brazil too, outstanding best-practice examples exist alongside very opaque organisational practices (Bastian, 2019). The relationship to the public holds a special role in the Brazilian context because of several reasons: first, the country maintains a tradition of social movements and public demands for the democratisation of the public sphere and the communication sector, and second, neither the media organisations themselves nor the content they distribute adequately represent the rich (cultural and geographical) diversity of the country (Bastian, 2019). This intense relationship between all three parties – the media, the public, and the political sector – and the respective differences between the Brazilian, Russian, and Dutch media accountability infrastructures give reason to expect these differences to be reflected in the documents that are analysed in this study.

Media selection

In addition to the comparison of the different ways in which local news outlets communicate their algorithmic practices, we investigated the difference between quality (broadsheet) and popular (tabloid) news outlets. Our decision to introduce this criterion of sampling is based on the alleged difference between quality and popular media in terms of adopting new technological solutions (Jönsson & Örnebring, 2011; Karlsson & Clerwall, 2013). More specifically, it relies on the assumption that popular media may be more open to innovations that are designed to entertain or better target their users (e.g., by attracting more clicks; Karlsson & Clerwall, 2013). Similarly, we expect quality newspapers to be more responsive to societal concerns about privacy and the use of these new technologies.

Table 1: Sample of news outlets (by country and type)
Outlet type Brazil Netherlands Russia
Popular Super Notícia, Extra De Telegraaf, Algemeen Dagblad Mosskovskii Komsomolets, Argumenty i Fakty
Quality O Globo, Folha de S. Paulo NRC Handelsblad, Het Financieele Dagblad Rossiiskaia Gazeta, Izvestiia

Based on these two criteria, we selected two quality and two popular outlets for each of the three countries, as shown in Table 1. As a primary selection criterion, we used the audience size based on the publicly available estimations from the end of 2017 to the beginning of 2018 (see Mediascope [2018] rankings for Russia, rankings by Grupo de Mídia São Paulo [2018] for Brazil, and SVDJ data [Bakker 2018] for the Netherlands). The secondary criterion was related to the use of personalisation by the respective outlet. The personalisation could be either a user-driven one (e.g., the possibility of subscribing to a certain author or topic) or a system-driven one (e.g., the individualised selection of stories in the “Recommended for you” section); if at least one of the types of personalisation was used, we included the outlet in our sample. The assessment of the presence/absence of personalisation was made via a close examination of the digital versions of the respective outlets and the testing of different options for disseminating the content. Because we were particularly interested in what is visible to the newsreader by default (i.e., without a substantial commitment to the news organisation, which would, for instance, involve buying a subscription), our examination of the use of personalisation focused on the front-end features that are accessible without going beyond a (possible) paywall.

For Russia, we chose the following four press outlets: Mosskovskii Komsomolets, Argumenty i Fakty, Rossiiskaia Gazeta, and Izvestiia. All four outlets are federal-level and predominantly pro-government newspapers; they are also daily newspapers, with the exception of Argumenty i Fakty, which is weekly. With a daily circulation of 606,000 copies, Rossiiskaia Gazeta is the youngest of the four newspapers (it was founded in 1990) and serves as the official outlet of the government of the Russian Federation. Izvestiia (1917) was the official outlet of the Soviet government but was privatised following the dissolution of the Soviet Union and currently has a daily circulation of 322,900 copies. Mosskovskii Komsomolets (1919; current circulation of 513,200 copies a day) and Argumenty i Fakty (1978; current circulation of 4,572,700 copies per week) are two popular outlets with a strong focus on entertainment content. Unlike the quality outlets mentioned above, both Mosskovskii Komsomolets and Argumenty i Fakty have multiple regional editions.

Officially, only Rossiiskaia Gazeta is state-owned; the other three newspapers are commercial enterprises. However, in all three cases, there is a strong relationship between commercial owners and the Russian state that reflects a tendency for political parallelism in the Russian media system (Vartanova, 2012). All four newspapers have digital versions, which are available free of charge without any additional paywalls. The subscriptions for all four outlets (around 16 euros per month on average; the most expensive is Izvestiia at approximately 30 euros per month for the subscription) provide physical copies of the respective newspaper that are delivered by post.

In the case of Brazil, we selected the following newspapers: Super Notícia, Extra, O Globo, and Folha de S. Paulo. Both popular newspapers – Super Notícia and Extra – are relatively recent outlets that were founded in 2002 and 1998, respectively. With their low purchase price, their target group is very broad; the newspapers aim predominantly at the poor and working class. According to Grupo de Mídia São Paulo (2018), Super Notícia has a daily circulation of 219,200 copies, whereas Extra has a circulation of 116,500. In contrast, founded in 1921, Folha de S. Paulo is one of the oldest newspapers in the sample and is the one with the largest daily circulation (300,500 copies). Its competitor O Globo has a circulation of 240,900 copies. Their primary target group is well-educated Brazilians. Brazilian media organisations have often been active in political developments; the most explicit example is probably O Globo, which is criticised for the fact that it supported and benefited from the military regime.

For the Netherlands, we chose De Telegraaf and Algemeen Dagblad, which represent the popular press, and the quality newspapers NRC Handelsblad and Het Financieele Dagblad. With a circulation of more than 350,000 copies, De Telegraaf, which was founded in 1893, is the largest Dutch daily newspaper. Algemeen Dagblad, which is a more recent daily newspaper, was founded in 1946 after the Second World War and has a circulation of approximately 300,000 copies. Unlike De Telegraaf, which has a single nationwide version, Algemeen Dagblad has multiple regional versions (similar to the Russian popular outlets we selected). NRC Handelsblad (1970) and Het Financieele Dagblad (1943) have smaller circulations than their popular competitors and produce around 80,000 and 50,000 copies a day, respectively.

Unlike Russian outlets, whose digital newspaper versions were not paywalled, Dutch outlets usually required a subscription in order to access the full content. The price varied substantially between quality outlets (from 26 to 42 euros per month) and popular outlets (between 4 and 12 euros per month). For the popular outlets, the subscription included access to the so-called “premium” articles, whereas for quality media, it served as a means of getting behind the paywall, appearing after the reader viewed a few freely available articles.

In summary, the selection criteria of this explorative study are based on the notion of diversity. We have included different media systems and different types of newspapers to explore the width and depth of how newspapers engage with societal concerns regarding personalisation. Therefore, the focus of this study lies in finding similarities and differences between quality and popular media in different countries. By selecting the most popular newspapers that enable (any form of) personalisation, we ensured that we focus on the cases which are important for the respective media systems. Because this is an explorative qualitative study, its aim is not to find law-like generalisations across time and place, but to advance the current understanding of the ways in which the use of ANRs is communicated by news organisations to their readers.

Data analysis

After establishing a sample of news outlets for our study in July–August 2018, we proceeded to the analysis of front-end communication practices related to the use of algorithmic recommendation. Specifically, we focused on privacy policies both because of their importance (i.e., as binding legal agreements between content providers and users) and because of their explainability potential (i.e., being a major source of available information for users about the ways in which their data are processed and collected) (Wilson et al., 2016).

We divided our analysis into two sections. In the first section, we summarised the front-end personalisation features used in our sample. We started by visiting the site (from an internet protocol [IP] address in the EU) and examining the type of personalisation that is visible to the users (user- or system-driven), and then we looked at the accessibility of the privacy policies (how easy/difficult it is to locate them on the websites of the respective organisations). Finally, we examined the title of the website section discussing the matters of personalisation and privacy to check for possible differences between the country- or type-based categories of outlets. All of these steps were conducted using a desktop device, so the findings below are applicable to a rather specific scenario – that is, the users engaging with the news outlet via its native website and accessing it via the desktop browser. We also took into account sponsored – that is, third-party – news materials (e.g., news from partner organisations) that were found on the website and included these in our analysis if such materials were disseminated with the help of user- or system-driven personalisation.

In the second section of our analysis, we focused on the communication of personalisation in the privacy policies of news organisations. For this purpose, we used a document analysis approach to find out in more detail how newspapers communicate their personalisation practices (Bowen, 2009). Our analysis focused on seven theoretically informed characteristics of privacy policies that feature prominently in the studies dealing with data protection and privacy (see, for instance, Ausloos & Dewitte, 2018; Kuner, 2005; Kuner et al., 2016; Organisation for Economic Co-operation and Development, 2013). These characteristics are as follows: a) the mentioning of personalisation; b) the complexity and type of language (formal/colloquial) used; c) the kind of data collected; d) the purposes of the data collection; e) data storage and sharing; f) data processing; and, finally, g) data subject rights.

Both steps of the analysis were conducted by three of the authors of this paper in September 2018. To implement the analysis, the authors used Google spreadsheets. The contents of the privacy policy documents were coded according to the set of questions (e.g., Is personalisation visible on the website? What type of personalisation is mentioned in the privacy policy?) concerning the visibility of front-end personalisation features (step 1) and the seven characteristics of privacy policies listed above (step 2). The results were then discussed in a series of personal meetings. Because the analysis required knowledge of Dutch, Portuguese and Russian, each author was conducting analysis for the specific country in the language in which he or she was most proficient.

To minimise potential researcher bias, we used concise and simple questions in the case of front-end personalisation features (i.e., to decrease the possibility of disambiguation and disagreement among the coders) and discussed the coded parts of the privacy policies among the coders. Despite this, we cannot fully exclude the possibility of some of our interpretations being influenced by translation bias caused by the stronger or weaker emphasis placed on certain aspects of personalisation communication by coders working with a particular language. To address this, the coding results were checked by other authors (in the cases in which their language knowledge allowed for it – i.e., Brazilian and Dutch), and disagreements were discussed among the authors. A similar procedure was used in the cases in which the main coder had doubts concerning the attribution of a particular policy aspect to one of the above-mentioned characteristics. In these cases, the section dealing with the respective aspect was translated by the coder and discussed with the other two authors to consensus-code it. While other sources of bias can also influence our findings (e.g., the fact that all the coders are professional scholars could imply a more positive attitude towards quality media compared with popular ones), we assume that the use of clear criteria to identify the presence or absence of specific features related to personalisation communication should limit the possible effect of such biases.

Front-end personalisation

Personalisation visibility

We started our analysis by looking into front-end personalisation features. Our observations suggest the absence of substantial differences in the type of personalisation used by quality and popular media in all three countries. In almost half of the cases (five out of 12), we detected a combination of both types. System-driven personalisation is more commonly used by the Brazilian and Russian outlets, whereas the Dutch ones focus more on user-driven personalisation. Furthermore, we found that quality media tends to offer user-driven personalisation services more often. While this observation can hardly be viewed as generalisable considering the size of our sample, it raises the question of whether the provision of more control to the users via user-driven personalisation can be part of quality news services.

Our analysis indicates that for the majority of outlets, system-driven personalisation features are presented through subsections on the front page and sub-pages. The titles of these subsections vary: in addition to “Recommended for you”, we found “Read also”, “What else to read”, “Partner stories” and “World is close”. The format of the personalised subsections is rather similar: each of these usually includes three to five links to the news stories, and they are often accompanied with story-related images.

User-driven personalisation is mainly offered through mail-based updates. Not all of these updates are personalised per se; this is especially the case with editorial newsletters that include the major news events of the day (Argumenty i Fakty). Other options are more individualised, such as in the case of individual subscriptions for specific stories (Rossiiskaia Gazeta). One particularly interesting example is the user-driven service MyNews, which is offered by the Dutch quality outlet (Financieele Dagblad). After free registration, it allows the readers to compose their own personalised news feeds by choosing which topics to follow. The list of topics varies from “Media” and “Stocks” to “Donald Trump” and “Technology”. Furthermore, the service provides an option to label certain articles as “Favourites” and to see one’s “Recently read articles”.

Accessibility of privacy policies

Following the examination of the types of personalisation that are used, we looked at the location and accessibility of the privacy policies. In the case of the Netherlands, all outlets present a cookie notification before readers can enter the sites. These notifications inform the readers about the tracking of their personal data on these sites and provide links to the more detailed cookie statements. The cookie notifications vary both in terms of their length and language style. In the case of popular media (Telegraaf and Algemeen Dagblad), the notifications are rather long and formal, whereas quality media (NRC Handelsblad and Financieele Dagblad) put more effort into making their notifications clearer and more understandable.

Neither the Brazilian nor Russian outlets use cookie notifications to inform users before they access the websites. This distinction can be attributed to the absence of EU-style data protection legislation. Instead, three out of four Brazilian outlets make their privacy policies visible at the bottom of the front pages. An exception is the popular outlet Super Notícia, whose privacy policies are detectable only after going through several sub-pages and reaching the larger news portal, O Tempo. In the case of Russia, accessing privacy policies is a non-trivial task: the privacy statements of Mosskovskii Komsomolets and Izvestiia are accessible only through several sub-pages (including ones with rather non-intuitive titles, such as “Third-party Advertisements” in the case of Mosskovskii Komsomolets). In other cases (Rossiiskaia Gazeta and Argumenty i Fakty), links to the privacy policies are offered only during user registration on the respective portals.

Names of privacy sections

Finally, we examined how websites’ sections on privacy policies were named. Similar to our observations on accessibility, we identified major differences between countries and not between media types. Both the Dutch and Brazilian outlets specifically referred to privacy in the sections’ names, using titles such as “Privacy” and “Privacy Policies”. In contrast, the Russian outlets tended to avoid such normative references: both quality outlets used the title “User Agreement”, one popular outlet (Argumenty i Fakty) called it “Confidentiality Politics”, and another discussed privacy in the “Third-Party Advertisements” section.

The lack of references to normative concepts (i.e., privacy) in the case of Russia can be attributed to the slow adaptation of Russian data protection to the changing digital landscape (Kukushkina et al., 2017). Despite the rapid deployment of cutting-edge technologies (e.g., online tracking and behavioural advertising) by Russian media industries, corresponding regulations have not yet been developed. Together with the weak civil society (Brenchenmacher, 2017) and growing use of online surveillance (Gainutdinov, 2017), these reasons may contribute to the limited visibility of privacy matters in the documents of Russian news media.

Personalisation and privacy policies

The mention of personalisation

We started our analysis by examining references to personalisation in privacy policies. We found that only half of outlets actually mention personalisation, which is usually done by providing a general reference to “personalised services and offers” (Izvestiia, n.d.). With the exception of Financieele Dagblad, which provides a detailed description of its user-driven personalisation system, references to personalisation generally remain vague and are usually presented as an extra reason to justify personal data collection and not as a specific service.

The vagueness of personalisation references is reflected in the lack of differentiation between advertisement and news personalisation, particularly in the non-EU quality outlets. While popular outlets (Super Notícia and Mosskovskii Komsomolets) often explicitly inform their readers about the use of advertisement personalisation, the quality outlets in Brazil and Russia tend to use more ambiguous language. O Globo, for instance, mentions personalisation without further specification; similarly, Izvestiia rather generally refers to personalised services without actually specifying which (editorial or commercial) content is personalised.

The lack of differentiation between types of personalisation results in a lack of clarity about their technical distinctions. For instance, it is unclear whether the same types of user data (e.g., demographic information or content interaction history) are used by both advertisement and news personalisation. The absence of such information not only limits the ability of users to control the algorithmic systems used by the specific outlet but also makes it harder to determine what forms of personalisation are actually at work. Such obscurity highlights the possibility of blending commercial and normative aspects of personalisation and limits users’ ability to control what information they receive (e.g., by preventing users from opting out of a specific form of personalisation because advertisement and news personalisation are not differentiated from each other).

The complexity and type of language used

After examining how privacy documents referenced personalisation, we evaluated the general use of language in relation to news personalisation. Our observations point out that the majority of quality news sources, together with some popular outlets, use rather colloquial language to communicate the use of personalisation. One illustrative example comes from the Dutch quality outlet NRC Handelsblad (n.d.), which explicitly states that “journalism is our product, not your data” and details its “privacy promises” with simple rhetorical questions, such as “Does NRC sell your data?”, followed by simple answers: “No. Never. Nowhere”.

In some cases, news outlets adopt more formal language, for instance, by referring to specific legislative documents or using jargon or technical terms. For instance, Financieele Dagblad (2019) describes in detail how it interacts with Amazon Web Services; however, despite their significant informative value, statements such as “We have only given a very limited, minimum required number of employees access to the data. And only if this is necessary for the performance of the function. Moreover, access to the data is only authorised for that part of the data that is necessary for the execution of that function” are probably not very engaging for the readers.

The kind of data collected

Following the examination of the language of privacy policies, we moved towards analysing the way in which these documents discuss the use of newsreaders’ data. The first part of our analysis – data collected by the outlet – indicates substantial similarities between popular and quality outlets. With the exception of Russia, where popular outlets which we examined remain rather tight-lipped about data collection (e.g., by making formal references to Russian data protection legislation), outlets of both types list a number of types of user data that are collected. Generally, the types of data listed depend on the service that is used (e.g., subscriptions, advertisement/special offers, or contact forms for approaching the news outlet).

The majority of outlets differentiate between two major categories of data. The first category includes data provided by the reader during registration or explicitly added after registration. Such data include name, surname, region, telephone number, email address, postal address, location, photo, date of birth, links to personal profile on social networking sites, and so on. The second category involves data collected automatically through the reader’s interaction with the website; these include user IP, cookies, browser type, device type, time of access, address of requested page, and user-agent data.

A number of outlets also note that they use data about their readers, which they acquire from third parties. All Russian outlets that we reviewed present demographic user profiles in the advertisement sections; these profiles are based on personal data collected by the unspecified third parties. Similarly, one Dutch popular outlet, De Telegraaf, mentions that it enhances its data with user data from third parties, such as customer database companies. Other Dutch news organisations (Financieele Dagblad and Algemeen Dagblad) state that they merge their user data with the data provided by their partners who participate in a joint digital advertising initiative called “Buymedia”.

The purposes of the data collection

After identifying different types of collected data, we examined the declared purposes of this data collection. With the exception of the Russian popular outlets, which scarcely note data collection, all outlets cite a number of reasons for collecting their users’ data. The two most common purposes (referenced by 10 out of 12 privacy policies) are communication with customers and improvement of newspaper services.

The former purpose includes both general communications (e.g., for processing user requests) and targeted advertisements (e.g., pushing updates about new offers). A similarly broad interpretation is used in relation to the second purpose: the services mentioned in the documents vary from the general improvement of the outlets’ products, due to a better understanding of how readers use digital services, to more concrete tasks, such as optimising the interactive experience of users’ navigation on the website (O Globo and Folha de S. Paulo).

We noticed that normative concepts are rarely used to communicate the purposes of data collection. The majority of outlets tend to describe data collection in rather instrumentalist terms and imply that it is necessary to optimise the services that are provided to the users. The single exception to this rule is found in the case of NRC Handelsblad, which states explicitly that it uses data collection not to build user profiles or to follow readers on the internet, but to improve its journalism.

Data storage and sharing

In contrast to the relatively detailed description of user data collection and its purposes, privacy policies usually remain obscure about data storage. Such obscurity is particularly pronounced in the case of the popular outlets in our sample, in particular the two Russian ones, which ignored the matter of data storage completely. When data storage is mentioned, it is often described in general terms that leave significant space for interpretation. For example, Rossiiskaia Gazeta (n.d.) notes that it “stores personal data but puts significant organisation and technical effort to protect users’ personal information from illegal or accidental access according to the legislation of the Russian Federation” (without specifying concrete legislative acts). Similarly, Folha de S. Paulo (2018) notes that it will use “all means required to protect data confidentiality, integrity, and availability”.

Concrete specifications regarding the geographical location of data storage and the length of the storage period are mentioned exclusively by quality media. The degree of concreteness varies significantly between countries and can be attributed to the different legislative contexts (e.g., the GDPR in the EU, which explicitly requires organisations to disclose who will receive the users’ data, how long the data will be stored, and any intention to transfer the data outside of the EU). The Brazilian outlets, for instance, state that data will be stored in the companies’ databases, which can be accessed only by authorised and qualified persons, but do not specify the geographical locations of their databases. The Dutch outlets explicitly mention the period of data storage, which varies from six months to five years depending on the specific regulations. Furthermore, one of these outlets (Financieele Dagblad) states that it stores all user data in one central data warehouse environment, which is physically located on the European mainland.

Concerning data sharing, almost all news outlets note that they can share user data with third parties. Usually, outlets state that they can share users’ data with partners that are involved in their business operations (e.g., suppliers, software builders, and advertising agencies), as well as with (tax) authorities, if this is required by legislation. The scale of this sharing and the degree of transparency about relations with third parties vary between countries. For instance, the two quality Russian outlets in our sample – Izvestiia and Rossiiskaia Gazeta – note that they can share data with third parties in cases that fall under Russian legislation (without, however, specifying such cases), whereas the popular Russian outlet Mosskovskii Komsomolets notes that it collaborates with third parties for advertisement purposes, which can permit the third parties to put cookies on user machines to identify them. Both quality and popular Brazilian outlets in our sample are similarly vague in terms of disclosing the degree of third-party data sharing and mention that some third-party partners can request users’ personal information from them (O Globo and Extra), whereas the Dutch outlets that were examined tend to provide more comprehensive information about their third-party sharing (e.g., giving concrete examples of what kind of data is shared with which parties [De Telegraaf] or noting the presence of processor agreements with the third parties that regulate such sharing [NRC]).

Data processing

Among all the issues related to data use, the actual processing of user data remains particularly obscure. In all three countries, the privacy policies of the analysed media mention that personal data are used for analytical purposes, but the exact procedures remain unclear. This is particularly true in the case of popular outlets, which usually omit the subject of data processing completely. The quality media (Izvestiia and Rossiiskaia Gazeta in Russia) note that readers’ data can be used for statistical and other types of research and that the research will rely on anonymised data (Rossiiskaia Gazeta) and comply with legislation (Izvestiia); however, no concrete details are provided. The obscurity of data-processing practices leaves a conceptual gap in the communication about news personalisation in both the EU and non-EU contexts. While readers are informed about the types of collected user data and the purposes for which these data are used, they remain in the dark about what happens between the data collection and the generation of outcomes. Whether or not readers are profiled, let alone how, is not described in the privacy policies.

Data subject rights

In the final part of our analysis, we examined how the privacy policies approach the topic of data subject rights. Our comparison indicates that only the Dutch outlets devote special attention to the rights of users regarding their personal data. This difference can be viewed as another example of the impact of the GDPR on the media industry. The respective sections are found in the privacy policies of all Dutch outlets, which detail the right of users to request all the data collected on them and, if needed, delete these data. The quality outlets (Algemeen Dagblad and NRC Handelsblad) also include the contact details of the official Data Protection Officer, who can be contacted for matters related to users’ data rights.

In the case of the Brazilian and Russian outlets, the subject of data rights is generally absent. Among four Russian outlets, only two newspapers touch the subject. A popular outlet (Argumenty i Fakty) notes that if users believe that their rights are breached as part of data processing, they can contact the newspaper by email. However, it is unclear which actual person is behind the email address mentioned by Argumenty i Fakty. In the second case, a quality newspaper, Rossiiskaia Gazeta, mentions that users own their data, but these data include only content that is produced through interactions with the website (e.g., comments or photos shared by users). While Rossiiskaia Gazeta also mentions the personal data of users (i.e., items viewed), it does not state whether users own the data or not and whether they can request or access the data.

Conclusions and discussion

While some organisations are transparent about the data they collect, the way in which the news media utilise the data of their users to personalise content delivery generally remains obscure. In line with the findings from the 2018 Reuters report (Newman, 2018), our analysis reveals a common usage of user- and system-driven personalisation in all three examined countries. However, this use of personalisation is rarely mentioned in the formal documents available to newsreaders, such as privacy policies. Even when personalisation is mentioned explicitly, it is often unclear whether the term refers to advertisement personalisation, news personalisation, or both. It is similarly unclear which data or techniques are used to implement personalised content delivery.

At the same time, the disclosures indicate that data collection becomes more intense for more committed readers as a result of their increasing use of services and the growing exposure of their reading behaviour. In many cases, these two processes amplify one another, as the use of additional services and the removal of paywalls enable more engaged reading and thereby more intense collection of readers’ data. This leads to a situation in which more committed readers are increasingly subjected to the effects of personalisation systems that largely remain black boxes to them. In particular, due to this power asymmetry, those readers who intensively use personalised content delivery systems (to exercise their right) to receive information are put in a vulnerable position. While the degree of influence currently remains unclear, as well as the degree to which users would be eager to familiarise themselves with the information about personalisation and change their behaviour, we argue that the observed obscurity of personalisation systems deprives newsreaders of the opportunity to understand how exactly their data are used and the degree to which this usage conforms to their privacy expectations.

This stands in sharp contrast to current policy discussions on algorithmic transparency and accountability. Such discussions have long since moved beyond arguing for the disclosure of data collection practices and towards the need for increased transparency of the algorithms that use these data and the resulting output (Diakopoulous & Koliska, 2017). The GDPR’s right to an explanation has featured prominently in this discussion, and it is typically argued that the public is at least entitled to information regarding the algorithms’ existence and general functioning, if not explanations of specific algorithmic decisions (Kaminski, 2019; Wachter et al., 2017). Similarly, upon its entry into force , Convention 108+ from the Council of Europe, of which Russia and the Netherlands are signatories and Brazil is an observer, will provide users with the right to request information regarding the reasoning underlying data processing. Beyond these discussions of legally required information, recent work has called for further transparency regarding, for example, counterfactual explanations or information about the output of algorithmic decision-making processes (Diakopoulous & Koliska, 2017; Wachter et al., 2017a).

Such transparency is a prerequisite for achieving the goals of data protection governance. Though disagreements remain over what exact information needs to be disclosed and to whom, organisations cannot be held accountable for the ways in which they use algorithms if their existence, functioning, and output are kept secret. This is true, first, at the individual level, where transparency serves as a precondition for the individual control tools that continue to be central to data protection law (Ausloos & Dewitte, 2018; van Drunen et al., 2019). User empowerment takes on added significance in the context of the media, in which it continues to function as a tool that can offer users some protection in the absence of stricter media regulation. At the same time, the fact that many users do not necessarily read nor act on the information that is provided to them (Nissenbaum, 2011) amplifies the important role that public-facing transparency can play in producing accountability either as a result of the disciplinary action by a few motivated individuals or through more collective pressure that is exercised through civil society or other media professionals (Ausloos & Dewitte, 2018). Though such parties are less hampered by the complexity of the language we identified, the ambiguity or plain lack of information also continues to hamper accountability from this perspective.

Our analysis shows that in practice, algorithmic transparency in the media lags far behind not only discussions of what information regarding algorithmic decision-making should be disclosed but also what information is argued to be required under the GDPR and soon Convention 108+. This indicates that more work needs to be done not only to propose new transparency requirements but also to find a consensus regarding what the minimal open transparency norms required by the GDPR are and how these norms can best be enforced. Until the gap between practice and discussions on algorithmic transparency is closed, algorithmic accountability studies that presume the availability of such information will need to take into account the fact that it is often unclear whether and how (news) personalisation is occurring.

The comparison between quality and popular media showed a few differences in the ways in which they approached news personalisation. Based on the limited sample that we used, quality media tends to provide more options for user-driven personalisation (i.e., self-tailored news feeds based on email notifications) by giving readers more control over the selection of news to consume. Furthermore, quality outlets offer more information about data storage and the purposes of data collection compared with popular outlets, in particular in the case of the non-EU countries (Brazil and Russia). The distinction between quality and popular media is likely not explained by data protection law, which treats quality and popular media alike; it may instead find its roots in the different expectations of the audiences, the media’s use of transparency to build trust, or the differences in media systems. It is important to note, however, that because of the size of our sample, these observations can be treated only as exploratory findings and that further research is required to make them generalisable.

The presence of such differences within the Netherlands also indicates that while the EU sets a high transparency bar for all media organisations to follow, there is a difference between quality and popular media in terms of meeting the requirements. Our observations indicate that quality media organisations tend to disclose more information in a more understandable manner that can be explained by their willingness to comply with legislation in a more intelligible manner that distinguishes them from popular media. In addition to the media’s ethical benefits, investments in transparency, explainability, and a solid relationship with the user could strengthen the media’s position vis-à-vis its competitors in terms of the trust and loyalty of its readers and could showcase the distinction between the ways in which the legacy and social media use personalisation. Research from both the perspective of trust in the media and trust in the use of data suggests a modest relationship between trust and engagement (Curry & Stroud, 2019; Felzmann et al., 2019; Strömbäck et al., 2020). However, emerging research that combines the two perspectives by analysing how trust in the media is affected by its data usage suggests a more complex picture: although indiscriminate data collection may erode the media’s trustworthiness, readers’ assessment of the media’s data collection disclosures is influenced by both their trust in the media organisation and in online data collection more generally (Sørensen & Van den Bulck, 2020; Steedman et al., 2020).

Finally, the cross-country comparison revealed a number of major differences concerning both the basic aspects of personalisation communication, such as the type of language used (e.g., the limited number of references to the normative concept of privacy in the context of personalisation communication in the case of the Russian media outlets that were included in the sample), as well as the interpretation of more specific concepts, such as data storage (e.g., clear definitions of the storage time/physical location of data storage in the case of the Netherlands). Mirroring Groenhart and Evers’s (2017) assessment of policy documents being an indicator of media accountability, the media organisations’ disclosures in privacy policies matched the broader media accountability infrastructure of their respective countries. The Netherlands, which has the most advanced media accountability infrastructure in our sample, is also the country where transparency and explainability are reflected most obviously in our data. Media organisations in both Brazil and (to a slightly greater degree) Russia, where the media accountability infrastructure suffers from greater shortcomings, are more hesitant to provide detailed information about their personalisation mechanisms. Many of these distinctions can also be attributed to contextual factors (especially legal regulations, such as the GPDR in the Netherlands) that emphasise the importance of context in analysing personalisation practices in different environments. This arguably reflects the added value of the GDPR, as its clear transparency obligations (e.g., regarding data collection and user rights) are reflected in our analysis.

At the same time, the strong differences between countries that are evident in our analysis give a different practical perspective to current discussions on the internationalisation of data protection law. Even though “[t]he EU sets the tone globally for privacy and data protection regulation” (Bradford, 2020, p. 132), our analysis indicates that strong differences remain between the countries regardless of any extraterritorial impact of European data protection law. This may indicate that the internationalisation of data protection law finds obstacles in sectors such as the media, which remains focused on national audiences, which are intertwined with the states in which they operate (in the case of Russia), and which, in the case of the EU, continues to be subject to a lesser degree of legal harmonisation (Erdos, 2016). At the same time, legal convergence is continuing. Brazil is in the process of adopting a GDPR-style regulation, and Russia has signed on to Convention 108+ and its stronger (algorithmic) transparency requirements. Future research will track whether this legal convergence translates into increased algorithmic transparency (and accountability) in practice.

Another aspect of future research on personalisation communication that is important to acknowledge is scaling of analysis, which will allow researchers to go beyond exploratory investigation and enable them to draw more generalisable conclusions about the relationship between media types or systems and the communication of algorithmic innovation. To implement such research, it will be important to look at a broader range of news outlets and to consider different scenarios in which news personalisation communication occurs. For the current study, we did not look at the scenarios in which users employ mobile browsers and/or mobile applications to access news from the respective outlets; however, we plan to do so in a follow-up study to determine whether there are any differences in the use of personalisation and its communication between desktop and mobile devices.


We would like to thank Noemi Festic, Sherine Conyers, Kristofer Erickson, Ricard Espelt, and Frédéric Dubois for their valuable peer-review comments and editorial review.


Aguirre, E., Roggeveen, A. L., Grewal, D., & Wetzels, M. (2016). The personalization–privacy paradox: Implications for new media. Journal of Consumer Marketing, 33(2), 98–110.

Akhrarkhodjaeva, N. (2017). Instrumentalisation of mass media in electoral authoritarian regimes: Evidence from Russia’s presidential election campaigns of 2000 and 2008. Columbia University Press.

Anderson, C. W. (2011). Between creative and quantified audiences: Web metrics and changing patterns of newswork in local US newsrooms. Journalism, 12(5), 550–566.

Ausloos, J., & Dewitte, P. (2018). Shattering one-way mirrors: Data subject access rights in practice. International Data Privacy Law, 8(1), 4–28.

Azzi, A. (2018). The challenges faced by the extraterritorial scope of the General Data Protection Regulation. Journal of Intellectual Property, Information Technology and E-Commerce Law, 9(2), 126–137.

Bakker, P. (2018, April 18). Digitale oplage kranten blijft fors stijgen. stimuleringsfonds voor de journalistiek.

Balkin, J. M. (2009). The future of free expression in a digital age. Pepperdine Law Review, 36(2), 427–444.

Bastian, M. (2019). Media and accountability in Latin America: Framework – conditions – instruments. Springer VS.

Bastian, M., & Helberger, N. (2019, September). Safeguarding the journalistic DNA: Attitudes towards value-sensitive algorithm design in news recommenders [Paper presentation]. Future of Journalism Conference, Cardiff.

Bastian, M., Makhortykh, M., & Dobber, T. (2019). News personalization for peace: How algorithmic recommendations can impact conflict coverage. International Journal of Conflict Management, 30(3), 309–328.

Bastos, M. (2016). Digital journalism and tabloid journalism. In B. Franklin & S. Eldridge (Eds.), The Routledge companion to digital journalism studies (pp. 217–226).

Berry, S. (2009). Watchdog journalism. Oxford University Press.

Bertrand, C.-J. (2000). Media ethics & accountability systems. Transaction Publishers.

Bodó, B., Helberger, N., Eskens, S., & Möller, J. (2019). Interested in diversity. Digital Journalism, 7(2), 206–229.

Bowen, G. A. (2009). Document analysis as a qualitative research method. Qualitative Research Journal, 9(2), 27–40.

Bradford, A. (2012). The Brussels effect. Northwestern University Law Review, 107(1), 1–68.

Bradford, A. (2020). The Brussels effect: How the European Union rules the world. Oxford University Press.

Brechenmacher, S. (2017). Civil society under assault: Repression and responses in Russia, Egypt, and Ethiopia. Carnegie Endowment for International Peace.

Brüggemann, M., Engesser, S., Büchel, F., Humprecht, E., & Castro, L. (2014). Hallin and Mancini revisited: Four empirical types of Western media systems. Journal of Communication, 64(6), 1037–1065.

Bruns, A. (2019). Filter bubble. Internet Policy Review, 8(4).

Bucher, T. (2017). The algorithmic imaginary: Exploring the ordinary affects of Facebook algorithms. Information, Communication & Society, 20(1), 30–44.

Büchi, M., Fosch Villaronga, E., Lutz, C., Tamò-Larrieux, A., Velidi, S., & Viljoen, S. (2019). Chilling effects of profiling activities: Mapping the issues. SSRN.

Bulck, H., & Moe, H. (2018). Public service media, universality and personalisation through algorithms: Mapping strategies and exploring dilemmas. Media, Culture & Society, 40(6), 875–892.

Burrell, J. (2016). How the machine “thinks”: Understanding opacity in machine learning algorithms. Big Data & Society, 3(1), 1–12.

Chadwick, A., Vaccari, C., & O’Loughlin, B. (2018). Do tabloids poison the well of social media? Explaining democratically dysfunctional news sharing. New Media & Society, 20(11), 4255–4274.

Coddington, M. (2015). Clarifying journalism’s quantitative turn: A typology for evaluating data journalism, computational journalism, and computer-assisted reporting. Digital Journalism, 3(3), 331–348.

Council of Europe. (n.d.). Chart of signatures and ratifications of Treaty 223.

Curry, A. L., & Stroud, N. J. (2019). The effects of journalistic transparency on credibility assessments and engagement intentions. Journalism.

de Lima, V. A. (2011). Regulação das comunicações. História, poder e direitos. Paulus.

Diakopoulos, N. (2016). Accountability in algorithmic decision making. Communications of the ACM, 59(2), 56–62.

Diakopoulos, N., & Koliska, M. (2017). Algorithmic transparency in the news media. Digital Journalism, 5(7), 809–828.

Drunen, M. Z., Helberger, N., & Bastian, M. (2019). Know your algorithm: What media organizations need to explain to their users about news personalization. International Data Privacy Law, 9(4), 220–235.

Dubois, E., & Blank, G. (2018). The echo chamber is overstated: The moderating effect of political interest and diverse media. Information, Communication & Society, 21(5), 729–745.

Erdos, D. (2016). Statutory regulation of professional journalism under European data protection: Down but not out? Journal of Media Law, 8(2), 229–265.

Eskens, S. (2019). A right to reset your user profile and more: GDPR-rights for personalized news consumers. International Data Privacy Law (Online First.

Eskens, S., Helberger, N., & Moeller, J. (2017). Challenged by news personalisation: Five perspectives on the right to receive information. Journal of Media Law, 9(2), 259–284.

Eslami, M., Rickman, A., Vaccaro, K., Aleyasen, A., Vuong, A., Karahalios, K., Hamilton, K., & Sandvig, C. (2015). I always assumed that I wasn’t really that close to [her]”: Reasoning about invisible algorithms in news feeds. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 153–162.

Esser, F. (1999). Tabloidization of news: A comparative analysis of Anglo-American and German press journalism. European Journal of Communication, 14(3), 291–324.

Eurobarometer. (2019). Special Eurobarometer 487a: The General Data Protection Regulation (No. 487a; Special Eurobarometer). Publications Office of the European Union.

Felzmann, H., Villaronga, E. F., Lutz, C., & Tamò-Larrieux, A. (2019). Transparency you can trust: Transparency requirements for artificial intelligence between legal norms and contextual concerns. Big Data & Society, 6(1).

Fengler, S., Eberwein, T., Mazzoleni, G., Porlezza, C., & Russ-Mohl, S. (Eds.). (2014). Journalists and media accountability: An international study of news people in the digital age. Peter Lang.

Financieele Dagblad. (2019). Privacy statement FD Mediagroep. Financieele Dagblad.

Fletcher, R., & Nielsen, R. K. (2019). Generalised scepticism: How people navigate news on social media. Information, Communication & Society, 22(12), 1751–1769.

Folha de S.Paulo. (2018). Política de privacidade – Folha de S. Paulo.

Gainutdinov, D. (2017). Russia’s surveillance state is giving us a false sense of security. Open Democracy.

Gillespie, T. (2014). The relevance of algorithms. In T. Gillespie, P. J. Boczkowski, & K. A. Foot (Eds.), Media technologies: Essays on communication, materiality, and society (pp. 167–194). The MIT Press.

Groenhart, H., & Evers, H. (2017). The Netherlands: From awareness to realization. In T. Eberwein, S. Fengler, & M. Karmasin (Eds.), The European handbook of media accountability (pp. 170–179). Routledge.

Grupo Mídia São Paulo. (2018). Mídia dados Brasil 2018.

Haak, B., Parks, M., & Castells, M. (2012). The future of journalism: Networked journalism. International Journal of Communication, 6(16), 2923–2938.

Hallin, D. C., & Mancini, P. (2004). Comparing media systems: Three models of media and politics. Cambridge University Press.

Hampton, M. (2010). The fourth estate ideal in journalism history. In S. Allan (Ed.), The Routledge companion to news and journalism (pp. 3–12). Routledge.

Hanitzsch, T. (2011). Mapping journalism cultures across nations. Journalism Studies, 12(3), 273–293.

Hanusch, F. (2013). Sensationalizing death? Graphic disaster images in the tabloid and broadsheet press. European Journal of Communication, 28(5), 497–513.

Hanusch, F., Tandoc, E. C., Dimitrakopoulou, D., Rafter, K., Ramirez, M. M., Rupar, V., & Sacco, V. (2019). Transformations: Journalists’ reflections on changes in news work. In T. Hanitzsch, F. Hanusch, J. Ramaprasad, & A. S. Beer (Eds.), Worlds of journalism: Journalistic cultures around the globe (pp. 259–283). Columbia University Press.

Harambam, J., Bountouridis, D., Makhortykh, M., & Van Hoboken, J. (2019). Designing for the better by taking users into account: A qualitative evaluation of user control mechanisms in (news) recommender systems. Proceedings of the 13th ACM Conference on Recommender Systems, 69–77.

Harambam, J., Hoboken, J., & Helberger, N. (2018). Democratizing algorithmic news recommenders: How to materialize voice in a technologically saturated media ecosystem. Philosophical Transactions of the Royal Society, 376(2133), 1–21.

Hardy, J. (2008). Western media systems. Routledge.

Hardy, J. (2014). Critical political economy of the media: An introduction. Routledge.

Helberger, N. (2011). Diversity by design. Journal of Information Policy, 1, 441–469.

Helberger, N. (2019). On the democratic role of news recommenders. Digital Journalism, 7(8), 993–1012.

Helberger, N., Eskens, S. J., Drunen, M. Z., Bastian, M. B., & Möller, J. E. (2019, May). Implications of AI-driven tools in the media for freedom of expression. Ministerial Conference. Artifical Intelligence – Intelligent Politics: Challenges and opportunities for media and democracy, Cyprus.

Helberger, N., Karppinen, K., & D’Acunto, L. (2018). Exposure diversity as a design principle for recommender systems. Information, Communication & Society, 21(2), 191–207.

Hindman, M. (2017). Journalism ethics and digital audience data. In P. J. Boczkowski & C. W. Anderson (Eds.), Remaking the news: Essays on the future of journalism scholarship in the digital age (pp. 177–193). The MIT Press.

Hoeve, M., Heruer, M., Odijk, D., Schuth, A., & Rijke, M. (2017, August 27). Do news consumers want explanations for personalized news rankings? FATREC Workshop on Responsible Recommendation.

Humprecht, E., & Esser, F. (2018). Mapping digital journalism: Comparing 48 news websites from six countries. Journalism, 19(4), 500–518.

Izvestiia. (n.d.). Polzovatel’skoe soglashenie.

Jönsson, A. M., & Örnebring, H. (2011). User-generated content and the news: Empowerment of citizens or interactive illusion? Journalism Practice, 5(2), 127–144.

Kaminski, M. E. (2019). The right to explanation, explained. Berkeley Technology Law Journal, 34(1), 189–219.

Karimi, M., Jannach, D., & Jugovac, M. (2018). News recommender systems: Survey and roads ahead. Information Processing & Management, 54(6), 1203–1227.

Karlsson, M., & Clerwall, C. (2012). Patterns and origins in the evolution of multimedia on broadsheet and tabloid news sites: Swedish online news 2005–2010. Journalism Studies, 13(4), 550–565.

Karlsson, M., & Clerwall, C. (2013). Negotiating professional news judgment and “clicks”: Comparing tabloid, broadsheet and public service traditions in Sweden. Nordicom Review, 34(2), 65–76.

Karppinen, K. (2013). Rethinking media pluralism. Fordham University Press.

Kitchin, R. (2017). Thinking critically about and researching algorithms. Information, Communication & Society, 20(1), 14–29.

Kukushkina, E., Mzhavanadze, G., & Perevalov, V. (2017). Russia. Privacy, Data Protection and Cybersecurity Law Review, 284–296.

Kuner, C. (2005). Privacy, security and transparency: Challenges for data protection law in a new Europe. European Business Law Review, 16(1), 1–8.

Kuner, C., Svantesson, D. J. B., Cate, F. H., Lynskey, O., & Millard, C. (2016). The language of data privacy law (and how it differs from reality. International Data Privacy Law, 6(4), 259–260.

Lee, E. J., & Tandoc, E. C. (2017). When news meets the audience: How audience feedback online affects news production and consumption. Human Communication Research, 43(4), 436–449.

Lehtisaari, K., Villi, M., Grönlund, M., Lindén, C. G., Mierzejewska, B. I., Picard, R., & Roepnack, A. (2018). Comparing innovation and social media strategies in Scandinavian and US Newspapers. Digital Journalism, 6(8), 1029–1040.

Li, T., & Unger, T. (2012). Willing to pay for quality personalization? Trade-off between quality and privacy. European Journal of Information Systems, 21(6), 621–642.

Macnamara, J. (2010). The 21st century media (r)evolution: Emergent communication practices. Peter Lang.

Makhortykh, M., & Bastian, M. (2020). Personalizing the war: Perspectives for the adoption of news recommendation algorithms in the media coverage of the conflict in Eastern Ukraine. Media, War & Conflict.

Makhortykh, M., & Wijermars, W. (2019, May 24). Can echo chambers protect information freedom? Algorithmic news recommenders and the public sphere in Eastern Europe [Paper presentation]. International Communication Association Conference.

Matos, C. (2009). Journalism and political democracy in Brazil. Lexington Books.

McBride, K., & Rosenstiel, T. (2013). Introduction: New guiding principles for a new era of journalism. In K. McBride & T. Rosenstiel (Eds.), The new ethics of journalism: Principles for the 21st century (pp. 1–6).

McDonald, A. M., & Cranor, L. F. (2008). The cost of reading privacy policies. I/S: A Journal of Law and Policy for the Information Society, 4, 540–565.

McQuail, D. (2000). Some reflections on the Western bias of media theory. Asian Journal of Communication, 10(2), 1–13.

Mediascope. (2018). Chitatel’skaia auditoriia Rossii [Report]. Mediascope.

Meyer, P. (2009). The vanishing newspaper: Saving journalism in the information age. University of Missouri Press.

Mitchelstein, E., & Boczkowski, P. J. (2010). Online news consumption research: An assessment of past work and an agenda for the future. New Media & Society, 12(7), 1085–1102.

Möller, J., Trilling, D., Helberger, N., Irion, K., & de Vreese, C. H. (2016). Shrinking core? Exploring the differential agenda setting power of traditional and personalized news media. Info, 18(6), 26–41.

Möller, J., Trilling, D., Helberger, N., & van Es, B. (2018). Do not blame it on the algorithm: An empirical assessment of multiple recommender systems and their impact on content diversity. Information, Communication & Society, 21(7), 959–977.

Muhlmann, G. (2010). Journalism for democracy. Polity Press.

Newman, N. (2018). Journalism, media and technology trends and predictions 2018 [Report]. Reuters Institute for the Study of Journalism.

Nissenbaum, H. (2011). A Contextual Approach to Privacy Online. Dædalus, 140(4), 32–48.

Nozal Cantarero, T., González-Neira, A., & Valentini, E. (2020). Newspaper apps for tablets and smartphones in different media systems: A comparative analysis. Journalism, 21(9), 1264–1282.

N.R.C. (n.d.). Onze journalistiek is ons product. Niet uw gegevens.

Organisation for Economic Co-operation and Development. (2013). The OECD Privacy Framework. Organisation for Economic Co-operation and Development.

Örnebring, H., & Jönsson, A. M. (2004). Tabloid journalism and the public sphere: A historical perspective on tabloid journalism. Journalism Studies, 5(3), 283–295.

Pariser, E. (2011). The filter bubble: What the Internet is hiding from you. Penguin Press.

Pasquale, F. (2015). The black box society: The secret algorithms that control money and information. Harvard University Press.

Penney, J. (2017). Internet surveillance, regulation, and chilling effects online: A comparative case study. Internet Policy Review, 6(2).

Petre, C. (2015). The traffic factories: Metrics at chartbeat, gawker media, and the New York Times [Report]. The Tow Center for Digital Journalism.

Rossiiskaia Gazeta. (n.d.). Polzovatel’skoe soglashenie o razmeshenii kommentariev i inoi informatsii pol’zovatelei na internet-portale “Rossiiskoi gazety”.

Rotaru, V. (2018). Forced attraction? How Russia is instrumentalizing its soft power sources in the “near abroad”. Problems of Post-Communism, 65(1), 37–48.

Sørensen, J. K., & Bulck, H. (2020). Public service media online, advertising and the third-party user data business: A trade versus trust dilemma? Convergence, 26(2), 421–447.

Sørensen, J. K., & Hutchinson, J. (2018). Algorithms and public service media. In G. Lowe, H. V. Bulck, & K. Donders (Eds.), Public service media in the networked society (pp. 91–106). Nordicom.

Starr, P. (2005). The creation of the media: Political origins of modern communication. Basic Books.

Steedman, R., Kennedy, H., & Jones, R. (2020). Complex ecologies of trust in data practices and data-driven systems. Information, Communication & Society.

Steinfeld, N. (2016). I agree to the terms and conditions”: (How) do users read privacy policies online? An eye-tracking experiment. Computers in Human Behavior, 55, 992–1000.

Stohl, C., Stohl, M., & Leonardi, P. M. (2016). Managing opacity: Information visibility and the paradox of transparency in the digital age. International Journal of Communication, 10, 123–137.

Stoycheff, E. (2016). Under surveillance: Examining Facebook’s spiral of silence effects in the wake of NSA Internet monitoring. Journalism & Mass Communication Quarterly, 93(2), 296–311.

Strömbäck, J., Tsfati, Y., Boomgaarden, H., Damstra, A., Lindgren, E., Vliegenthart, R., & Lindholm, T. (2020). News media trust and its impact on media use: Toward a framework for future research. Annals of the International Communication Association, 1–18.

Sunstein, C. R. (2017). #Republic: Divided democracy in the age of social media. Princeton University Press.

Toepfl, F., & Litvinenko, A. (2018). Transferring control from the backend to the frontend: A comparison of the discourse architectures of comment sections on news websites across the post-Soviet world. New Media & Society, 20(8), 2844–2861.

van Dijck, J., Poell, T., & De Waal, M. (2018). The platform society: Public values in a connective world. Oxford University Press.

Vartanova, E. (2012). The Russian media model in the context of post-Soviet dynamics. In D. Hallin & P. Mancini (Eds.), Comparing media systems beyond the Western world (pp. 119–142). Cambridge University Press.

Vartanova, E., & Lukina, M. (2017). Russian journalism education: Challenging media change and educational reform. Journalism & Mass Communication Educator, 72(3), 274–284.

Wachter, S., Mittelstadt, B., & Floridi, L. (2017). Why a right to explanation of automated decision-making does not exist in the general data protection regulation. International Data Privacy Law, 7(2), 76–99.

Wachter, S., Mittelstadt, B., & Russell, C. (2017). Counterfactual explanations without opening the black box: Automated decisions and the GPDR. Harvard Journal of Law & Technology, 31(2), 841–887.

Wang, G. (2011). De-Westernizing communication research: Altering questions and changing frameworks. Routledge.

Whitman, J. (2003). The two Western cultures of privacy: Dignity versus liberty. Yale Law Journal, 113, 1151–1222.

Willnat, L., Weaver, D. H., & Choi, J. (2013). The global journalist in the twenty-first century: A cross-national study of journalistic competencies. Journalism Practice, 7(2), 163–183.

Wilson, S., Schaub, F., Dara, A., Liu, F., Cherivirala, S., Leon, P., Andersen, M., Zimmeck, S., Sathyendra, K., Russell, C., Norton, T., Hovy, E., Reidenberg, J., & Sadeh, N. (2016). The creation and analysis of a website privacy policy corpus. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 1330–1340.

Ziewitz, M. (2016). Governing algorithms: Myth, mess, and methods. Science, Technology, & Human Values, 41(1), 3–16.

Zuiderveen Borgesius, F. J., Trilling, D., Möller, J., Bodó, B., Vreese, C. H., & Helberger, N. (2016). Should we worry about filter bubbles? Internet Policy Review, 5(1).

This article by Mariella Bastian, Mykola Makhortykh, Jaron Harambam, Max van Drunen, originally published on The Internet Policy Review is licensed under aCreative Commons Attribution 3.0 Germany(CC BY 3.0 DE) license.

What do you think?

1483 points
Upvote Downvote

Written by Internet Policy Review

Internet Policy Review is an open access and peer-reviewed journal on internet regulation. Scholars, regulators, journalists, activists, and other stakeholders publish in the journal in a variety of formats


Leave a Reply

Your email address will not be published. Required fields are marked *


file 20201012 19 lvc5nh

as the ultimate political survivor, Judith Collins prepares for her ultimate test

02 min 1 scaled

Platform power in the video advertising ecosystem