manipulation

By Jan Michael Bauer

◦ 2 min read ◦

Even before COVID, people have spent more and more time online. Particularly mobile devices have become a large part of our daily routines and for many there are few moments when the phone is not within direct reach. While studies have shown that even teenagers think they waste too much time online, surprisingly little is done to stop this trend.

But how did we get here? Several dovetailing factors enabled this development and give me little hope that this trend will slow down any time soon. While technological advancements in mobile internet and device components were necessary conditions that allow for an easy and enjoyable interaction with platforms and services at all times and places, the real champions of compulsive internet use are social and data scientists driven by monetary incentives and unrestrained by a lack of proper ethics training.

Despite the frequent regrets about the many hours wasted on the internet, people are struggling with self-regulation and apps, like “RescueTime”, with to sole purpose to block oneself from using other apps are becoming increasingly popular.

While internet addiction has not been officially recognized as a disorder by the WHO, close parallels can be drawn to officially acknowledge gaming and gambling addictions.

And this is certainly no coincidence as tech companies hire psychologists and designers to make their products and services as tempting as possible, frequently borrowing elements from the gambling industry. However, even though some tweaks based on the knowledge of capable social scientists will increase user engagement, much more can be learned about consumer behavior and how to manipulate it through the application of the scientific method itself. The use of experimentation, collection of big user data and application of machine learning algorithms are the big guns in the fight for user attention and their money.

All these efforts are used to make social media more “engaging” but ultimately sales and advertising campaigns more effective. To do so, user interfaces and features are explicitly designed to grab attention and contain what has been termed as “dark patterns”. Design elements that often tap into the subconscious decision-making processes and therefore manipulate user through purposefully curated interfaces. While such practices benefit the company, they can have detrimental effects on individuals and society as a whole.

We know that individual choices reflect individual preferences only under certain conditions, including the absence of deceptive choice architecture or marketing messages. Hence, I can’t stop wondering about the opportunity costs and side-effects of these miraculous little devices in our pockets that have grown into an ugly hybrid between a snake oil salesman and one-armed bandit.

We have free markets based on the belief that they create value for society and make people better off by efficiently satisfying their needs. The recent U.S. opioid scandal has shown that for some products, sellers’ profits might not be positively related to consumer value. It certainly gives me pause that the best offline equivalent to the “RescueTime” App is probably the Betty Ford Clinic.

We are faced with many pressing issues that would require our full attention, while people are increasingly plagued by credit card debt, the planet is suffering from overconsumption and we spent 30,000 years alone, watching Gangnam Style on YouTube.

Regarding the larger point that any efforts against these trends would hurt innovation, jobs and growth; let us take one step back and point out that the Western world has made it an imperative to ensure individual property rights and outlaw the use of violence with the explicit goal to increase investment and productivity. People can just do more good stuff, when they do not have to spend time protecting their property and family. Given our current technology and knowledge from the behavioral sciences, I think we have seen enough and should start treating distraction and manipulation as similar threads to human flourishing.

So, what could we do? In the short run, we need to find ways to reduce the stream of big data feeding these efforts, force these practices out in the open and raise awareness about their use and effects, and find effective regulation to limit manipulation efforts in a dynamic attention economy. In the long run, we probably need to go beyond those patches as these issues not only hurt individual lives and careers but also the fabric of our democracy.

About the Author

Jan Michael Bauer is Associate Professor at Copenhagen Business School and part of the Consumer & Behavioural Insights Group at CBS Sustainability. His research interests are in the fields of sustainability, consumer behavior and decision-making.

Source: photo by ROBIN WORRALL on Unsplash

By Dieter Zinnbauer & Hans Krause Hansen.

Few statist policy blueprints on matters pretty technical have captured our collective imagination as has the Chinese Social Credit System (SCS). Announced by China’s State Council on June 14, 2014, and building on experimentation with related mechanisms since the early 2000s, it sets out a hugely ambitious effort, officially described to instil societal trust, integrity and cohesion in a highly complex society. To get there it seeks to combine cutting-edge technology and vast amounts of data to create incredibly granular behavioural profiles of both companies and individuals. Good and bad behaviours are meant to be recorded in and through elaborate rating systems and blacklists, and made public on digital platforms. The expectation is that punishments and rewards will deter deviance and incentivise good conduct in close to any sphere of life.

With the West in the mirror

After years of relative in-attention, the SCS has loudly burst onto the Western media landscape. Here, it is typically described in Orwellian terms as a totalitarian system of surveillance and control. On closer inspection, the SCS is in fact embryonic, fragmentary and faced with enormous implementation challenges.

But the scale, scope and level of invasiveness associated with the data collection effort currently emerging in China should not look so shockingly unprecedented to Western publics once they begin to scrutinize their own backyards. Take the use of social media in the policing of protests as an example. Here the UK government engages in the analysis of big data to predict, pre-empt and respond in real time to a range of issues, including public dissent. Take information on someone’s physical whereabouts as another example. As it turns out the exact location of cell phone owners in 95% of the US is being tracked with the help of all major carriers in close to real time (ok, with a 15 seconds delay) and related data is being available to nudge people’s behaviour for a wide variety of purposes, e.g. by sending them last-minute campaign pitches when they wait in line outside a particular polling station or anti-abortion messages when they are found to linger outside health clinics that carry out these procedures or by sending political messages when they wait in line outside a particular polling station.

Or take the most popular new media companies. They are collecting extremely granular dockets of what their users do, say and who they socialise with on their own platforms. But less in the spotlight they also track users and non-users alike across millions of other websites and across the bulk of the most popular mobile applications, recording anything from detailed surfing behaviour down to the modes of movement – is the user currently cycling or on the train? What’s more, they increasingly merge theses profiles with billions of data points collected by other parties. One leading new media company claims to have access to information on 70% of all credit card purchases and thus approximating a rather totalitarian 360 degree, 24/7 view of user conduct, all the way to – no kidding – the barometric pressure of the users’ environment.

Public and private entanglements

A special matter of concern in the West relating to SCS is its fusion of socialist government and private sector capabilities, technical affordances and interests that make such a system feasible in the first place.

However, long gone in the West are the times when governments were the main purveyors and guardians of data about their citizens. Even the holy grail of state information prowess, the census is not immune to private sector resources and influences. The UK government for example is exploring ways to make its census more cost-effective with the help of other big data sources and acknowledges that this will also have to include privately-held ones.

And there is also a proximity of big tech and political actors on a much more fundamental level. Tech companies evolved into some of the most vocal and most prolific donors and lobbyists on the political scene. An entirely legitimate democratic engagement, but it raises questions about outsize influence given the scale of these efforts. Yet, much more unnerving, the leading social media and tech companies in the US seconded staff as pro-bono experts to become part of the support teams of most presidential candidates in the run up to the 2016 presidential elections, giving them unique insights and connections into the affairs of some of the leading politicians in the country.

Subtle social sorting and weak institutional safeguards

A factor that explains the extraordinary attention that the SCS has received might pertain to the breadth of sanctions and consequences that these early uses have already resulted in. Bad social credit makes it more difficult for Chinese citizens to travel, find a home or get a job. Unfortunately, this is nothing new and happens all over the world. Under the label of risk- management citizens whose criminal record or financial credit history contains some irregularities have long been subjected to inferior treatment when renting a home, looking for a job or seeking insurance.

In principle, the protection of individual rights and limits on state over-reach and surveillance in most western countries relies on a host of elaborate institutional safeguards, checks and balances. While some of the egregious examples referenced above have actually been remedied when they were exposed, thus attesting to some degree of efficacy of legal and broader societal protections, other incidences have not been resolved and are somehow even seen as acceptable.

So shifting some of the attention and moral outrage that is being directed towards the Chinese SCS back to the home turf, and to investigate what troubling data practices and regulatory gaps that are germinating over here is more than warranted. In the wake of the Facebook and Cambridge Analytica scandals this has begun to happen and more commentators are noting the troublesome parallels between Chinese SCS and emergent data surveillance and discrimination issues in the West.

Enter the urgent business of business

And this is where business and its social responsibility comes in. Because one of the fundamental differences between the SCS and many issues in the West is that the disciplinary power, control functions and discriminatory implications of big data-driven social scoring are not primarily organised and instrumentalised through government, but deployed by the private sector and working their way into everyday lives.

Egged on by a growing populist Tech-lash, a whirlwind of new regulatory efforts and undoubtedly also in many cases by a deeper sense for doing no harm, the new tech companies have begun to take note, moving from denial to a gradual re-examination of some of their working principles, practices and normative anchoring.

Yet, the proof is still in the pudding whether this is a substantive change of minds and hearts. The Performance of the new tech sector on some standard measures of corporate integrity and transparency is still mediocre and lagging many other established industries.

The ways to a much more comprehensive, proactive and transformational integration of corporate social responsibilities into the strategy and practice of tech will have to coalesce around a broad band of issues, ranging from responsible stewardship of data, platform power and emergent artificial intelligence capabilities to bread and butter CSR issues such as responsible corporate political activity and supply chain and subsidiary integrity.

Think tanks and tech activists are putting forward a sprawling pool of ideas and initiatives from data collaboratives or privacy by design standards to high-profile research endeavours into artificial intelligence ethics. Meanwhile European regulators are putting into force trailblazing rules as we write this column.

But a big tech embrace of a substantive and comprehensive notion of corporate social responsibility is urgently required to stave off the threat of an even more populist, illiberal, unequal, misogynistic and fragile future in which the tech industry is more part of the problem than a solution to it.

Dieter Zinnbauer is Governing Responsible Business Research Fellow at Copenhagen Business School in the Department of Management, Society and Communication.

Hans Krause Hansen is Professor at the Department of Management, Society and Communication at Copenhagen Business School. He teaches and researches about various aspects of public and private governance, including corruption, anti-corruption and transparency regimes in the global North and South.

Pic by Alias, Flickr.