It’s well known that companies like Amazon and Google perform constant experimentation on their digital platforms, with users unknowingly engaging in a whole range of A/B tests as the companies test different tweaks to see if performance can be improved.

Research from Imperial College London highlights how similar experiments are rampant across the various gig economy platforms. The paper examines what impact these experiments might have on the people who depend on gig platforms for their livelihood.

Human experimentation

The researchers trawled through a couple of decades worth of data from a huge digital labor platform and found that there were three distinct phases of experimentation on the platform. Each of these phases was characterized by a unique approach to experimentation, both in terms of its nature and the way in which workers responded.

Experimentation began as an explicit process that was only performed on those who had opted into it, but it quickly became a more covert process with workers experimented on without their knowledge or consent. This evolved to the point where experimentation was running rampant, with a wide range of experiments underway at any one point.

This process resulted in a shift from experimentation that increased worker autonomy to diminishing it, with the final phase then normalizing that diminishment.

A key process

By tracking the evolution of the platform over a long timeframe, the researchers began to understand how important experimentation had grown to be for the company.

Drawing from corporate announcements and online forum exchanges, the researchers discovered that the platform initiated its initial forays into worker trials circa 2007, thereby inaugurating what the authors have dubbed the “explicit experimentation regime.”

During this phase, the platform publicly shared information about the features being tested, such as modifications to the “job search” function that improved how clients and workers connected. In an open forum post, they invited workers to take part in the beta testing of the new feature and provide their feedback.

Workers were also given the opportunity to suggest areas of the platform to be subject to experimentation and could choose to opt out of any experiment. This phase not only empowered workers with greater autonomy but also set a precedent for what they could expect from the company in the future.

Shifting habits

A noticeable shift occurred in 2014, when workers on the platform discovered that the company was running experiments with the design and display of various metrics in worker profiles without any opt-in required. While the initial changes seemed somewhat benign, they were the thin end of the wedge and encouraged the platform to grow more ambitious with their use of experimentation.

This use of experimentation was later made official by the company, but the disclosure didn’t prompt them to scale back their efforts. What’s more, the experiments began to take on a more sinister tone, with some users sent automated messages saying that their accounts would be suspended if they attempted to communicate with clients outside of the platform.

This period underlined what the researchers refer to as the “concealed experimentation regime”, in which experiments were commonly performed without the consent of workers. It’s a phase in which the autonomy of workers fell as they could no longer control whether they participated or not.

Running wild

This seemed to embolden the company, which then began to significantly increase the number of experiments they were running so that they both covered a wider range of features on the platform and also had no clearly defined endpoint. Indeed, workers would often be subject to multiple experiments at any one point in time.

This process not only diminished the sense of autonomy that workers felt but also often inhibited their ability to do their jobs. The shift in the attitude of the company was typified by the fact that they offered no acknowledgment that experimentation was taking place, much less asking for permission to do so.

Interestingly, the shift was met with passivity by the workers themselves, who didn’t either voice concern publicly or begin to leave the platform and vote with their feet. The researchers believe this reflects a grudging acceptance that they would be experimented on consistently throughout their time on the platform.

This should not be taken to mean this is good practice, however. Indeed, I’ve written previously about the importance of establishing trust when monitoring what people do remotely, and this study is a perfect example of a platform abusing the trust it had initially established with workers.

If workers cotton on to the fact that monitoring and experimentation are not designed in a way to be either inclusive or mutually beneficial then morale and productivity will inevitably fall, while worker turnover will inevitably rise.