Gamma testing

What is it?

Once upon a time, there was a dedicated team of medical researchers developing a new drug. To make sure the drug cured what it was supposed to cure, and was safe while doing so, the team subjected it to a battery of tests, first on animals then on humans. After many years, the tests were finally complete and the team was satisfied the drug was both effective and safe. The drug was released onto the market, perfectly formed.

Once upon a time, there was a dedicated team of car designers developing a new car. To make sure the car performed powerfully and was durable, the team subjected it to a battery of tests. They tested it while driving down cobblestone roads, they tested it in wind tunnels and they tested it in car ovens. After many years, the tests were finally complete and the team was satisfied the car performed powerfully and was durable. The car was released onto the market, perfectly formed.

Once upon a time, there was a dedicated team of software designers developing a new computer game. To make sure the game played smoothly and was free of glitches, the team subjected it to a battery of tests. They undertook alpha tests in-house, assessing the game against a long list of performance objectives. Then they undertook beta tests, nudging the game out into the world, where it was required to survive the handling of its first batch of strangers, and live or die based on their criticism. After many years, the tests were finally complete and the team was satisfied the game played smoothly and was free of glitches. The game was released onto the market, perfectly formed.

But recently, something strange happened. A game was released onto the market, though it was not quite perfectly formed. There were features yet to be added, expansions to the game world yet to be designed, balances between characters yet to be tweaked. The core product was complete, but its future shape had not yet been determined.

What followed next, we call the gamma test.

What do we think?

The gamma test is a new approach to software design that is changing the way we, as consumers, engage with our products. Facilitated by widely accessible, high speed internet and online software markets like Apple‘s App Store, the release of a program onto the market no longer necessitates that it be complete.

The previously distinct phases of development and use are significantly blurred. Bug fixes, patches, entirely new features can all be incorporated into a program after it is already in use. Technically, this is achieved via centralised control – a game whose software resides not on users’ computers but online on the developer’s own servers; an online App Store that acts as a gateway between users and regular software updates. Experientially, this blurring is in fact integrating development and use with hitherto unachievable interactivity.

For example:

Version 1.0 of a game is released and users start to play. Within days, the first discussions about bugs and glitches appear on online forums. Version 1.1 is released, correcting the imperfections of its predecessor. As users rack up more game time, they discover imbalances in character strengths – more discussions appear, not only identifying the imbalances, but suggesting myriad ways of rectifying them. Version 1.2 is released, following some of the more creative proposals. Users running unusual hardware come across a few more bugs. Versions 1.2.1 fixes those. Yet more discussions appear as users begin to imagine all manner of improvements to the game and expansions to its gameplay. Version 2.0 responds to the most popular ideas, improving the user experience and adding many more months to its longevity.

The gamma test moves beyond the traditional boundaries of development, shedding discrete objectives and deadlines in favour of a crowd-sourced approach that allows the users of a program to dictate its future development.

This process provides access not just to the hardcore enthusiasts willing to sign up for the controlled environment of a beta test, but to every user every day. The user reviews on the App Store not only furnish potential customers with previous users’ opinions, they are a mechanism via which developers can obtain free and honest feedback from their users. The gamma test is ongoing, limited only by the preparedness of a developer to follow its customers’ suggestions.

What should we learn?

We would love to see this approach find its way into our practice of architecture. As the creators of one-off projects, we are almost never presented with the opportunity to tweak an element of our design to align with unexpected usage patterns. Our designs must emerge from the page and into the world complete, perfectly formed.

Only through the adoption of prefabricated processes and repetitive detailing are we able to challenge this paradigm. Through prefabrication, we force a continuous feedback between design, production and use. Similarly, though more subtly, repetitive detailing from project to project enables us to refine an aspect of a project – a typical handrail for instance, or a bathroom vanity – gradually improving upon it across decades of design.

Compared to software development, compared really to almost anything, architecture is a slow undertaking. Any incremental changes will be similarly slow. We must learn to be more proactive in seeking feedback from our clients, even when that feedback might be negative. Only an architect prepared to risk hearing his design is terrible can possibly learn how good his work really is.

About Warwick Mihaly

Warwick Mihaly is a principal architect of Mihaly Slocombe, teaches design and construction, and writes.
This entry was posted in Architecture, Product design, Technology and tagged , , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s