Monday, February 25, 2013

Should Automakers Be Selling “Beta” Cars?

In the introduction to a January 2009 BusinessWeek excerpt from his book “What Would Google Do?,” author Jeff Jarvis extolled the idea that the Detroit automakers should start releasing cars in “beta” form, much the way the Silicon Valley tech giant had been doing with its software products. Despite the obvious dangers of open public beta testing in the car business, it seems that a number of automakers, including Tesla, Fisker, BMW, Ford, General Motors, have been doing exactly that to varying degrees over the past several years.

One of the mantras of Silicon Valley technology companies is “Ship Early and Ship Often.” The idea is that as soon as you have a remotely viable product, you get it into the hands of users and then rapidly iterate to make the product better. For a small shop with a handful of developers producing a smartphone app, or even a larger team producing a Web application like GMail, that concept works great. In the capital-intensive world of building complex physical products like cars, that process can be a very costly kiss of death. The question is, should automakers use paying customers as beta testers without making it clear what they are doing?

The history of beta testing

The concept of structured alpha and beta testing of software has its origins with IBM. Back in the 1950s, the computing pioneer defined alpha testing as the internal product verification before a public announcement. Beta testing was the verification of the product by users before it got released to production.

Until Google went all-in on publicly releasing software that was labeled as beta in the past decade, beta releases were mostly restricted to select groups of testers who committed to providing regular feedback about problems in the product. Google’s GMail was the stereotypical example of an open beta, retaining that status for more than five years from its initial release in April 2004 until July 2009.

In the automotive realm, with a few notable exceptions, manufacturers have generally kept their equivalent of beta testing strictly in-house, using engineers and test drivers to evaluate new vehicles and systems. Even the final stage of evaluation before vehicles go on sale to the general public has been restricted to employees. For example, at General Motors, anywhere from several dozen to several hundred production-representative examples are distributed to managers and other employees in the final months before the on-sale date in what is known as the captured test fleet.

Public beta testing in the auto industry

Perhaps the best-known early example of a public beta test in the car industry was the Chrysler Turbine car program. Between 1963 and 1965, Chrysler built 55 turbine-powered coupes and loaned them to more than 200 drivers around the United States for three months at a time to gather real-world data and driving impressions.

By the end of that decade, politicians and lawyers had taken direct aim at the auto industry. Within a few years, the American auto industry had become one of the most highly regulated industries in the world, and the screws have only gotten tighter since. The combination of general product liability claims with safety, emissions, and fuel economystandards from federal and California regulators has made it increasingly difficult just to conduct the basic business of building and selling automobiles. Doing anything as daft as loaning or selling experimental cars to ordinary consumers was seemingly out of the question.

Enter the California Zero-Emissions-Vehicle mandate

Then in 1990 along came the California Air Resources Board (CARB) with a new rule mandating that automakers start building zero-emissions vehicles (ZEVs). By 1998, at least 2 percent of the vehicles from the seven largest automakers in the state had to be ZEVs, a fraction that was to increase annually to 10 percent by 2003. The major automakers immediately began two parallel efforts to develop battery-powered vehicles they could sell while at the same time lobbying to kill the mandate.

By 1996, General Motors had its all-electric powered EV1 ready, as did other manufacturers. And it also looked increasingly likely that the industry would succeed in eliminating the mandate. As a result, most of the automakers that prepared EVs — including GM and Honda — opted to only lease rather than sell these first-generation battery-powered vehicles.

With the demise of the original ZEV rule, GM opted to take back the 1100-plus EV1s it had leased and sent most of them to the crusher, arguing that the bespoke plug-in shared almost nothing with any other GM vehicle, and inspiring the film “Who Killed the Electric Car?” GM gathered a great deal of data during the five years the EV1s on the road that would later be put to use in both the fuel cell and Volt programs. For all intents and purposes, the EV1 ended up being a public beta test much like the Chrysler Turbine program.

Unlike the Turbine program, which remains dead nearly five decades later, electric propulsion never entirely went away. When it became clear that late 1990s-era battery technology wasn’t up to the task for a mainstream audience, automakers turned their attention to hydrogen fuel cells. Along with the usual internal testing of fuel-cell vehicles, several manufacturers, including Ford, Hyundai, Honda, and Nissan, began loaning out hydrogen-fueled cars to government agencies, universities, and other organizations to gather real-world data. Honda even went so far as to lease one of its first-generation FCX cars to a California family in 2005.

GM, on the other hand, made plans for the biggest public beta test since the Chrysler Turbine trials. General Motors R&D built a fleet of more than 100 electric drive Chevrolet Equinoxes with electricity provided by hydrogen fuel cells. In late 2007, GM began loaning these vehicles free of charge to ordinary drivers in California, Washington, D.C., and New York for three months at a time as part of Project Driveway. Over the next three years, the fleet accumulated more than 2 million real-world miles.

At around the same, Honda introduced its first completely bespoke fuel-cell vehicle, the FCX Clarity, and began leasing them to customers in 2009 for three-year terms. Only a few dozen of these cars are on the road in U.S., but Honda remains publicly committed to the technology.

To pay or not to pay?

In some respects, the GM and Honda fuel-cell programs were very similar in that they put “ordinary” American drivers behind the wheels of some of the most technologically advanced vehicles in the world and allowed them to use them however they pleased for months or years at a time. The distinction is that Honda is charging its users a $600-a-month lease and officially calls the Clarity a regular series production vehicle.

Since these fuel-cell test programs launched, BMW has followed suit with two waves of test programs for battery-electric vehicles. The first wave began in 2009 and put more than 500 battery-powered Mini Coopers into the hands of drivers in America and Europe for one-year terms. That was followed 18 months later by a second-generation EV prototype based on the 1 Series coupe, dubbed the Active E. While BMW charged participants $850 per month to lease the vehicles, they were clearly identified as limited-time test programs.

Like traditional tech industry beta programs, each of these manufacturers is collecting usage data from these vehicles and subjecting participants to regular debriefings about their experiences.

Shipping beta products as production-ready

As long as manufacturers are completely transparent about the fact that what they are delivering is not yet ready for prime-time, inviting the public to drive experimental vehicles is probably a really good idea. No matter how much planning engineers do, they can never anticipate and test every possible real-world scenario. Even the most conventional vehicles still display some hardware and software bugs when they first go on sale. For example, the 2013 Ford Escape saw five recalls in the first eight months after going on sale, four of which involved the 1.6-liter engine.

As automakers have transformed their products into something that often seems more akin to consumer electronics in the last few years, customers have had to come to terms that they could be treated like beta testers. Among the more prominent examples of this were the early adopters of BMW’s iDrive more than a decade ago and more recently those who opted for the MyFord Touch system in their Ford and Lincoln vehicles.

As cars have integrated more technology features and drivers have demanded the ability to use mobile devices on the road, automakers have scrambled to devise new human-machine interfaces. The more basic systems that just added auxiliary audio or USB inputs to allow music playback and hands-free calls have been relatively trouble-free. However, those that incorporate either control knobs or touch interfaces along with voice recognition to access computer-like menus have caused some customers grief. Ford has been pounded mercilessly in quality surveys for the past two years because of the complex and slow interface in MyFord Touch along with its propensity to crash and reboot, a process that can take several minutes. Since the debut of the similar CUE system in Cadillacs in 2012, GM’s luxury brand has been getting similar complaints. Both Ford and Cadillac are scrambling to develop software updates to improve the performance and reliability of these systems.

Among the newer entrants in the industry, Fisker and Tesla seem to be going even further in using their customers as beta testers and information conduits. One early Fisker customer posted extended video reviews of his new ride, including a 41-minute opus that focused just on the infotainment system.

Similarly, Tesla has had issues that seemingly should have been ironed out prior to production with the Model S. Posts on the Tesla Motors Club forum have complained about issues ranging from problems with the massive 17-inch touch-screen center console to the door latching system. Because Tesla has opted to incorporate virtually all of the usual center console controls into the touch display, having it refuse to wake up is even more of an inconvenience than glitches experienced by Cadillac and Ford owners.

More troubling are the issues with the doors. Tesla has opted to use electric door handles that sit flush with the skin and are supposed to pop out when touched. Some customers are reporting that they frequently don’t work properly, so either the doors won’t open or they open randomly while driving. Fisker and Tesla have shipped numerous software upgrades, but the cars are still far from perfect.

To beta or not?

During and after the 2008 financial meltdown and subsequent bankruptcies at Chrysler and General Motors, many pundits from the technology industry claimed that the auto industry should be more like Apple and Google, taking more risks and being more innovative. Many automakers are now doing exactly that with varying degrees of success or failure.

What those tech luminaries often fail to recognize is that the auto industry is more like the aerospace industry than Silicon Valley. The American auto industry is among the most heavily regulated businesses in the world, and even when automakers meet the letter of every law, liability lawyers are still happy to pounce at the slightest hint of a problem. Automotive engineers have to keep in mind that while a crashing smartphone is annoying, a software or hardware bug at 70 mph can be lethal.

Companies that produce nothing but software seem to be able to get by just slapping a restrictive end-user license agreement on the launch screen that absolves them of any responsibility for what happens with their products. Even computer makers such as Apple or Dell are selling products that cost from a few hundred to a few thousand dollars and are not expected to last more than a few years. Car buyers expect a car to run relatively reliably for a decade or more, and even the cheapest cars can last more than 20 years. The expectations are completely different.

When automakers make it clear that they are running a test program like Project Driveway or BMW’s EV evaluations, customers can go in with their eyes open and know that they might experience some problems. Even if customers pay for the privilege of participating, they probably won’t mind, and the engineers can learn a lot.

On the other hand, when paying tens of thousands of dollars for a regular production vehicle, mass-market customers expect it to work from the start, whether they are buying from startups like Tesla and Fisker or century-old stalwarts like Ford, GM, or BMW. Every one of these companies regularly market their new technologies as features that will make customers’ lives better, but sometimes they don’t.

Perhaps the solution is for auto manufacturers to develop ways to do more real-world beta testing of new technologies so they can find out what doesn’t work before they commit to production, making sure everyone involved knows exactly what to expect.