[nukeblitz]

To start off, i'll draw a parallel between development of computer software and that of cars. During the production of most computer programs, broadly, and perhaps somewhat inaccurately, they go thru the following stages.
1. Developers builds - these are shared between programmers working on different components of the same program.
2. Alpha test builds - more or less complete builds which are tested inhouse within the production company.
3. Beta test builds - these are completed programs that have passed the alpha stage. These are released to selected beta testers who are not part of the company or to the general public who choose to download them.
4. And finally, after ironing out all the problems discovered in the last stages, a finished product is released which may go thru many versions and revisions.

And coming to cars, which, since i have much less experience with, i'll make a few assumptions.
1. Development - the engineers will probably share designs and make different parts work together.
2. Alpha stage - i'll assume these are the heavily disguised test mules which v get scoops of since all the testing is secretive and inhouse.
3. Beta stage - barely exists. Will come to this later.
4. Finished product which goes through many revisions, versions, generations and facelifts.

My question is, why is the beta stage ignored? If i was a car manufacturer, and i was designing a car, then after the alpha stage where i do a ton of testing inhouse, i would want my car to go through a beta stage. This would include.
1. The lease of a few test mules to selected beta testers for feedback, problem reports and suggestions.
2. There beta testers would include various car magazines, forums like team-bhp etc. I'd also give a few test mules to my car dealerships which either have large numbers of customers or have customers from my target groups.
3. Let the beta testers use the test mules as they like for as long as i deem necessary. Maybe 6 months or so and collect feedback.
4. Use the feedback to improve my vehicle and to price it right and to fix long term issues so that it's perfect from the start.

Here are the advantages i see.
1. Public reaction, review and feedback. Basically from the average indian customer.
2. Longterm reliability issues can be corrected as feedback can be collected as the mules will be driven over long distances for long periods than is possible inhouse.
3. Professional reviewers can help finetune the car. Punto 90hp is a good example of where this would have helped.
4. Car magazines would give free publicity for the upcoming vehicle along with suggestions.
5. There are a lot of customers who think twice before buying a new vehicle and would probably feel safer if knew the car underwent a lot of public, realworld development.
6. Realworld usage and silly annoying issues can be highlighted and fixed. The vw vento's remote locking system, for example.

The disadvantages i see are
1. Cost of production of test mules. Shouldn't be of much weight compared to the valuable data that would be collected.
2. Time factor. Final release would probably be delayed. But i would want to want put out a perfect product even if i'm a little late than an average product too soon.
3. My product in the same segment would probably take a sales hit. But that's generally only if it's a refresh or an upgrade and those wouldn't really need all that testing. A brand new product would probably hit the sales of all cars in the same segment.
4. No thunder on launch. Dunno if that's all that of a bad thing. Many people don't trust new products.
5. Early bad reviews that might hurt the image of the vehicle. The trick would be to make everyone understand that it's just a test vehicle and that all the problems in the test vehicle would be addressed in the final product.

It looks like tata did this to some extent with the aria. But i'm talking about a more extensive program all together.

Corrections, ideas and any debate would be welcome.

[auto-one]

nukeblitz, you have a valid point. I was also wondering about the same.
You have answered most of it in your post itself. I wish to add that companies may not like disclosing the specifications / design / features, which cannot be hidden during the beta test. The car market is too competitive and companies cannot afford to disclose certain things related to the vehicle.
However, I feel, every new launch / facelift should have a beta test,for which the companies should have their own set of dedicated testers.

[kaushikr]

Beta testing of Vehicles tends to be slightly different than that of software IMHO. In software beta testing, if something fails/software crashes, it is usually the process of gathering data and sending it to developer and restarting the software. The same cannot be done in case of vehicles.

The first problem with beta testing (apart from the disadvantages already listed) is safety. There can be no 'disclaimer' from auto OEM's on that, and labeling a product 'beta' does not help there.

1) The testing feedback is usually taken from experienced in-house testers before a product is launched. These engineers understand the risks involved with the test vehicle and are capable/trained to deal with it. Even-though the vehicle has passed 'alpha' stage by now, the vehicle is still not production ready and hence might have some undetected flaws.

2) Any new vehicle introduction is completely tested for about 6-12 months on road before release. These test mules as they are called are backed up by support vehicles and tested under different terrains and conditions.

3) Once that is done, when the upgrade does not involve design changes to the vehicle, i.e. engine/similar upgrades are usually thoroughly 'beta' tested. AFAIK in case of CV's these vehicles are given to a select customers, clearly stating the changes and risks involved in driving the 'test' vehicle and the feedback is solicited.

4) From the reliability point of view, all components are tested on a test bay over varying conditions like load, ambient conditions, running hours etc.. for performance and reliability.

5) Also, usually the price point of the product is decided before the product design begins.

Though the advantage listed are realizable, auto companies have customer focus groups, collect feedback etc to design and develop a product apart from using the above steps to make sure that a reliable and safe product is launched. Probably that is why they avoid 'beta' testing on potential customers and critics.

Dear friends (nukeblitz, auto-one)

Every auto manufacturer in India and abroad has broadly these developments stages: Alpha stage, Beta stage, Pilot stage, Production Proving stage, Pre launch stage.
The most important and substantial work is done in alpha and beta stage. But the fine tuning and actually customer related points are addressed in Pilot and Production proving stage.

Any other bhpians who are working for auto companies in India or abroad would agree with my explanation.

Also since you draw an IT analogy, in the same way that an IT company has a support group to monitor any project/pilot in the initial stages, every OEM has a product support group which monitors every new product in the first 6-12months. All OEMs suffer from initial problems. Its the capability of the OEM support group to handle the initial problems which crop up and take CAPA (Corrective Action-Preventive Action).

Also lastly i would want to clear another fact that i read often amongst us bhpians. No manufacturer can bring in a perfect product first time around. There are factors which just cannot be discovered however vigorous be the OEMs testing procedure.
Even the most reliable manufacturers Honda and Toyota have suffered Global Recalls in the recent past, this doesnt mean that Honda or Toyota have downsized their testing or new product development plans/expenditure.

[aargee]

Beta test is not excluded, but rather done in a very closed loop or restrictions. If it weren't there, then we do not have so many test drives.
http://www.team-bhp.com/forum/test-d...ve-review.html
http://www.team-bhp.com/forum/test-d...ve-review.html
http://www.team-bhp.com/forum/test-d...ve-review.html

I'm sure there're other participants from various other popular auto magazines.

TVS is doing one such test with its Max 4R; source - The Hindu Business Line : Bearing the rural burden

But I'm wondering, if such tests are being carried over in western countries & why alone India?

[GTO]

Nukeblitz, you'd want to stop by this thread too: How vehicle testing is done

[k_nitin_r]

I just thought I'd mention that manufacturers do have beta tests. A former classmate from school had a Jeep Cherokee test vehicle that was bought at a discounted price for entering the 'beta' program. They had to let the technicians collect data from the devices that were located underneath the seats as a part of the program. This was in Sharjah in 1998-1999 and is the only beta test that I know of. Most people I know wouldn't buy a vehicle that isn't mass-market (like the present-day Suzuki Swift) because if the crowd is buying it then it must be with minimal flaws, aftersales service, and re-sale value.... or so the thought-process goes.

Originally Posted by nukeblitz

Beta stage - barely exists. Will come to this later.

My question is, why is the beta stage ignored? If i was a car manufacturer, and i was designing a car, then after the alpha stage where i do a ton of testing inhouse, i would want my car to go through a beta stage. This would include.
1. The lease of a few test mules to selected beta testers for feedback, problem reports and suggestions.
3. Let the beta testers use the test mules as they like for as long as i deem necessary. Maybe 6 months or so and collect feedback.

[Captain Slow]

Well the first few figo owners did all the Beta testing for FORD!! My car is from this batch!

[Arch-Angel]

The analogy is in the correct sense but with the competition between the manufacturers, you can rest assured that even the final stages of pre release testing of the vehicle, majority of its features will be hidden from prying eyes. If i assume that the vehicle testing is in perfect parallels with the software world, we would have something called as the UAT(user acceptance testing) which is of utmost importance. I feel this would have relevance to auto testing since every customer has an opinion of what he wants or what he likes to see. With the inputs from the user, the car manufacturer can decide what to add or improvise to provide a better experience to all the potential customers when the car rolls out of the production line.

Another thing that software companies follow is iterations of builds and phasing, where the product in development would go through a multi stage testing and modification and after each iteration, a new build is created which would be the baseline for the future builds. Car manufacturers seem to be in a hurry when releasing a new car and therefore, over cost concerns, time is of utmost importance. That is the main reason you don't see the complete process very often. If this did happen, you would have a wonderful car.

Originally Posted by Captain Slow

Well the first few figo owners did all the Beta testing for FORD!! My car is from this batch!

Sir, even after this "beta testing" the facelift isn't upto anyone's expectations. It is a good car but thats about it.

[tsk1979]

Its not just cars now. Nothing is really properly testing. The market focus is now on
"Presentation".
Engineering takes a back seat. Most of the budget and money is enmarked for smoke and mirrors. Whatever trickles down goes to engineering. Engineering folk usually make the cut from testing budget.
The main reason for this is that in this quarter to quarter driven world. smoke and mirrors have a great impact, however impact of solid engineering takes years to show on the bottom line. since most executives don't even know where they will be working the next month, polishing the next quarter is the primary agenda, and "Proper testing" somehow does not fit in, as the fruits of this labor are usually reaped in years.
So its more of an attitude problem.
Unfortunately, "PPT can fix everything" attitude is percolating all over the Automobile Industry. The race is on to get out the next version as soon as possible. That's what looks good on magazine covers and that is what brings footfalls.
Solid engineering is mostly dying.

[Arch-Angel]

Originally Posted by tsk1979

Its not just cars now...Solid engineering is mostly dying.

I couldn't agree more Tanveerji. This is nothing but an unfortunate impact of the "quantity" driven world we live in currently. Earlier, it was fun to see the ford mustang being tested and also from what i have heard, brands like Koenigsegg and other premium manufacturers test their vehicles extensively even now. One might say that these brands have the capital to indulge in such processes. Though it is true, it is just one side of the story, the other being the quality focus these companies have. Every manufacturer who releases new cars in the markets tests their vehicles extensively for issues and only then is the vehicle given the all clear to be released. Manufacturers are even skimping on providing safety gear on cars. As long as the price is low and the car has an engine, people will buy it. Such cars would be boycotted by other countries until the manufacturers fix it. We have but ourselves, with our "chalta hai" attitude, to blame.

As far as engineering is concerned, it is a dying art. Cars of the yesteryears are more attractive and better put together than some of the cars today. Whoever said "Old is gold" sure knew what he was talking about.

OT:

One more unfortunate thing is that manufacturers reserve the good stuff for the "developed markets". Why? Indian car market is by far the most challenging one to crack with the only success story being that of MSIL.
I really hope there is good competition and we, the customers benefit most out of it. Its just pointless seeing good cars running without safety gear.