Managing the cellular handset certification bottleneck.

As of mid 2007, there were more than 700 Global System for Mobile Communications/Universal Mobile Telecommunications System (GSM/UMTS) networks in over 200 countries (1) with 2.5 billion subscribers using an increasing range of cellular services. With an expanding range of devices being introduced, the mobile user has never had so much choice.

[ILLUSTRATION OMITTED]

However, there also is a rising expectation in the quality of service and performance of these cellular services. An increasing number of households now depend solely on a cellular telephone, with no fixed line installed. The traveling user has realistic expectations that devices will work anywhere while roaming nationally and internationally.

The business user increasingly relies on mobile access to e-mail while mobile TV already is available in some markets. The merging of cellular and global positioning system (GPS) technologies is set to enhance the safety and navigation capability of cellular users.

Modern devices are taking on a wider range of formats, from classic candy-bar to PC data card to powerful Windows-based PDAs. Ensuring it all works reliably and consistently is challenging--but manageable.

For devices using 3rd Generation Partnership Project (3GPP) technologies such as GSM and UMTS, the answer lies in the extensive test and trial programs conducted at many stages during a device's lifecycle. During the development process, extensive testing is conducted on all parts of the design: protocol stack, RF section, Subscriber Identity Module/Universal Subscriber Identity Module (SIM/USIM), and audio. Following completion of the development phase, the new device must go through the certification process.

So What Is the Problem?

Before cellular devices can be sold to the major network operators, they must pass the conformance tests and interoperability trials required by either of the two major certification bodies. In the Americas, this is the PCS Type Certification Review Board (2) (PTCRB); in Europe and many other parts of the world, it is the Global Certification Forum (3) (GCF).

This testing covers RF performance, signaling protocol, SIM/USIM, and audio aspects as well as the underlying enabling protocols behind the new applications such as the Multimedia Messaging Service (MMS) and Java. To test just the signaling protocol of a 3G device can, depending on the capability of the device, require more than 500 test cases to be executed.

The RF test cases must be performed in all supported frequency bands at a range of operating temperatures and battery voltages. So although there are not as many individual RF test cases, it can take hundreds of hours to run them under all conditions.

Why Do It?

The certification process has proven effective in improving the quality of service offered to users. Opening a recent GCF meeting, a senior manager for one of the European network operators stated that certified terminals suffered half the rate of dropped calls on their network compared to noncertified ones.

How Is It Done?

The conformance testing must be carried out in an approved lab. There are a number of specialist labs around the world that offer certification test services using test equipment approved by GCF and PTCRB. The number of authorized test labs has increased over the past 18 months, resulting in more competition and, consequently, lower hourly rates. However, device certification still is an expensive process, requiring hundreds of hours of test time.

Larger manufacturers, with sufficient numbers of models introduced each year, can justify the expense of establishing their own labs with independent management separate from the development teams. Smaller manufacturers make use of the commercial test labs, often using more than one to achieve the test coverage and time scale required.

Either way, there is a significant management overhead involved in the certification process, which lies in the direct time-to-market path of a new device. It is essential that the certification process happens smoothly and predictably to allow all aspects of the product launch to be planned accurately. A stalled product launch due to test failures can result in a new design being scrapped.

As in many complex developments, the earlier that realistic testing can take place, the sooner faults can be detected and corrected. It is well known that the cost of error correction gets higher the later it happens.

For that reason, a precertification test phase should be considered to increase the number of errors detected before the device leaves the lab. Ideally, precertification testing will use the same type of test equipment as that used in a formal approval lab, such as PTCRB- or GCF-listed equipment, or equipment capable of executing the same test cases. The cost of this extra test phase needs to be set against the potential losses made by late delivery of new products.

Once this investment has been made, it becomes more practical to conduct cyclic regression tests on every new software build, especially if automation capabilities are available. This precaution will prevent the buildup of multiple errors that result in a big-bang approach to final testing prior to release.

Impact of Test Failures

Test failures at the certification stage are among the most expensive failures, causing bottlenecks while the test lab diagnoses the cause of a fault. Many test labs have a single system to cover each type of testing such as protocol and RF due to the high cost of the test systems.

Full conformance test systems are expensive due to the specialized nature of the equipment. In addition, the systems must be continuously revalidated to accommodate changes in the standards and functionality added to the conformance test regime by the certification bodies.

[ILLUSTRATION OMITTED]

Device Design and Development

For a smaller manufacturer, the typical design cycle of a device involves buying a chipset, licensing the protocol stack, and developing a customizable user-interface, possibly based on a standard operating system.

The main development activity is to successfully integrate the various components and then verify that the design meets the specifications. This is greatly helped by ensuring that the suppliers already have tested the protocol stack and chipset, but this increases the cost and is not always available.

In this scenario, the development and integration team, unlikely to have access to an in-house conformance test system, will rely on standard instrumentation to do as much testing as possible before sending the device to an external test lab for certification. This increases the risk that the device will not pass first time.

Larger manufacturers tend to use multiple chipset sources, including both in-house developments and bought-in ones, usually with an in-house protocol stack and user interface. There typically will be multiple design teams working in parallel on different device types covering a model range, usually based at different locations around the world.

The company's in-house test lab will provide the equipment and expertise needed to achieve the appropriate certification, depending on the market to be addressed. However, even the largest test labs usually would expect to subcontract some testing to other labs. This can be done to handle temporary overflows due to nonaligned delivery from multiple design centers or small gaps in test coverage.

One hundred percent test coverage would require the purchase of equipment from practically all approved test-equipment vendors since nearly all have some unique test cases. This is a situation the GCF is working to eliminate. A centrally managed certification test lab, often a multisite operation itself, represents a significant investment for the large manufacturers.

The efficiency of such a large operation relies on the quality of the designs being received for certification. Poor, nonconformant designs will result in unpredictable test-cycle times, making it difficult for the lab manager to schedule resources accurately.

Field Trials

Conformance tests ensure that a device complies with the relevant core specifications under repeatable laboratory conditions. Field trials guarantee that the device will work correctly in real-world conditions using a variety of infrastructure elements from different vendors.

Field trial-qualified networks in specific locations are used to execute a predefined test plan to ensure consistency from one device to the next. The field trials form an essential companion to the conformance tests, providing an important balance between lab-based and field-based testing.

More About the Protocol Test Cases

To illustrate the level of detail that the certification process involves, let's take a more detailed look at some of the UMTS protocol test cases, just a subset of the tests that a 3G device will need to pass. These tests are described in detail in the 3GPP technical specification TS 34.123.

The first part of this specification, which runs more than 5,000 pages, contains the prose-form of the test cases. For each test, the following are specified:

* Conformance Requirement -- relates back to a specific feature detailed in the core specifications.

* Test Purpose -- what is being verified.

* Method of Test -- test conditions, test parameters and procedure, expected signaling sequence.

* Test Requirement -- the condition that must be achieved to pass the test.