Print this article

Comment

01 May 2004

Purchasing vs. developing a test data management system

By Richard House

Once recognizing the benefits of a database-centric test data management (TDM) system, you then face the decision of: "Should I use an off-the-shelf test data management software product, or develop a proprietary system myself?"

Although it may seem logical, even enticing, to develop your own test data management software, there are hurdles—some obvious, others lurking behind the scenes.

As a result, the answer in most cases is that an off-the-shelf solution has lower cost, higher quality, and more flexibility than a proprietary system you develop or outsource.

Required skill

To develop a proprietary TDM system, the development team needs database, communication, test software, and Web technology skills and knowledge. The team will need:

• to know how the commercially available database products operate and perform and how to tune the specific vendor's database for proper performance
• database design skills such as normalizing the database, establishing data relations, and replicating and synchronizing the data
• to know the database technologies for your proprietary TDM system to interface your test system with the database
• database security knowledge

The team will need to know the data performance and reliability technologies available. These technologies solve the problems of buffering your data, storing data locally when the network goes down, and then reconnecting and retransmitting data anytime there is a data communication problem.

Your system may be crossing networks and firewalls, which demands familiarity with virtual private networking and firewalls.

Because TDM software interfaces the network and database to popular test, measurement, and analysis software, a good understanding of the software you will use to design your test system is also necessary.

If you want to view your data and reports using a standard Web browser, there are several technologies the team will need to know.

Because it is unlikely one person in your company has expertise in all the areas of a proprietary TDM system, it is very important you have a good software manager to coordinate the different software engineers, so the project accomplishes its mission.

Many companies fall into the trap of thinking "I only need a subset of a complete TDM solution so I can do it for less." But a development effort originally planned for a short period often grows into an extended effort.

The biggest expense in doing it yourself is the rarely considered fact that people move on and off the project because of other priorities, requiring the project to be put on hold or development knowledge to transfer to new personnel.

Whereas off-the-shelf TDM software can be up and running within days, a proprietary system will probably take a minimum of months before running the first trials. More often than not, companies take years before the entire system is complete.

The most common result of companies trying to develop proprietary TDM systems is the projects never finish.

Another cost in a proprietary TDM system is the cost of using the system. The main purpose of off-the-shelf TDM software is to save test engineers time in managing their test data. Most proprietary systems require additional time from the company's database and information technology resources, as future changes are required.

All of that said, there may be times when developing your own TDM software makes sense. But think twice. An off-the-shelf TDM software product from an established company that specializes in TDM may be more cost effective and of higher quality.

Behind the byline

Richard House is president of VI Technology, Inc., an Austin, Texas–based systems integrator, and a registered member of Control and Information System Integrators Association (CSIA). His e-mail is richard.house@vi-tech.com.