Testing FAQ

Questions about Automated VC Model Testing

Raising the bar on how modeling and release teams create and maintain configurable product models and data is a focus for eSpline’s products and services.

The ConfigScan Automated VC Testing solution is a standalone testing application that attaches to your SAP system where you build your test cases once and use them many times. The test cases are integrated into the eSpline Avenue Managing VC solution where you can view the product model, document it, compare two models for differences and submit the model for a health check.

"Can you reduce the staff hours we spend today on manual testing?"

Yes. The ConfigScan automated testing solution enables a “write once, run many times” technique for test cases, such that the labor hours you now invest in manual testing are instead applied to creation of a suite of regression-preventing test cases. You’ll recover the investment in building the test case library the very first time it is run.

“Can we increase test coverage and quality via automatic testing?”

Yes, the time saved by automatic testing frees your staff to focus on adding more tests, troubleshooting, and repairing defects as they are uncovered.

"Is there an economical way to test more frequently?

We currently only test manually before our most significant events such as release of a new product, or major changes to an existing product We’d like to test all productive changes to our product models, instead of waiting for our customers to discover problems?"

Yes, definitely. The cost of automatically running an existing library of test cases is negligible compared with equivalent manual testing. So you can run your tests a) more frequently: not only before a major release, but before a minor or patch release, b) after a single change or set of changes by a single modeler as a unit test of his or her changes. Catching defects earlier in your build and release process reduces the cost impact of each defect.

“We don’t have a written specification for most of our configurators. How do we decide what test cases to create? “

ConfigScan provides several techniques for acquiring test cases. You can also specify whether a test is expected to succeed or fail.

build them by hand in a model object-aware environment in SAP ECC, from product specifications or existing data sources, and based on a test plan.
create en masse from configuration results on line items in documents in the sales order history
create en masse via XML import, from test case XML files that can be created by a program or even manually via an editor

“How do we tame our growing maintenance problem for larger numbers of and higher complexity of product models?“

If you can carve out the time for configurable product specifications, it might help to describe new, changed, or deleted products, write tests against the specifications, and maintain the specifications, models, and test cases. This is similar to how software vendors describe the behavior of their products. Unfortunately this often has limited attraction and ROI to VC customers, because of the extra ongoing maintenance.

Avenue Managing VC with ConfigScan takes a more pragmatic approach today, with solutions such as:

periodically compare model revisions extracted from SAP, highlight the changes, document and test the change
automatically create test cases from existing data sources such as order history (successful and rejected orders)

“How can we catch errors due to differences between SAP’s ECC and web configurators - the Variant Configurator (VC) and Internet Pricing & Configuration (IPC)?”

Your library of ConfigScan test cases runs on both the VC and the IPC, so you can create test cases that detect differences when the configurators run. In addition, some differences can be caught before the configurator is run, by Avenue Health Check, which scans VC models and reports on any of a large number of potential differences or incompatibilities it may find during the scan.

“Can we reduce time-to-market via automatic testing?”

Yes, because manual testing takes longer, and diverts effort into repetitive regression testing - or worse yet, allows defects to pass on undetected to later stages of your build/release process.