CfMC FAQs

Frequently Asked Questions

CfMC testing and certification

CfMC software is the principal software package in marketing research and opinion polling operations worldwide. Many clients depend on Survent, webSurvent, webCATI and Mentor to deliver core competencies over their operations.

CfMC strives to release software that meets and exceeds the expectations of the most discernable user. We recognize that if our software doesn’t run properly, our clients' bottom lines are impacted.

Unlike users of other software packages, CfMC users expect their old jobs and spec (instruction) files to continue to run, even as many new features are being added to the software. This is the greatest challenge in testing and ultimately certifying a new CfMC software release.

For a fully certified release, all jobs that have worked in the past must continue to do so with the same results. When this standard is not met, a conscious decision has been made by CfMC developers and staff to implement and accept the new technique as correct and an improvement on the software’s capabilities.

Mediating and documenting this process is a major part of the events leading up to a fully certified version of the software. Bugs and wishes reported since the previous release are evaluated for overall value and usefulness. Updates determined to be important to users (and therefore to CfMC) must be fixed or implemented before the new software is released.

testing suites

Evaluating the current state of the software is achieved by running what is now a very large suite of tests. There are just short of 4,000 numbered test setups and over 10,000 spec files overall in the current (December 2007) test suite.

Mentor, Survent, webSurvent, webCATI, Fonebuld, Foneutil and other CfMC programs are all a part of the testing process. It may seem almost unbelievable but all of these tests are run for each new compile of the software. In order to handle such a large number of tests, most of the results are checked by machine against a set of what has been determined to be correct results.

testing (con'd)

A software compile that has no errors (except those that are expected) and no differences between the base line files and the current files, gets a status of “beta”. The baseline files include the prt, dlm, asc, htm, csv, def, ban, lpr, tab, rft, rfl, hrd and chk files. The extent of this mechanical testing means that software that has a “beta” status is very good. However, this status is achieved without any human having looked at the results.

The final steps of the certification process include hand-testing interactive setups, live web-mode testing and tests that involve platforms other than Linux (where the bulk of the testing is done). If no new and unexpected problems appear then the software becomes a certified compile.

beta testing

Central in our testing and certification process is the cooperation received by clients who conduct field testing of beta versions of the software.

By installing and running beta versions of CfMC software, users assist the user community as a whole by verifying the software will work in real-life production scenarios.

Beta-testing clients are encouraged to “hammer-away” at the newly-certified software and are in constant contact with CfMC developers and engineers, providing feedback about how to improve the intended capabilities and functionality.

Beta testers enjoy the benefits of being “ahead of the curve” and having access to software that few other users are experiencing; thus enhancing their competitive edge. They also get to give their input regarding the details of the development of new features.