Conducting fair, accurate and well-documented tests has been a cornerstone of The Tolly Group's mission since Kevin Tolly founded the company.
The Tolly Group's reports clearly identify the sponsor of each test and strive to include all relevant information so that others can reproduce the tests should they so desire.
With competitive benchmarks, The Tolly Group strives to ensure that all participants are dealt with fairly.
Vendors involved in a competitive test are able to review the test plans as well as the specifics of the devices or services under test. When not testing with default settings, The Tolly Group invites competing vendors to provide the configurations they desire for their device or solution. When testing is complete, we provide their results to them. If they believe that they are incorrect, we work with them to reconcile their results with ours. Finally, we provide them with the opportunity to give us an official commentary on their results, the essence of which is included in any documents we might publish.
We endeavor to be flexible in our application of our Fair Testing Charter as new circumstances arise. Our goal, however, is always the same: provide a fair opportunity to competing vendors to make certain that their products, components or services are benchmarked accurately.
When testing consumer-class products and/or testing products using default settings, The Tolly Group may deem that notification of and interaction with competing vendors is not necessary.
In every case, our published reports comment on the level of interaction with competing vendors. Comments from competing vendors may be included as appropriate and may be edited and/or condensed for space and clarity by Tolly.
Tolly notes that any individual test project must, necessarily, be a subset of "all possible tests". Reports clearly state the scope of tests and requests by competing vendors for increased or different scope often cannot be complied with for a variety of logistical and/or timing issues. Ultimately, in every case, Tolly will conduct the test as it sees fit and report accordingly.