Search AIIA.biz:

Acceptance Testing

WHAT IS ACCEPTANCE TESTING?

Acceptance testing is the process that the parties to an agreement use to determine whether a product or service meets the requirements of the agreement. There are different types of acceptance tests depending on the product or service being provided. In this section we focus on acceptance testing for fixed price software development agreements.

Why is Acceptance Testing Important?

Acceptance tests are important because:

They define whether the deliverables provided by the supplier meet the requirements agreed by the parties;
The consequences of failing to pass an acceptance test are usually serious for the ICT supplier; and
Acceptance usually triggers a substantial milestone payment, or payment of the total fee.

WHAT TO LOOK FOR IN ACCEPTANCE TESTING CLAUSES

What is being Acceptance Tested?

Each different type of deliverable usually has a different type of acceptance test process, and/or criteria. It is critical to fully describe the subject matter that is to be tested to ensure that the acceptance tests are applicable to that subject matter. For custom developed software the full functionality and performance criteria of the software should be described.

Particular care should be made when describing the following items:

The tests need to be carefully designed so that they accurately test against only the requirements in the agreement and that they are to be repeatable so that an independent third party could objectively determine whether the deliverable meets the requirements of the agreement. If the specifications for the deliverables in the agreement are not sufficiently detailed to allow an objective assessment of whether the deliverable has meet the requirements in the agreement then the parties run a significant risk of dispute. The size of the exposure to both parties may be many times the size of the fees for the original agreement.
Any performance characteristics; how will these be measured and over what period? Are there any external factors which will effect performance?
Documentation is usually visually tested for completeness, accuracy and format.
If there is phased delivery of components, which items of functionality are accepted at which times? What are the consequences for previously accepted components if the overall suite of deliverables is rejected later when they are tested for integration issues at the final acceptance test?
The ICT supplier should not be responsible for any components of any system or solution that are not provided by the ICT supplier (e.g. hardware, software, networks or coding that have been provided by the customer or items that are acquired by the ICT supplier as agent for the customer).
Ensuring that any changes to the specifications or deliverables are reflected in updated Acceptance Test criteria.

What standard of testing is being used?

The standard to which the deliverables are tested should be an objective one. To the extent that a subjective test is used, i.e. “the reasonable satisfaction of the customer”, then the IT supplier runs the risk of not being able to pass the acceptance tests because of the “opinion” of the customer. In particular, there are significant risks with certain types of “user” acceptance testing, as the “users” have no understanding of what the agreement requires the ICT supplier to deliver, and therefore they may provide many objections to the deliverables which were not based on the obligations or specifications required under the agreement.

As custom built software and associated documentation will almost inevitably contain some errors, the “materiality” of these errors is important. The ICT supplier will want to have any errors categorised by their impact on the overall deliverable and have different consequences apply depending on the number and severity of the errors. It is suggested that if there are errors that are “material” (i.e. they substantially affect the overall use of the deliverable) then the Acceptance test result is a fail, however if there are only immaterial errors then these can be remedied within a warranty service, then the deliverable should be conditionally accepted (and trigger payment for acceptance). Any errors that are only superficial should be corrected only if it is reasonable to require the ICT supplier to remedy them. Alternatively the Acceptance test clause can specify that the Acceptance test will be passed when the deliverables are in “substantial conformity” with the agreement requirements.

ACCEPTANCE TEST RESULTS

If there are any errors found during acceptance testing then it is essential that the customer gives the ICT supplier sufficient details of the error to enable correction. This means that the customer should provide an accurate description of the error (including the expected test result and the actual test result), how that error means that a specific requirement in the agreement has not been met, and the impact of the error on the overall usability of the deliverable. It would be unreasonable to require the customer to have to state the cause of the error, as it is the ICT supplier’s responsibility to diagnose the cause.

If the customer fails to run acceptance tests (where it is the customer’s responsibility to do so) or having run the tests the customer does not issue a notice specifying that the tests were failed within a prescribed time frame, then the ICT supplier should be entitled to assume that the deliverable is accepted. This is usually dealt with by including a deemed acceptance clause in the agreement that states that if there has been no notice of failure of the test within the Acceptance Test period the deliverable is deemed accepted.

Further, if the customer puts the deliverable into production the agreement should expressly state that the deliverable is deemed to be accepted and that the customer accepts the risk of not putting the deliverable through the testing process. There are significant risks to both the customer and the ICT supplier of allowing a customer to run live data in a production environment without going through thorough acceptance testing. An alternative is to include in the agreement a provision that prohibits the deliverable going into production use prior to passing the customer’s Acceptance Test.

What are the consequences of failing to pass Acceptance Tests?

The agreement should set out the remedies for failing to pass the acceptance test. These remedies usually include some or all of the following:

Rejection of the deliverable and a refund of all fees paid. (As it is likely that all deliverables will contain some errors upon initial delivery it is recommended that rejection is only available after the ICT supplier has been given an opportunity to remedy the errors and resubmit the deliverable for testing and the subsequent test has also been failed).
Termination of the agreement (this right should only be available when the customer is entitled to reject the deliverable and has exercised that right).
A requirement to remedy the errors within a specified period at no cost to the customer.
The customer being able to remedy the errors themselves, or through using third parties, at the ICT supplier’s expense.

In addition the ICT supplier will be liable for damages for breach of contract, and in some cases, will be liable for liquidated damages for failing to deliver on time.

If the Acceptance Test has been passed has the liability for delivery finished?

Generally, the parties will be free to specify in the agreement what the consequences of acceptance will be and will be bound by their agreement. For instance, provided the agreement is clearly drafted to require the customer to accept a deliverable which passes testing and to pay the price, the customer cannot then avoid these consequences simply because it still has some dissatisfaction with the deliverable.

However, there are a few qualifications to this including:

The customer may still have a claim in damages notwithstanding acceptance where the deliverable does not comply with the contract specifications;
The Trade Practices Act 1974 (Cth) will imply non-excludable conditions and warranties (for instance, that services are performed with due care and skill or that goods are of merchantable quality) into some contracts (generally those for a value less than $40,000). These may entitle the customer to a remedy notwithstanding acceptance;
There may be separate liability under the Trade Practices Act for misleading or deceptive statements, acts or omissions or false statements that are made about the deliverables;
There may be separate liability for misrepresentations made about the deliverables and for negligent acts or omissions; and
The acceptance tests themselves must comply with the requirements of the agreement. For instance, if the ICT supplier was required to develop tests which “ensure” that the deliverable meets the requirements of the agreement, the customer might be able to require new tests to be developed and performed if the tests were incomplete, inappropriate or inaccurate and did not test the deliverable properly.

Next: Legal protection for software and documentation.

Consulting Business Models Acceptance Testing Legal protection for Software and Documentation Specialist Consulting Agreements Key Commercial Issues for Consulting & Development Tips to Make Money

Master Time & Materials Consulting Agreement Fixed Price Software Development Agreement Master Design Services Agreement Master Website Development Agreement Contractor Agreement (for sell through)

Cart (0 item)

Total: $0.00 AUD

Which contract is best for me?

Clicking on the link to one of the specific Online Contract Templates listed on this page will give you a description of when that particular contract should be used, and help you decide if it is the right document for your needs.