E-Commerce Application Testing

e-Commerce is a fast-growing area of software development. This workshop focuses on how to test Web-based e-Commerce applications. You will learn various testing techniques you can use to improve confidence in your e-Commerce systems. You will also learn how to develop scenario-based testing for e-Commerce applications and review some sample test plans
Topics Need to remember
  • Functional testing
  • Modeling
  • Usage scenario modeling
  • Security testing
  • Performance and scalability testing
  • Stress testing

Ten Key Principles of Effective E-Commerce Testing
Over the decades since Information Technology (IT) became a major factor in business life, problems and challenges such as those now faced by the e-commerce community have been met and solved. Key testing principles have emerged and these can be successfully applied to the e-commerce situation.

  • Principle 1. Testing is a risk management process. The most important lesson we have learned about software testing is that it is one of the best mechanisms we have for managing the risk to businesses of unsuccessful IT applications. Effective testing adopts a strategy that is tailored to the type of application or service being tested, the business value of the application or service, and the risks that would accompany its failure. The detailed planning of the testing and the design of the tests can then be conformed by the strategy into a business-focused activity that adds real business value and provides some objective assessment of risk at each stage of the development process. Plans should include measures of risk and value and incorporate testing and other quality-related activities that ensure development is properly focused on achieving maximum value with minimum risk. Real projects may not achieve everything that is planned, but the metrics will at least enable us to decide whether it would be wise to release an application for live use.

  • Principle 2. Know the value of the applications being tested. To manage risk effectively, we must know the business value of success as well as the cost of failure. The business community must be involved in setting values on which the risk assessment can be based and committed to delivering an agreed level of quality.

  • Principle 3. Set clear testing objectives and criteria for successful completion (including test coverage measures). When testing an e-commerce site, it would be very easy for the testing to degenerate into surfing, due to the ease of searching related sites or another totally unrelated site. This is why the test programmer must be properly planned, with test scripts giving precise instructions and expected results. There will also need to be some cross-referencing back to the requirements and objectives, so that some assessment can be made of how many of the requirements have been tested at any given time. Criteria for successful completion are based on delivering enough business value, testing enough of the requirements to be confident of the most important behavior of the site, and minimizing the risk of a significant failure. These criteria – which should be agreed with the business community - give us the critical evidence that we need in deciding readiness to make the site accessible to customers.

  • Principle 4. Create an effective test environment. It would be very expensive to create a completely representative test environment for e-commerce, given the variety of platforms and the use of the Internet as a communications medium. Cross-platform testing is, naturally, an important part of testing any multi-platform software application. In the case of e-commerce, the term ‘cross-platform’ must also extend to include ‘cross-browser’. In order to ensure that a site loads and functions properly from all supported platforms, as much stress and load testing as possible should be performed. As an absolute minimum, several people should be able to log into the site and access it concurrently, from a mixture of the browsers and platforms supported. The goal of stress and load testing, however, is to subject the site to representative usage levels. It would, therefore, be beneficial to use automated tools, such as Segue’s SilkPerformer or Mercury Interactive’s Loadrunner, for performance/load testing.

  • Principle 5. Test as early as possible in the development cycle. It is already well understood and accepted in the software engineering community that the earlier faults are detected, the cheaper the cost of rectification. In the case of an e-commerce site, a fault found after shipping will have been detected as a failure of the site by the marketplace, which is potentially as large as the number of Internet users. This has the added complication of loss of interest and possibly the loss of customer loyalty, as well as the immediate cost of fixing the fault. The fact that e-commerce development is rapid and often based on changing requirements makes early testing difficult, but testing strategies have been developed by the RAD community, and these can be mobilized for support. Perhaps the most important idea in RAD is the joint development team, allowing users to interact with the developers and validate product behavior continuously from the beginning of the development process. RAD utilities product prototypes, developed in a series of strictly controlled ‘time-boxes’ – fixed periods of time during which the prototype can be developed and tested – to ensure that product development does not drift from its original objectives. This style of web development makes testing an integral part of the development process and enhances risk management throughout the development cycle.

  • Principle 6. User Acceptance Testing (UAT). The client or ultimate owner of the e-commerce site should perform field testing and acceptance testing, with involvement from the provider where needed, at the end of the development process. Even if RAD is used with its continuous user testing approach, there are some attributes of an e-commerce site that will not be easy (or even possible, in some cases) to validate in this way. Some form of final testing that can address issues such as performance and security needs to be included as a final confirmation that the site will perform well with typical user interactions. Where RAD is not used, the scope of the provider’s internal testing coverage and user acceptance testing coverage should be defined early in the project development life cycle (in the Test Plan) and revisited as the project nears completion, to assure continued alignment of goals and responsibilities. UAT, however, should not be seen as a beta-testing activity, delegated to users in the field before formal release. E-commerce users are becoming increasingly intolerant of poor sites, and technical issues related to functionality, performance or reliability have been cited as primary reasons why customers have abandoned sites. Early exposure of users to sites with problems increases the probability that they will find the site unacceptable, even if developers continue to improve the site during beta testing.

  • Principle 7. Regression testing. Applications that change need regression testing to confirm that changes did not have unintended effects, so this must be a major feature of any e-commerce testing strategy. Web-based applications that reference external links need regular regression testing, even if their functionality does not change, because the environment is changing continuously. Wherever possible, regression testing should be automated, in order to minimize the impact on the test schedule.

  • Principle 8. Automate as much as possible. This is a risky principle because test automation is fraught with difficulties. It has been said that a fool with a tool is still a fool, and that the outcome of automating an unstable process is faster chaos, and both of these are true. Nevertheless, the chances of getting adequate testing done in the tight time scales for an e-commerce project and without automation are extremely slim. The key is to take testing processes sufficiently seriously that you document them and control them so that automation becomes a feasible option – then you select, purchase and install the tools. It will not be quick or cheap – but it might just avoid a very expensive failure.

  • Principle 9. Capture test incidents and use them to manage risk at release time. A test incident is any discrepancy between the expected and actual results of a test. Only some test incidents will relate to actual faults; some will be caused by incorrect test scripts, misunderstandings or deliberate changes to system functionality. All incidents found must be recorded via an incident management system (IMS), which can then be used to ascertain what faults are outstanding in the system and what the risks of release might be. Outstanding incidents can be one of the completion criteria that we apply, so the ability to track and evaluate the importance of incidents is crucial to the management of testing.

  • Principle 10. Manage change properly to avoid undoing all the testing effort. Things change quickly and often in an e-commerce development and management of change can be a bottleneck, but there is little point in testing one version of a software application and then shipping a different version; not only is the testing effort wasted, but the risk is not reduced either. Configuration Management tools, such as PVCS and ClearCase, can help to minimize the overheads of change management, but the discipline is the most important thing.

    E-commerce is both familiar and novel. Some of the technology is relatively novel, and the application of that technology to a complete business is certainly novel, but the problems of creating business processes to operate a business in a wholly new environment overshadow all of that novelty with some familiar and intractable problems. Paradoxically, it is in the more familiar areas of the technology that the most serious problems arise, because the emergence of e-commerce has placed new and challenging requirements on this relatively old technology that was designed for a quite different purpose.
    Testing is crucial to e-commerce because e-commerce sites are both business critical and highly visible to their users; any failure can be immediately expensive in terms of lost revenue and even more expensive in the longer term if disaffected users seek alternative sites. Yet the time pressures in the e-commerce world militate against the thorough testing usually associated with business criticality, so a new approach is needed to enable testing to be integrated into the development process and to ensure that testing does not present a significant time burden. The very familiarity of much of the technology means that tried and true mechanisms will either be suitable or can be modified to fit. Rapid Applications Development (RAD), in particular, suggests some promising approaches. Like most new ventures, though, e-commerce must find its own way and establish its own methods. In this paper we have suggested some testing principles that have stood the test of time and intermingled them with some lessons learned from similarly challenging development environments to give e-commerce testers a staring point for their journey of discovery.

  • Thanks & Regards ,

    Prashant Vadher | QA Engineer


    Post a Comment

    Design by Prashant Vadher