Directions: For this final, you’ll be asked to consider a range of verification tasks for a specific software project. You will not need to write any code! But you will need to clearly articulate what steps you would take for each portion, and for every question you must justify your answers (explain why that answer is correct, or explain your reasoning for a subjective choice) for full credit. In particular, if you are asked to make an open-ended choice, more credit will be awarded for sensible justification rather than a specific choice. Some of the questions have no single right choice, and the grade will be based entirely on your reasoning (i.e., making a reasonable choice for a good reason).

Directions:

For this final, you’ll be asked to consider a range of verification tasks for a specific software project. You will not need to write any code! But you will need to clearly articulate what steps you would take for each portion, and for every question you must justify your answers (explain why that answer is correct, or explain your reasoning for a subjective choice) for full credit. In particular, if you are asked to make an open-ended choice, more credit will be awarded for sensible justification rather than a specific choice. Some of the questions have no single right choice, and the grade will be based entirely on your reasoning (i.e., making a reasonable choice for a good reason).

Many of the questions are phrased in a very simple way that suggests a one-line answer: none of the answers should be a single short sentence. None of the questions can be answered in full by simply collecting terms of phrases from the lecture slides or textbook, though such a search might give you partial answers and point you in the right direction. As a result, this final is open book / notes / slides / lecture video.

For the purposes of this final, assume you (and some peers) are tasked with implementing the site for a new e-book retailer that also supports purchasing hardcopies on a print-on-demand basis. Thus, the site must include functionality for:

• Customers must be able to register accounts, save payment information for later, and access ebooks of previous purchases.
• Authors must be able to register as authors, and look at sales information for their books.
• Customers should be able to place orders for ebooks.
• Customers should be able to place orders for physical books, which are printed on-demand at the time of the order by a specialized machine.
• Customers should be able to enter new payment information for orders, or reuse previously-entered (non-expired) payment information if they opted to save it with your site.
• Customers should be able to choose their shipping address for physical orders.
• Administrators should be able to register (with validation of their role) and create, manage, and discontinue promotions (e.g., cyber Monday sales)
• As a US business, customers should be charged sales tax based on the address used for their order.

You do not need to be an expert in e-commerce, payment systems, taxes, banking systems, or anything else to do well on this; none of the questions require obscure specialized domain knowledge to answer.

Question1: (10pts) How would you review the initial requirements and specification? (Provide details.)

Question2: (20pts) You cannot test the payment system by way of the bank on a frequent basis, because it would be slow, and the bank does not want to be a heavily-used part of your testing infrastructure. (Banks do offer “test servers” to send fake payment requests to in order to ensure you can actually talk to a real server, but this is still too slow for unit tests.) How would you instead test payment processing logic, without relying on the bank? Similarly, how would you test print-on-demand logic, without wasting expensive ink and paper and requiring a human to read the results?

Question3: (20pts) Closed-box and open-box testing approaches are different, but complementary. Please describe the trade-offs, and give examples of things each approach is good for finding that the other approach is poorly suited to finding. (Recall that open-box testing includes code coverage, but also more than just code coverage.) Given these trade-offs, are there parts of the system that really should have a heavier emphasis on one type of testing rather than the other? Specifically, identify areas where closed-box testing would be especially important/appropriate, and places open-box testing would be especially important/appropriate.

Question4: (10pts) Customers and authors alike must both be able to log in and access relevant information. Customers can view their past orders, re-download previous ebook purchases, and change payment information. Authors should be able to log in, access statistics on how their books are selling. Store administrators can create and disable coupon codes. Granting or denying access to these pieces of functionality incorrectly would be a major issue. Give 6 tests you would write for checking user access to different functionality.

Question5: (10pts) For the first version of the software, usability is important, but you have a limited usability budget in both time and funds: you can pay 6 prospective users (few) for one hour each (relatively little time) to participate in usability studies. What kinds of studies do you pursue? What kind of users would you find for these? Justify your choices.

Question6: (5pts) What kinds of accessibility testing should you perform on the web interface? How (at a high level) might you implement some of these via something like Selenium, and are there other kinds you might need to check manually?

Question7: (10pts) What is equivalence partitioning (examples are insufficient, but can add clarity), and what are some examples of useful places to apply equivalence testing specifically that might arise in a project like this?

Question8: (10pts) What kinds of performance-related testing would you do for a system like this, and why?