Assignment

Case Study Implementation

Designing Case Studies for Technology Evaluations
A case study is an evaluation that uses a real-life situation as the context for the study. For an emerging application of technology, you could develop prototypes or pilot implementations of the technology and then observe what happens (how are they used, how can they be attacked).
A case study is appropriate when: 
(a)    a prototype or design pilot (for products or systems) exists
(b)   there is an available setting in which the use and impact of the technology can be studied
For an example of a case study, read:
Spain, K. A. & Phipps, C. A. (2001). Data collection in the palm of your hand: A case study. International Journal of Human-Computer Interaction, 13(2), 231-243. Available from: http://ezproxy.umuc.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=bth&AN=5563208&site=ehost-live&scope=site

The case study designed by Spain and Phipps (2001) could be adapted to study the impact of movement and exercise upon the use of security features built into wearable computing devices ( e.g. smart watches, skin suits / medical monitors, etc. ). 
One important point to note: in a case study, the researchers do not always know what measures or measuring techniques they will end up using. They begin with a general strategy for collecting information to find answers to their research problem or questions. Then, as the case study proceeds, they record their observations and collect data; if necessary, they adjust their information collection strategy as the study proceeds. Afterwards, they analyze the collected observations and other information to extract useful and relevant measures.
If you choose this approach, develop a scenario or use case that involves your emerging technology and one of the following: 
1. protecting information used by or processed by the technology
2. protecting information systems which connect to or which incorporate your emerging technology
3. protecting infrastructures (includes information used in the infrastructure and information systems used in the infrastructure) in which your technology plays a part
If you wish, your scenario or use case can further restrict the focus of your case study to one or more of the five pillars of Information Assurance (CIA, authentication, nonrepudiation) and/or one or more of the five pillars of Information Security (protect, detect, correct / remediate, document, prevent).

Designing Delphi Method Studies for Technology Evaluations

A Delphi Method study is an evaluation that is based upon the opinions of subject matter experts (SMEs) as they respond to a series of questions. The goal of a Delphi study is to arrive at a consensus opinion or answer for each question or set of questions.
For an emerging application of technology, you could present the design characteristics of the technology to the SMEs and then ask these experts a series of evaluative questions about cybersecurity issues or impacts related to the proposed technology design.
A Delphi Method study is appropriate for an emerging technology evaluation when: 
(a)    design specifications exist for the technology
(b)   a series of evaluative questions can be developed in advance
(c)    subject matter experts can be identified and are available to participate
For an example of a Delphi Study read:
Marchau, V. A. & Van der Heijden, R. E. (2000). Introducing advanced electronic driver support systems: An exploration of market and technological uncertainties. Transportation Review, 20(4), 421-433. Available from http://ezproxy.umuc.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=a9h&AN=3818186&login.asp&site=ehost-live&scope=site

The study performed by Marchau and Van der Heijden (2000) could be repeated (replication study) with a slightly different focus – security features of driver support systems and/or autonomous vehicles currently under development in vehicle design labs or transportation systems R&D labs.

How Delphi Studies Work

The Delphi method uses a group of subject matter experts to address a series of questions and, through a round-robin question-response-question-response process, the experts develop a consensus opinion regarding their responses to the questions. Sometimes, the experts develop and then answer new questions. 
If you choose this approach, develop a scenario for the “experts” to consider that involves your emerging technology and one of the following: 
1. protecting information 
2. protecting information systems 
3. protecting infrastructures (includes information used in the infrastructure and information systems used in the infrastructure) 
If you wish, your scenario can further restrict their focus to one or more of the five pillars of Information Assurance (CIA, authentication, nonrepudiation) and/or one or more of the five pillars of Information Security (protect, detect, correct / remediate, document, prevent).
 
Next, explain where you will recruit your experts from (industry, professional groups, senior engineers in the firm, etc.) and how they will communicate (one day-long face-to-face meeting, email round-robin, skype or WebEx, etc.).  
Finish out with the remaining elements required in the assignment.

Designing a Pilot Implementation or Pilot Study for a Technology Evaluation

A pilot study or pilot implementation is an evaluation that uses a temporary situation as the context for the study. For an emerging application of technology, you could develop use cases or test scenarios and then study how the prototypes or pilot implementations of a technology perform under test conditions. A pilot implementation is more structured than a case study.
A pilot study is appropriate when: 
(a)    a prototype or design pilot (for products or systems) exists
(b)   there is an available test setting which can be manipulated or controlled (test environment)
(c)    the use and impact of the technology can be measured using predetermined metrics
For an example of a pilot study please read: 
Pal, R., Sengupta, A., & Bose, I. (2008). Role of pilot study in assessing viability of new technology projects: The case of RFID in parking operations. Communications of the Association for Information Systems, 23, 257-276.
Available from:http://aisel.aisnet.org/cgi/viewcontent.cgi?article=3375&context=cais  
If you choose this approach, develop a scenario or use case (“business process”) involves your emerging technology and one of the following: 
1. protecting information used by or processed by the technology
2. protecting information systems which connect to or which incorporate your emerging technology
3. protecting infrastructures (includes information used in the infrastructure and information systems used in the infrastructure) in which your technology plays a part
If you wish, your scenario or use case can further restrict the focus of your case study to one or more of the five pillars of Information Assurance (CIA, authentication, nonrepudiation) and/or one or more of the five pillars of Information Security (protect, detect, correct / remediate, document, prevent).
The pilot study design used by Pal, Sengupta, and Bose (2008) could be adapted to test emerging technologies such as wearable computers which have the potential to act as access control tokens.

A Second Pilot Study Example and Walkthrough

Scenario: 

Your company owns a 5 building business campus and occupies one of the five buildings. Your organization is also the “landlord” for a bunch of tenants that range from IT companies to law offices to medical clinics. Your CEO has decided that she wants your company to look for ways to reduce overall energy consumption so that your business campus can attain an energy efficiency certification for “green energy use” (like the Largo UMUC building has). The Chief Security Officer has concerns that bringing in untested technologies could result in compromised security. The CSO brings an article to the CEO’s next “energy reduction task force” meeting … read it here: https://krebsonsecurity.com/2014/02/email-attack-on-vendor-set-up-breach-at-target/ 

The CEO tells the CSO — Find me secure technologies. I want that certification and the recognition as a green energy business campus.
Your team has already been investigating the use of AMI to reduce energy consumption to the levels required for “green certification.”
The CSO asks you “what will it take to determine if these new meters are high risk or low risk when it comes to the five pillars of IA?”
[Note all of the above is just to set the stage and help you understand a possible context. You do not have to put this into your study plan … except, you would want to say “where” the pilot will be done, e.g. 5 building business campus where individual tenants each have their own electric meter]

Pilot Study Design

You could propose a pilot study that would swap out the regular meters (one per tenant) for smart meters for a period of 48 hours while the buildings are closed for a long holiday weekend. Tenants would have to agree to participate in this pilot implementation so, figure out an “incentive” to offer them (rent, rebate, ???). Tenants who participate would also have to shutdown all computer equipment that is not on an UPS and/or any other electrical/electronic equipment that might suffer if power is shutoff.
During the 48 hour test period, you run sniffers on the networks that are used by the AMI equipment and collect as much information as you can during the first 24 hours. During the second 24 hours, you run penetration tests to try and take over the smart meters or otherwise breach their security (if any). You’re looking for some sort of “active” attack that would allow you to not only steal data but also compromise the equipment (much as you’d try to compromise a PC or a server or a router during a penetration test).
At the end of the test period, you put the regular meters back on the buildings, collect all the “testing” notes and journals from the pen testers (which say who did what, how they did it, and when), wrap up your data in a nice digital package, and take the test meters, the testing notes/logs/journals, and the data back to your lab. Everything goes into a database (big data analytics) for analysis. At a minimum, you propose that the data be analyzed for (a) sensitive information that could be used by attackers (breach of confidentiality) and (b) evidence that the pen testers successfully compromised the AMI devices (compromising integrity of the devices and possibly loss of availability for electrical service).
Your timeline would need setup time (to install the meters — say half a day), the testing period (48 hours), the analysis period, say 4 weeks or so. At the end, you produce a report.
 

Using Quasi-Experimental Designs for Technology Evaluations

Quasi experimental means an experimental research design where hypotheses are developed from research questions and then tested. This design differs from a traditional experiment because it lacks both a control group and randomization of tests.
A quasi-experimental study is appropriate when: 
(a)    a prototype or design pilot (for products or systems) exists
(b)   there is an available setting in which the technology can be studied
(c)    specific, testable questions can be written 
(d)   specific measures can be taken during the experiment
(e)    data collected during the experiment can be used to accept or reject the null hypotheses.

Simple Experiment

For an example of a relatively simple technology evaluation experiment read:
Vink, P., Franz, M., Kamp, I. & Zenk, R. (2012). Three experiments to support the design of lightweight comfortable vehicle seats. Work, 41, 1466-1470.
Available from: https://ezproxy.umuc.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=bth&AN=71928720&login.asp&site=ehost-live&scope=site

Complex Experiment

For an example of a more complex technology evaluation experiment review the article and presentation from the researcher of the journal article below:
Coppolino, L., D’Antonio, S., & Romano, L. (2014). Exposing vulnerabilities in electric power: An experimental approach. International Journal of Critical Infrastructure Protection, 7, 51-60.
Available from:https://www-sciencedirect-com.ezproxy.umgc.edu/science/article/pii/S1874548214000043

Presentation summary available from: https://pdfs.semanticscholar.org/daa5/48e5d18068625018e661efce5ebd868a2d44.pdf

If you choose to use an experimental approach, you must develop research questions that address the impact of your emerging technology on one or more of the following: 
1. protecting information used by or processed by the technology
2. protecting information systems which connect to or which incorporate your emerging technology
3. protecting infrastructures (includes information used in the infrastructure and information systems used in the infrastructure) in which your technology plays a part
Here are some examples:
Research Question #1: What is the impact of [this technology] upon the confidentiality of information processed by [the system]?
Research Question #2: What is the impact of [this technology] upon the availability of information processed by [the system]?
Normally, the next step in designing the study is to write test hypotheses from the research questions. You are not required to write test hypotheses for your study plan for this course.