The presentation was connected with the review paper of Hasmik Atoyan recently accepted by JIPS (Journal d’Interaction Personne-Système). The paper is done in collaboration with Dr. Jean-Marc Robert (École Polytechnique de Montréal) and Dr. Jean-Rémi Duquet (Lockheed Martin Canada).

The paper is a part of Hasmik’s PhD research at École Polytechnique related to uncertainties in complex systems. Automation can expose and create several types of uncertainties, especially in dynamic complex environments.This subject has become a topic of interest for Hasmik during her work with new technologies being developed at Lockheed Martin Canada.

Poorly designed automation can increase uncertainties in the system. These uncertainties would create additional uncertainties in the operator’s mind and are likely to increase cognitive workload, reduce situation awareness, and affect human performance. The presentation of uncertain information to human operators in complex environments is a challenge.

Based on theoretical, empirical and analytical studies, Hasmik and her colleagues has developed human factors guidelines to help operators to cope with uncertainties and help designers to develop systems that could mitigate the impact of uncertainties on human performance.

The next step is to test operators’ reactions towards different formats of presentation of uncertain information as a function of their level of expertise, of the task being performed, of the application domain (e.g., defence, medicine, business, etc.). The next meeting on the progress of this research is scheduled for September, 2010.

This talk from Dan Ariely is a must see (presented at the TED conference, posted May 2009)

People have visual illusions and cognitive (decision-making) illusions.

In visual illusion we can easily demonstrate mistakes. For example, we can measure to verify if our visual comparison is right.

Cognitive (decision-making) illusion is not easy to see, it’s much-much harder to demonstrate people the mistakes (for example, our selection for an option is influenced by how the information is presented).

We have a feeling, we think we are in control and we are making the decision. It’s very hard even to accept the idea that we have an illusion of making the decision rather than actual decision.

When decision is complex and difficult, and we don’t know what to do, we might pick up what just is chosen for us.  When we don’t know our preferences we are susceptible to influences from external forces such as defaults or particular option presented to us.

When it comes to build the physical world we understand our limitations and we build around it. But when it comes to the mental world, for example when we design the health care, stock markets, we need to remember that we have cognitive limitations.

We like this talk because it highlights our philosophy: “We believe in human”. We believe in human capability and power, which can reach its height when the design (of job or product) takes into account human cognitive limitations.

For us, it means taking into account human cognitive limitations (human factors) in the process of the product design and remove. Asking people to react rapidly for opening elevator door or asking an employee to specify complex business requirements in the head office do not take into account human nature, human cognitive illusion.

From talk of Dan Ariely:

http://www.ted.com/talks/dan_ariely_asks_are_we_in_control_of_our_own_decisions.html

The Quebec Worker Compensation Board (CSST) provides services to 3 million workers across the province of Quebec in Canada. The information technology department of the CSST was looking to improve the quality of their help desk.

The Information Technology support service provides on-line technical support and maintains the CSST network infrastructure.  There was indications unreliability in the services offered. Even if 95% of trouble ticket were resolved within 24 hours in respect with the internal service agreement, there was high level of complains.

Having heard of the reputation Cognitive Group for Business Process optimization. The Quebec Worker Compensation Board CIO decided to hire Cognitive Group to optimize the CSST’s customer services.

Approach

The project began with a review of the support services operations.  This initial stage involved Field Study approach to analyze the current process. Customers and members of the IT department were observed and interviewed.

Cognitive Group approach “The Cognitive Approach” make employees at ease. It gathers more facts and subtle information that are not normally verbalized with common interview techniques.

“With the cognitive approach and tools we obtained a real understanding of the client perspective on the quality of services that we offer.” (Benoit LeFrançois)

Following the review, it was recommended to restructure certain tasks.  For example to allow first line support agents to respond to basic network management request, to remove duplication of trouble ticket, to reduce the number of printer vendors.

Results

Six month after the restructuring the Quebec Worker Compensation Board improved the quality of the service up to a 95% satisfaction level[1] , now respecting deadlines and budgets.

“In the past there was a lot of instability and unreliability in our services. …Now our client satisfaction level with our services is up to 95%.” (Benoit LeFrançois)

One of the changes is that every major update is now carefully planned before visiting the client.  The purpose of the upgrade is explained and an appropriate date for the implementation of the upgrade is agreed in advance with the customer.  For customers in remote areas, a support member stays on site to provide help for one or two days after each system upgrade.

“All these improvements at the initial stage of a project with a client have resulted in fewer follow-up problems and reduced implementation time.” (Benoit LeFrançois)

“In our new electronic services, these changes have resulted in a cost reduction of approximately $2 million per year.” (Benoit LeFrançois)

Solution impact

• The cost of departmental operations decreased by $2 million per year.
• A better synchronization and relation with customers was established.
• Customer satisfaction with the new services increased to a 95% satisfaction level.

[1]

Daniel Engelberg 31/12/99 12:00 AM

My interpretation; please validate.

• I expect the system to provide me the information and tools necessary for me to accomplish my task

• I want to be able to accomplish my task in the natural flown of my thinking:

1. Search for information

2. Look at the information

3. Make a decision

4. Execute an action and

5. Look at a confirmation of the action I just took.

• I don’t want to have to go through unnecessary steps or take unnecessary actions. I also don’t want to be annoyed by information that is not necessary for me to accomplish my goal or task.

• When I enter text, I expect that my most common choices are already available and selected by default. I expected that mechanical task such as calculation, repetition and detection will be performed by the system or tool. I don’t want to have to recall or take notes in order to know what I input.

• When I am executing a task, I don’t want to be interrupted or distractedI want to be able to instantly understand text, symbols and graphic. If something is unfamiliar, I want access to a detailed explanation.

• I expect to receive feedback following my actions. If I don’t receive this feedback immediately and have to wait, I want to know how long that wait is going to be.

• I want to be able to read text and discriminate images.

• I want controls (i.e., devices for input and navigation) and tools to be familiar and behave as expectedI don’t want to be careful to prevent errors. I expect the system to be allowing input errors (for example to accept various dates format). If I commit errors, I expect the system to detect errors for me, provide an error recovery mechanism and provide clear and explicit message that convey status and corrective action required.

• I want the system to be flexible by providing a rapid method or route for tasks that I am already familiar with. However, it should provide guidance for unfamiliar tasks.I want to be the one in charge of the interaction and feel that as a user I am in control. To accomplish this I need to see what is going on behind the scene and have the ability to override the system at any time. I am not comfortable with hidden processes.

• I want to know here I am, where I come from, be able to go back where I was or go back at the home of the system.

Moody’s gave the best rating to PCAA in 2007. Desjardins lost 591 million because of PCAA in the last quarter of 2008.  In 2009, Moody’s express warning on the Risk Management at Desjardins since  they do not manage properly complex financial products such as the PCAA.

It appears that overconfidence in Moody’s was one of Desjardins biggest mistake. The irony here is that Moody’s rate negatively institutions (Desjardins) that are in difficulty because they relied on Moody’s positive rating of certain financial product. It sound like Moody’s rates Moody’s negatively.