The Delphi Method

One of the methods that was mentioned in the email that was sent was the Delphi Method. What is this Delphi Method?

From the readings this method is primarily used for forecasting and decision making. Like a lot of other research and technological innovations it began in the military. In 1946 a Project called RAND (short for Research and Development)

Definition: The Delphi Method is based on a structured process for collecting and distilling knowledge from a group of experts by means of a series of questionnaires interspersed with controlled opinion feedback.

Premise: Human judgment (Expert Knowledge) is acceptable in fields that have no yet developed scientific laws to make decisions.

A little Extra Information

The word Delphi is associated with Greek Mythology. Delphi was an Oracle, where the intermediaries would give forecast and advice to the gods. However, Helmer and Dalkey the founders of the method, would rather than the method not be associated with the Greek Oracle. But I guess that’s life names stick J

Advantages

- Experts make decisions

- Anonymity allows free formation of opinions that the social interaction of usual group discussion may inhibited

- Experts who geographically dispersed can deal systematically in complex task and problem solving.

- useful in answering one, specific, single-dimension question

How is it done?

In a Nutshell: a series of questionnaires are sent either by mail or via computerized systems, to a pre-selected group of experts. These questionnaires are designed to elicit and develop individual responses to the problems posed and to enable the experts to refine their views as the group’s work progresses in accordance with the assigned task. The group interaction is anonymous.

1. Formation of a team to undertake and monitor a Delphi on a given subject.
2. Selection of one or more panels to participate in the exercise. Customarily, the panelists are experts in the area to be investigated.
3. Development of the first round Delphi questionnaire
4. Testing the questionnaire for proper wording (e.g., ambiguities, vagueness)
5. Transmission of the first questionnaires to the panelists
6. Analysis of the first round responses
7. Preparation of the second round questionnaires (and possible testing)
8. Transmission of the second round questionnaires to the panelists
9. Analysis of the second round responses (Steps 7 to 9 are reiterated as long as desired or necessary to achieve stability in the results.)
10. Preparation of a report by the analysis team to present the conclusions of the exercise

Questions to ask before considering using the Delphi Method.

• What kind of group communication process is desirable in order to explore the problem at hand?
• Who are the people with expertise on the problem and where are they located?
• What are the alternative techniques available and what results can reasonably be expected from their application?

Disadvantages/critiques

- Unscientific

- Illusory expertise: some of the experts may be poor forecasters. The expert tends to be a specialist and thus views the forecast in a setting which is not the most appropriate one.

- The simplification urge: Experts tend to judge the future of events in isolation from other developments. A holistic view of future events where change has had a pervasive influence cannot be visualized easily. At this point cross-impact analysis is of some help.

- Format bias: it should be recognized that the format of the questionnaire may be unsuitable to some potential societal participants.

My Opinion: I am reading a book called Renaissance E-learning for another course and it has a section of quotes that were made against the realization of certain inventions that exist today. Most of the quotes were made by experts in the field. Example, “I have travelled the length and breadth of this country and talked with the best people, and I can assure you that data processing is a fad that won’t last out the year.” Editor in charge of business books for Prentice Hall. 1957

This goes to show that sometimes experts are not necessarily good forecasters. However, I like the fact that it’s anonymous and it brings together the opinions’ of experts from several geographical locations. It has merits but, as with most Data collection methods should not be used alone and should be used with caution.

I think even though it's used primarily for forecasting it can also be used to develop solutions to complex organizational problems.

http://www.iit.edu/~it/delphi.html

Questionnaire redesigned

http://www.freepatentsonline.com/7121830.html

This article describes an intervention to redesign questionnaires that measure human performance, primarily of those in leadership positions. The article offers some salient limitations to the current use of evaluation and frequency scales. The intervention/invention seems feasible to me but the last part is a bit statistical or more quantitative (so those among us who use this aspect of the left brain a bit more than me can review and tell me what you think :) .

Proposed arguments

• The author distinguishes between evaluation scales and frequency magnitude. The former judge how well the individual performs and the latter tell how often the individual displays or engages in a behavior.
• These scales often report two extreme findings the underdone and the overdone. He asserts that it easily identifies the underdone but assumes that high scores are good and people often make invalid inferences. “For example, a high rating of 4 (“Frequently, if not always”) in response to the item “Speaks up in a group” using the frequency/magnitude scale could mean one of two things: either the individual speaks up appropriately often or that the individual dominates group discussions. Therefore, a high score does not mean that the individual is an effective communicator and group leader, etc. Again, there is no provision for the overdo extreme.”
• These scales often do not provide the reasons why the individuals were given the ratings they received.
• They are lopsided as they do not assess complimentary attitudes and characteristics.

Solution

• Have a questionnaire that requires participants to rate along a continuum. Where the middle indicates balance, represented by a zero (0). The deficient end will be represented by increasingly negative (-) figures and the overdone end is represented by increasingly positive (+) figures.

← Too little (-1 or more)-The right amount (0)-Too much (+1 or mote) →

• Second the questionnaire will assess complimentary characteristics and attitudes but will do so implicitly and the questions for matching characteristics will not be placed beside each other. (This is where the statistics comes in)

Human? Google?

It's amazing the search results that come up when one types in human performance in a search engine. Well for me I was looking for methods/guidelines/heuristics that were used to collect data on human performance. Most of the search results were not directly related to human beings but were related to technologies that were designed to function as human beings.

Another set were about Psychology or scientifically related which, I did not mind as it's good to learn from other fields but then they started using some statistical data to explain findings and I just went to another source. Among the scientific topics were human enhancements.


Second, am I the only one that's late? Did you know that Google had a program called Analytics.

http://www.google.com/analytics/index.html

http://64.233.179.110/analytics/tour/index_en-US.html

The New Google Analytics

Google Analytics has been re-designed to help you learn even more about where your visitors come from and how they interact with your site.

Discover. Share. Act.

The new Google Analytics makes it easy to improve your results online. Write better ads, strengthen your marketing initiatives, and create higher-converting websites. Google Analytics is free to all advertisers, publishers, and site owners.

Skill Measurement Systems: Technology to the rescue

Everything boils down to what’s your goal or objective, even developing a competency model.

“As a consultant, one of my main roles is to help program decision-makers establish a strategic linkage between the competency modeling program and key business objectives,” says Maul.

http://www.brainbench.com/pdf/STG_CompModeling.pdf, Article Title: Skills Measurement for Effective Competency Modeling—Lessons from the Field.

The article discusses June Maul’s experience with competency model and highlights the benefits of using skill measurement systems to help in the process of creating competency models. To do it manually can be long and cumbersome. Technology to the rescue :)

She recommends the following in order to successfully implement a competency model:

• Planning—Link Program Implementation to Measurable Business Goals
• Establish Communications—Convey Clear Goals to Create Champions and Motivate Participants
• Create Incentives—Link Program Participation to Career Development

Human Assets as a competitive advantage

Eureka! Eureka! I have got it!! Like Archimedes I have made a discovery. O.k, it won’t change history but something just dawned on me. There has been a shift in the focus of businesses today. I have no research to prove it yet, but I believe that the fear that permeated at first is no longer there. I know you are wondering what I am talking about….O.k let’s cut the chase.

When the computer, the internet and other information technologies were introduced in the work place, some employees became fearful that they would be replaced by technology. The fear led to great resistance and the anti-technology syndrome. Managers saw the technology as a way to reduce the size of the workforce and save money. However, the focus has shifted from primarily reducing the size of the labor force with the use of technology to improving the performance of the workforce, with and without technology.

Businesses have recognized that in the competitive market it’s not who has the most high tech computer systems; it’s the company that has the greatest minds that will have the edge.

http://edweb.sdsu.edu/people/ARossett/pie/Interventions/career_1.htm : The article, Career and Competency Pathing: The Competency Modeling Approach is short. You should read it!!

The article defines competency and competency model quite well:

“Competencies are behaviors that encompass the knowledge, skills, and attributes required for successful performance.”

“Competency modeling is the activity of determining the specific competencies that are characteristic of high performance and success in a given job.”

According to Boulter, et al (1998), there are six stages involved in defining a competency model for a given job role. These stages are:

1. Performance criteria - Defining the criteria for superior performance in the role.
2. Criterion sample - Choosing a sample of people performing the role for data collection.
3. Data collection - Collecting sample data about behaviors that lead to success.
4. Data analysis - Developing hypotheses about the competencies of outstanding performers and how these competencies work together to produce desired results.
5. Validation - Validating the results of data collection and analysis.
6. Application - Applying the competency models in human resource activities, as needed.

Hewlett-Packard was noted as one of the companies that has been using competency modeling to improve performance and they have seen good results. It is perceived that the use of competency modeling will increase and that development in technology will enable more.

"The ways human resource activities are performed must change substantially to respond to business challenges. New technology applications will be the most critical enabler."