Seven Questions to Shape Your Search for a Valid Workplace Assessment

Our clients and potential clients often ask what sets Emergenetics apart from other tools and what makes our approach so reliable and effective to people and organisation development. While there are several elements that set Emergenetics apart, one of the most notable I like to share with our prospects is its scientific validity.

Many of today’s widely used assessments are derivatives of the original psychometric tools that were created in the 1940s. These assessments continue to be utilised even though scientific knowledge, validity criteria and psychometric data analysis have evolved considerably in the last 75 years.

Before you put your organisation’s trust and finances into an assessment, there are some important questions you should ask, to make an informed decision regarding an assessment’s accuracy and reliability.

#1: When was the research conducted?

Take great care before using psychometric assessments that predate the brain science that has exploded since the mid-1980s. I personally advise to use particular caution if they are derivatives of Carl Jung’s work, which many experts now question. The truth is that some organisations and practitioners still utilise older tools out of comfort or habit.

We also now know that individuals should not be negatively labelled by an assessment, which tends to happen with these older tools. The more effective assessments were designed to celebrate one’s strengths, which is a core tenet of Emergenetics.

#2: What was the size and makeup of the population that was used to develop the assessment?

For some tools, less than 1,000 people were surveyed to form the original hypothesis. Some well-known assessments had less than 25. For other tools, the founding hypothesis was simply created to make the tool marketable.

In addition, the composition of the original population used to create the assessment may not reflect the audience for which the tool is being used today. Some assessments were conceived by observing family members or specific groups that do not represent the general population. If you are planning to use an assessment for your business, it is important to ensure that the tool you utilise was created using a survey population of working adults.

#3: How often are the assessment’s norms recalculated?

Global population norms evolve over time given progress in education, differing attitudes towards work and other factors. Many tools are decades old, or more, and not all have been regularly re-validated as the characteristics of the working population have evolved.

Social attitudes, educational approaches and technology will all affect how we think and behave and should be accounted for in the norm groups against which each individual is compared when they take an assessment.

At Emergenetics, we are proud to say we globally re-norm the results of the Emergenetics Profile every two years.

#4: Is there a Technical Report on statistical validity available (for free)?

Many of the questions posed in this blog post will be answered by a Technical Report, which should exist (and be free) for all credible tools.

It should raise serious questions about the validity of the assessment if a Technical Report does not exist, or if it is not freely available upon request.

#5: How reliable are the questions in the assessment’s questionnaire?

It is important that the questions used in the assessment have been refined for accuracy and consistency. You should be aware that some question sets may fail to meet minimal professional test development standards, have unsupported theories, poor test-retest reliability and/or weak validity.

For those who enjoy data analysis, let’s get technical here. And, for those who do not, I hope you’ll bear with me for a moment as we refer to the term “Inter-Item Reliability.” This measurement reflects how effective the assessment’s questions are in reliably and validly isolating the attributes the tool is trying to identify.

For reputable psychometric assessments, Cronbach’s Coefficient Alpha (which is used to measure Inter-Item Reliability) falls within the range of 0.70 to 0.85. This Coefficient Alpha is a useful data point to request as one test of the validity of a tool.

#6: Are the questions and reports straightforward and written in plain language?

The question sets for some psychometric assessments can be difficult to understand, leaving individuals confused when completing the questionnaire. In addition, the language and phrasing of the reports that accompany some tools can be convoluted. Ideally, the assessment’s reports should be straightforward and memorable if the output is to be used effectively by individuals and organisations.

There are simple checks you can make to address this question. When analysing a tool, consider if there are words that you do not immediately understand in the assessment or accompanying reports. If so, how can your employees be expected to understand the language? A further consideration about the reporting language is whether it is expressed in positive terms.

There is one final aspect to this question: do you want a very prescriptive report from your assessment, or one that provides a more helpful foundation for a trainer, coach, manager or individual to build upon, without too many assumptions being made? No tool can ever be fully cognisant of an individual perspective or context, so those that provide a constructive foundation for participants are often more useful and applicable.

#7: Does the tool offer ways to strengthen learning and impact?

As with any learning process, psychometric assessments have a more lasting impact on performance or behaviour when there is repeated exposure to the tool.

Continued usage of results is a challenge for most assessment providers. They often leave responsibility for ongoing support to trained practitioners who do not always have the time or resources to facilitate extended learning. At Emergenetics, every profile holder is given access to our free mobile app, which provides the opportunity for every learner to continue their engagement with their results multiple times every day.

Psychometric profile assessments, and the training that is provided alongside them, can represent a considerable and potentially powerful investment for an organisation.

To maximise impact, it is vital that your selection process for the tools used in learning and development and in recruitment is as rigorous as all other procurement activities.

By using the questions provided above in your selection process, you can begin to understand the validity of each tool you evaluate and take a disciplined approach when selecting the assessment that is best for your organisation.

If you would like to learn more about the reliability and validity of the Emergenetics Profile and how our team can support your organization’s goals, please fill out the form below.

 Print This Post