The Work Suite

O*NET Analysis by John F. Reeves, MA
Home | Companies/Products | Info | Products by Function | About Us | Users

O*NET versus DOT – You Have To Admit This is Getting Interesting

The latest news, March 2008 near the bottom of this screen, might be of interest.

Since it was first published in 1939, the Dictionary of Occupational Titles (DOT) was the world’s largest and most comprehensive reference for occupational information – but it was getting cumbersome and expensive to update. True to the 1990 mission of the Advisory Panel on the Dictionary of Occupational Titles ((APDOT), the Department of Labor finally released Version 1.0 of O*NET – the replacement to the DOT - in late 1998. You can go directly to the latest on-line version of O*NET at http://online.onetcenter.org.

If you are used to the DOT’s 80 or so variables on 12,762 occupations . . . brace yourself. O*NET is very different from the DOT even though it is still largely based on aggregates of DOT titles and descriptions. Many existing paradigms in the vocational sciences have been changed in O*NET – the nine levels of Specific Vocational Preparation requirements have been collapsed into five Job Zones, interest code definitions have changed, and physical ratings have changed, to name a few. O*NET emphasizes cognitive-oriented work skills to try to reflect today’s knowledge-based occupations, rather than the psychomotor, physical and functional behaviors and abilities that were used to define work in the past (and very conveniently for rehabilitation and ADA professionals today). Many of the currently 900 or so O*NET occupations are aggregations of 15 or more DOT occupations without regard to strength or other physical requirements. My opinion is that this aggregation approach is a major drawback for vocational evaluators.

In general, O*NET seems to be a good resource for general career exploration – although most counselors use Career Information Delivery Services (CIDS) that are more targeted to various age groups. I suspect that school guidance counselors would find it even more useful if local outlook and training programs could be added – but this information can currently be found at America’s Career Information Network, on the internet at www.acinet.org. Human Resource departments will probably continue to develop their own job descriptions from scratch and largely ignore the generalized O*NET job descriptions, much in the way they ignore the DOT. As far as using O*NET for curriculum development, I would say the task statements would be a good reference, but they are much too general. Although some references to other web sites are now included in Help, I could not find Help information that would be useful for using O*NET to determine compliance with the Americans with Disabilities Act (ADA). They have improved the ability to Browse the O*NET database on individual dimensions, e.g., you can easily see a list of O*NET occupational units that require Gross Body Equilibrium, but you cannot combine the search criteria (e.g., add also Dynamic Flexibility to the Browse criteria). Perhaps future Help menus will address how this can be used for ADA.

Using O*NET

It has to be mentioned that the term "O*NET" is not Google-friendly. Most search engines treat the asterisk in O*NET as a wild card, so if you Google O*NET, you will get over 3 million hits, but see very few references to the O*NET discussed here.

My first tour of O*NET at http://online.onetcenter.org/ started with the On-line Help button. It is both searchable and "context sensitive" – meaning that it will go to the Help section that addresses the part of the O*NET program you are in when you click on Help. A quick scan of topics and I felt assured that I could forge ahead and jump back into the help menu if I got lost.

Back in the main screen of the O*NET On-line: at the top I clicked on the button for Find Occupations, which gives you the ability to search on title. You can also search on code, skills job family and several crosswalks. Once you find the occupation and select it, the main screen allows you to view the occupation’s tasks, tools, technology, knowledge, skills, abilities, work activities, work context, job zone, interests, work styles, work values, related occupations, wages and employment.

Having finally found one of my own occupational titles (sort of) under the Search Menu – Educational, Vocational, and School Counselors – I decided to review the Summary Report. I wondered how these new requirements and characteristics would be assessed in individuals. I do not recall seeing many of the physical and cognitive abilities on the functional capacity assessment forms that professionals use. Cognitive abilities like "Problem Sensitivity - Being aware of others' reactions and understanding why they react as they do." might be a bit tricky to nail down. Based on a few Google searches, the only way to determine this is with a self assessment. I then went to the Related Occupations button and saw the "similar or transferable skills" occupations that I "may" be able to perform according to O*NET. Unfortunately, all on my list would require at least two years of additional education and certification even though I have a masters in Human Resource Development and have worked as a career counselor for over 20 years. The DOT would yield at least ten "transferable skills" occupations that I could perform tomorrow. My suspicions were confirmed that O*NET needed some close scrutiny before using it in the real world.

During several years of reviewing O*NET, the main question that I sought to answer was: "When should I abandon my DOT-based systems for assessment and occupational information?" My answer today would have to be, "Not yet, not for quite awhile, and, unless major changes are made – not ever." Here are some practical reasons:

First, O*NET 98 was a prototype, as is it's latest version. The initial version, O*NET 98, was based on the Occupational Employment Statistics (OES) data structure, but the D.O.L. said this structure would change to the revised Standard Occupational Classification (S.O.C.) system by the end of 1999. It did indeed change in mid-2000. At that point the number of occupations described in O*NET went from the 1100+ of the OES structure to the 974 occupations of the revised SOC structure. But it is still a work in progress. You would expect that the initial occupational analysts’ ratings in O*NET would be changed as the incumbent worker ratings are collected and added over the next several years. Even more interesting, the actual O*NET rating items were in for another change (from 483 original descriptor variables to 400 in the current version). So O*NET 10.0, as latest release is referred to, is in some ways a completely different animal than the database structure they originally collected data for. This has proven to be a difficult situation for people who are currently developing computer applications, assessment tools, employment matching methodology and other programs.

Second, as of July 2002, it appeared there are still no computerized assessment tools available for O*NET, and the objective "Ability Profiler" appeared to be delayed indefinitely. It is finally available, but uses 1970’s "bubble sheet" scoring technology, requiring an expensive scanner. The Technical Support Staff at the National Center for O*NET Development answered my query in September of 1999 with the following:

"In answer to your question about assessment instruments developed for the O*NET system, there are no products currently available that measure all of the O*NET skills.

However, the Ability Profiler, which will replace the General Aptitude Test Battery (GATB), is in final stages of completion. The GATB, which you may be familiar with, was developed by the US Department of Labor and measures nine aptitudes that are important for most jobs. It has been in use for many years. Unfortunately, the Ability Profiler is not a self-administered instrument, but its results will relate to O*NET occupations. Other assessment tools nearing completion are a new interest inventory, the Interest Profiler, and a measure of work values/needs, the Work Importance Profiler. These two instruments are self-administered. Accompanying manuals, user guides, and documentation for all three instruments are in final stages of completion. We anticipate their release to be sometime in the next year. All of these assessment tools will be part of the O*NET Career Exploration Tools, along with the Workplace Literacy Tests."

The O*NET Interest Profiler and the Work Values assessments are available, but these are not exactly breakthroughs in vocational assessment. Also, the D.O.L. has agreed that major revisions to the data structure are needed to serve the needs people who do disability determination and vocational rehabilitation. According to a memo from a member of the Interorganizational O*NET Task Force (IOTF) whose mission is to address O*NET shortcomings:

" The Department of Labor now agrees the O*NET is not suitable, or appropriate for use by professionals who need an occupation-specific database to assist in the delivery of their services. These professionals include occupational and physical therapists, expert witnesses, vocational rehabilitation specialists, attorneys, LTD and industrial indemnity insurers, state vocational rehabilitation agencies, the Social Security Administration, and other government and non-government organizations who must rely on occupation-specific data for their use. In a meeting consisting of officials representing the US Department of Labor (DOL), Social Security Administration (SSA) and the Interorganizational O*NET Task Force (IOTF) which was held in Baltimore on 10/30-31/01, it was agreed that an occupation-specific database should be developed to meet the needs of those professionals who cannot use O*NET in their work. In essence, they need an acceptable replacement for the Dictionary of Occupational Titles (DOT). A brief development plan was prepared and will be presented to SSA and DOL decision makers for final discussions and general agreement.

While not yet officially sanctioned, DOL, SSA and IOTF are to develop an occupation-specific database to replace the DOT. In the meantime, the life of the DOT has been extended to some degree. "

So the Department of Labor and related agencies are either back to the drawing board or O*NET will be in for yet another major re-design. Incorporating new fixes and changes into O*NET will not be easy. From an earlier life as a database consultant, I know that databases are like a room full of marbles. You move one marble and, perhaps unseen to you, a few marbles on the other side of the room shift slightly. Because of this, debugging any computer system is maddening (you might recall Bill Gates crashing the pre-release Windows 98 at a national TV press conference). Future significant changes to O*NET are sure to move a lot of the internal database marbles, and probably a number of marbles in the assessment tools and employment processes that are supposed to interface with O*NET.

Implications for Professionals Who Make Vocational Recommendations

My main questions about O*NET for career counselors and vocational evaluators: If you can’t measure objectively, how can you recommend? And if the measurement is not validated, how can you defend it? Working out these issues will take time, regardless of the resources that are available. And, to date, the O*NET people seem to be a bit cavalier in their treatment of existing vocational practices. I personally wish they had kept the 12 Guide for Occupational Experience (GOE) interest categories, which the Department of Labor developed and probably owned outright. Instead, they went with the more popular Holland model and changed it.

Occupational Interest in the O*NET structure is measured/assigned by R-I-A-S-E-C: Realistic/Investigative/Artistic/Social/Enterprising/Conventional. Most of us would assume that this reflects Drs. Holland and Gottfredson’s extensive research on occupational interests and preferred people environments. Indeed, the Dictionary of Holland Occupational Codes is perhaps the best-researched and most-widely-used model of occupational choice that we have. Anyone who has worked with Holland codes will quickly spot that O*NET’s interest codes are different. This brings up the question of, "Why?". It turns out that the analysts at O*NET came up with a different, alternative method to derive their codes using R-I-A-S-E-C. Since the interest codes assigned to occupations are different than the Holland codes, I wondered if this meant that the results of a Holland Code-based assessment, such as the Self-Directed Search or CAI, would be wrong when looking up occupations in O*NET. When asked about this, the Technical Support Staff at the National Center for O*NET Development replied:

"In answer to your questions about the interest information contained within O*NET, no, Holland is not wrong. The information within O*NET 98 is not Holland codes (Gottfredson & Holland, 1996). Occupational Interest Profiles (OIPs) are assigned to each occupation within O*NET. The OIPs are an independently developed vocational interest database that is compatible with Holland's (1985) R-I-A-S-E-C model. In
addition to its own definition of the six interest areas, the OIPs present a six-letter numerical profile of each interest area, along with an example set of high-point codes."

To be fair, since the O*NET analysts have changed the actual definitions of the R-I-A-S-E-C interest areas, their coding might be valid. But I find it inconvenient to change such a foundational paradigm as the Holland model, and I would not say the two are "compatible". Several examples of the differences in interest coding:

O*NET Code O*NET Occupational Unit Title 

171011 Architects, Except Landscape & Naval O*NET OIP Code: ARI Holland Code IAE

132021 Assessors O*NET OIP Code: CE Holland Code IER

132041 Credit Analysts O*NET OIP Code: CE Holland Code ESC

292021 Dental Hygienists O*NET OIP Code: SCR Holland Code SAI

518021 Stationary Engineers O*NET OIP Code: RC Holland Code REI

Does O*NET do Transferable Skills Analysis?

I also train vocational evaluators in my other role as a vocational software vendor. Several clients have asked about going into court as expert witnesses using O*NET data (which one software system I represent has now included side-by-side with DOT descriptions). While some may feel it is getting a bit hard to expect DOT occupational data from 1991 or before to be viewed as the best source – the O*NET Occupational Units are basically an aggregation of the old DOT task statements, so how is this better? The DOT still has more to offer for Transferable Skills Analysis (TSA) - namely Work Fields (WF); Materials, Products, Subject Matter and Services (MPSMS); and Specific Vocational Preparation (SVP). The DOT also offers more than ten times the number of occupations to choose from to represent a person’s work history and career options.

For performing TSA, O*NET offers the Related Occupations Matrix (ROM). After looking at the ROM for several occupations, my question was, "How are they related?". The Help reference under Related Occupations gives this explanation: "The lists of related occupations were empirically derived using a mathematical algorithm comparing the selected occupation to all other O*NET occupations. All the O*NET Knowledge, Skills, Abilities, Work Context and Generalized Work Activities variables were used to make these comparisons. The lists of related occupations were further refined through a multi-stage sensitivity screening process."

Perhaps O*NET’s empirical method for determining transferable skills occupations is true in theory, but how about in practice? No other answer as to how O*NET performs a TSA, or whether it will change in the next release, has come forth that I know of, but they did e-mail a colleague with:

"The Related Occupations Matrix (ROM) in the first version of the Viewer was designed to expedite the return of displaced workers to the workforce. The ROM . . . produces up to ten related occupations in rank order (the top occupation on the list is the closest match). Displaced workers would presumably look first for local area vacancies in the first occupation on the list, then move down from there."

The O*NET people are also saying the ROM can be used for "person-to-job" referral purposes in state employment security agencies. Looking at the "hands-on" world of work where the likelihood of worker injury (and lay-off) is high, let’s examine the ROM for Operating Engineer (O*NET Occupational Unit Code 47 2073.02). The "Transferable Skills" occupations listed in the Operating Engineer’s ROM up until 2004 were:

  1. Motorboat Mechanics
  2. Mobile Heavy Equipment Mechanics, Except Engines
  3. Small Engine Mechanics
  4. Air Hammer Operators
  5. Grinding and Polishing Workers, Hand
  6. Woodworking Machine Setter and Set-Up Operators, Except Sawing
  7. Roustabouts
  8. Construction Workers, Except Trade

Since the above criticism was easily found on the web for many years, it is not surprising that Motorboat Mechanic is no longer listed, although Small Engine Mechanic is still listed. With just a few jumps around in O*NET, I quickly found another puzzle: for Railroad Conductors and Yardmasters, the number one Related Occupation on the list is Chefs and Head Cooks.

Okay, I tell my vocational expert client, you are in court, and you have just used O*NET to recommend that an injured Operating Engineer could get a job as a Mechanic. Imagine the look of victory on the cross-examining attorney’s face as he says, "Would a reasonable and prudent person recommend that my client, who was not allowed to do anything more than check the oil on the equipment he operated, if even that, can now find a job working on high-performance engines?" Or, even in the updated O*NET, would a restaurant really hire a railroad Conductor or Yardmaster as a Chef? Are judges swayed by the announcement from the D.O.L. that O*NET completely replaces the DOT? There are years of case law history and perhaps billions of dollars of disability settlements that refer at least in part to the DOT. At this level of the vocational evaluation profession, it would seem the DOT is going to be around for a while.

The Social Security Administration came to this same conclusion, as shown in their June 30, 1999, notice sent out to field personnel regarding O*NET:

" . . . The organization of the data in the prototype version does not parallel SSA’s disability determination process, as does the DOT. For example, the O*NET measures for exertional requirements of work, such as strength, are defined and rated differently than those in the DOT. In addition, O*NET contains data on only about 1100 occupational units, compared to over 12,000 job titles in the DOT. Each O*NET unit is an aggregation of data for a number of DOT job titles that have been grouped in such a way that they are not readily useable in SSA’s disability decisions.

At this time, the prototype version of O*NET does not provide any advantage over the DOT (or other existing vocational resources). Therefore, DDS’s and other SSA disability adjudicators and reviewers should not use O*NET when making disability decisions. . . ."

What the front-line Employment Services workers will do with the Related Occupations Matrix is anyone’s guess, but I suspect that Job Developers who have worked hard to get a Chef opening listed in the state employment system will have some harsh feedback if Yardmasters are sent to the interview.

Preliminary Conclusion

From the outset many people saw some significant barriers to O*NET entirely replacing the DOT. Almost ten years later, questions still remain. Are 974 occupational units enough? How many users have taken the time to understand 400 occupational variables? How many evaluators will have to retool as cognitive psychologists to understand the new skills model? Have new functional capacity assessments been developed to address the new physical descriptors in O*NET? How many textbooks on vocational counseling and courses on vocational evaluation have to be revised? Are the millions of annual assessments currently using Holland codes going to have to change? And these are only the marbles that I can see being moved. The people at the National Center for O*NET Development point out that they are following their directive, and that we will eventually get over it:

"In answer to your question about O*NET replacing the DOT, it was DOL's Advisory Panel on the Dictionary of Occupational Titles (APDOT) that recommended the direction that DOL's occupational information database would take. There will be a period of transition as users adapt. We apologize for any inconvenience that this may create. However, it is felt that, for the most part, O*NET is an improvement over the DOT."

Perhaps all of these issues will eventually be addressed, as the DOL assures us. There are some good ideas in O*NET, but I can’t help thinking of the fate of the metric system, and the metric system is a much better-conceived system than O*NET.

In 1999 I stated that I would not be surprised if a consortium of major DOT stake-holders seriously considered revising the old DOT on their own (the SSA and several dozen large insurance companies came to mind). The DOT isn’t perfect, but it does some things very well. Some of us guessed in 1999 that the existing DOT would be around in several sectors for another five years at least, and eight years later this seems to be the case.

Recent Developments

Sure enough, in July of 2002, Robert E. Robertson, Director of Education, Workforce, and Income Security Issues, stated before the Subcommittee on Social Security, Committee on Ways and Means, House of Representatives: "Although Labor has been working on a replacement for the DOT called the Occupational Information Network (O*NET) since 1993, Labor and SSA officials recognize that O*NET cannot be used in its current form in the DI and SSI disability determination process. The O*NET, for example, does not contain SSA-needed information on the amount of lifting or mental demands associated with particular jobs. The agencies have discussed ways that O*NET might be modified or supplemental information collected to meet SSA’s needs, but no definitive solution has been identified. Absent such changes to the O*NET, SSA officials have indicated that an entirely new occupational database could be needed to meet SSA’s needs, but such an effort could take many years to develop, validate, and implement. Meanwhile, as new jobs and job requirements evolve in the national economy, SSA’s reliance upon an outdated database further distances the agency from the current market place."

In June of 2003, the Social Security Administration published two more documents on the internet. These documents reflected continuing discussions amongst SSA, the US Department of Labor and the Interorganizational O*NET Task Force (IOTF).

The first publication sought sources capable of providing SSA with a revised and updated version of the Dictionary of Occupational Titles Revised 4th Edition (DOT) and its companion publication, Selected Characteristics of Occupations Defined in the Revised Dictionary of Occupational Titles (SCO). Internet searches do not show any action on this idea as of March 2007.

The second SSA information request sought sources capable of developing an occupational database for use in SSA’s disability programs which encompass the disability evaluation process and SSA’s work opportunity and employment support initiatives. This would be a long-term solution that would benefit everyone in the vocational evaluation field. Unfortunately, no recent news on this effort can be found.

Although no longer listed at the International Association of Rehabilitation Professionals’ web site, http://www.rehabpro.org, it was previously reported that:

"SSA and DOL have been working, both independently and together, to prepare for the transition from DOT to an updated occupationally specific data. Both agencies are preparing a joint white paper that outlines recent and planned SSA and DOL efforts to support SSA’s development of the type of data required for disability programs and vocational rehabilitation services. This document will serve as a basis for executive-level agency discussions, leading to more formal collaborative arrangements, such as an interagency workgroup established under a Memorandum of Understanding. To address the changes anticipated by the eventual obsolescence of the DOT, SSA needs to work with DOL on strategies for SSA’s development of additional occupational data while simultaneously investigating options for updating its medical-vocational policies as well."

According to one participant in the Interorganizational O*NET Task Force, IOTF has done nothing further for the past several years. Martin Gerry (Deputy Commissioner for Disability) apparently put a hold on the DOT update that the policy people wanted, and had an RFP ready for. There is now a new Commissioner, Michael Astrue. Not surprisingly, SSA apparently has no discretionary dollars because of war in Iraq, so the need for an occupational database might be a unaddressed until after the next presidential election.

?

Interesting News:

  • November 3, 2002:  Several Universities conduct a comprehensive review of O*NET - and other systems - showing the inherent flaws in the O*NET system for working with people with disabilties.  Funded by the Social Security Administration and Disability Research Institute.   The best way to read this report is by cutting SSA valid OASYS careerscope and pasting it at www.Google.com and then clicking Search.   One of the first search result should be:  dri.uiuc.edu/research/p02-06c/final_report_p02-06c.doc  This is a huge document so, unless you have a lot of hard drive and fast internet,  you might go with the View as HTML option.  That way you only see reference to your key words.  (If you read this entire document you will perhaps know more about the history and scope of matching disabled people to jobs than many Professors and State Directors of VR.)
  • June 4, 2003: The Social Security Administration was soliciting bids to create a new occupational database that uses DOT variables.  As of November 2005 you could still find the reference at http://www.eps.gov/servlet/Documents/R/673663  As of January 2006 the entire www.eps.gov link no longer works.
  • 2004 Society of Industrial and Organizational Psychologists Conference (SIOP) Conference Presentation.  If you need to point out the technical flaws of O*Net from another profession's view, read this 11-page presentation: FJA Strategies for Addressing O*NET Limitations in a Post-DOT Environment by Sidney A. Fine of Sidney A. Fine Associates,  Robert J. Harvey of Virginia Tech and Steven F. Cronshaw, University of Guelph. (FJA stands for Functional Job Analysis.)  Read esp. Harvey's comments at http://www.pstc.com/documents/SIOP2004.Fine.Harvey.Cronshaw.in.Fine.symposium.pdf
  • Ongoing O*NET Updates at International Association of Rehabilitation Professional (IARP’s) web site at http://www.rehabpro.org/onetupdates_frameset.html
  • October 25th, 2005Many have noticed the apparent flaws in the O*Net Occupational Interest Profiler, but here is an in-depth look.  Most of us with some training in career guidance know Dr. Lawrence K. Jones has been a significant voice in the field for several decades.  See what he says about Harmful Career "Assessments" on the Internet at his excellent site: http://www.careerkey.org/asp/professional_resources/harmful_assessments.asp
  • March, 2008  Social Security Administration is planning to ressurect DOT.    Excerpt:  " . . . SSA defines sedentary, light and medium strength work in the same way they are defined in the DOT/SCO. However, DOL has not formally updated the DOT since 1991 and has no plans to do so. Instead, DOL has developed an entirely new system, the Occupational Information Network (O*NET). SSA has determined through contracted and other research that the Agency cannot use O*NET, or occupational information derived in part from O*NET, in SSA’s disability evaluation process."
    • Hopefully more to come on SSA project.

And finally, if you receive funding directly or indirectly from the  U.S. Department of Labor - it is perhaps better to receive their funding than to criticize their occupational database system.  This web page is to inform you about my opinions and the results of my investigation about O*NET - we will see how long O*NET will stand the test of time - in the meantime you should continue to work wisely!  Most of the products represented elsewhere in this site - especially CareerScope and OASYS - will report O*NET codes and occupations, even though the underlying processes are not based on O*NET methodology.