Recruitment & Reskilling Strategy
,
Training & Security Leadership
Proponents Favor Performance Tests Over Certs
Facing a huge gap between the cybersecurity workforce it needs, and the personnel it can recruit or train, the U.S. Department of Defense is for the first time piloting new skills-based assessments for its cyber hiring as an alternative to checking paper qualifications. Many certificates, officials say, don’t reflect the skills their cyber teams need in the real world.
See Also: Shortfall in Cyber Skills Leads to Organizational Risk
The assessments, two of which were on display for visitors to try out at the AFCEA Cyber Workforce Summit this week, are designed to realistically replicate true-life operational situations and test the skills needed to respond, said Matt Isnor, the division chief for cyber workforce development in the Defense Department’s CIO’s office. They’re being piloted with current staff, military and civilian.
Once they’re completed and validated, Isnor said, they will be used – along with existing assessment tools that measure non-technical skills, knowledge screening instruments and structured interviews – to assess and select candidates for hiring.
Isnor, who joked that he took one of the assessments and “didn’t do very well,” told Information Security Management Group that it is vital to go beyond multiple choice questions and paper certificates.
“We have a huge, huge gap inside of the department,” Isnor said. Workforce and recruitment issues are “our number one priority,” he added, “the biggest thing that we have to go after. Trying to find the best and brightest out there has always been an issue. This is one of the mechanisms to solve that solution.”
Last year, officials said that 20,000 military and civilian cyber jobs at the department went unfilled, a vacancy rate of just under 10 percent. Filling that gap, Isnor said, means expanding the pool of potential hires beyond the holders of advanced degrees and highly qualified professionals with many years of prior experience.
“This is the starting point,” said Isnor, “We really have to be able to identify these critical skills.”
The two assessments on display at the summit were 30-minute initial tests, conducted at a keyboard and hosted in the cloud. They were workshopped between 40 and 50 subject matter experts, Isnor said, hiring managers who explained the processes they currently use to try and validate an individual candidate’s skills.
One assessment is designed to measure a candidate’s ability to analyze malware. In the test, the candidate has on their desktop a forensic copy of a thumb drive found after a cyber incident, along with various reports about it compiled using automated security tools. In the test, the candidate has to answer a series of structured questions about the drive and what’s on it, using the same information they would need to prepare a report for management following a real cyber incident.
In the second assessment, which measures skill at network data analysis, the candidate must analyze and answer questions about data captures, designed to mimic the real log dumps analysts scour for clues about an incident’s origin.
The assessments are designed to map to the skills required for two of the 70-plus cyber work roles at the department: Cyber defense analyst, and cyber defense incident responder. Together, Isnor told the summit Thursday, vacancies in these roles make up almost half of the unfilled positions in the DoD cyber workforce.
The short assessments are designed to provide an initial screening. A longer, more comprehensive 90-minute test will provide more detailed assessments about the specific skills and knowledge required for each job.
The assessments were currently being validated, one contractor working on the program said. He asked for anonymity as he was not authorized to speak to the press. Groups of qualified cyber analysts and control groups of people without cyber skills were taking the tests
“We want the first group to all pass,” because they were qualified and had the requisite skills, the contractor said. “If the control group passes as well, then we would clearly have a problem.”
In the fall, Isnor said, once the piloting and validation was complete, the assessments will be rolled out for real. He said the aim was to have them used for 10 percent of hires for vacant roles by the end of FY2027, in 18 months time.