• Home
  • Standards
  • To Your Health: Are regulators pleased if everyone passes a test?
  • To Your...

John Kleeman Headshot

Posted by John Kleeman

We have all read about the huge regulator fines imposed  in the financial services industry, but regulatory activity in the health care industry is also rising rapidly. It seems to me that in this and other industries, there is a sea change: regulators are holding companies more to account and checking much more carefully that they are following the rules.
A stunning example of this is shown in the graph below from the US FDA showing the rise in the formal warning letters it has issued across industries in recent years – from 471 in fiscal year 2007 to 4,882 in fiscal year 2012.

FDA Warning Letters rising fiscal years 2007-2012

I have not read all these thousands of letters (!), but here are a few interesting excerpts relating to failing to train and assess well enough. You can see all warning letters on the FDA website:
From a letter to a medical devices company in 2012

“Failure to document that all personnel are trained to adequately perform their assigned responsibilities … Specifically, no documentation was provided which demonstrated that manufacturing technicians or customer service personnel received training for their assigned duties.”

From a letter to a contract pharmaceutical laboratory in 2012:

“your laboratory should establish a procedure for requiring, documenting, and periodically assessing all training”

From a letter to a pharmaceutical manufacturer in 2012:

“Your employees were not adequately trained in CGMPs as evidenced by the deficiencies listed in this letter. In your response to this letter, provide a plan to develop an ongoing and robust CGMP training program for your personnel, including an explanation of how you will assess training effectiveness.“

And from an older letter to a US blood bank:

There is no evidence an annual written test has been given to all laboratory personnel as required by the “Training Program” procedure for the production laboratory.

These are interesting evidence that regulators expect assessments or other evidence of training. I was prompted to write this blog article by one other excerpt from regulatory action by the FDA. This example fascinated me not because the organization failed to assess, but because it did assess, and the assessments failed to pick up the problems.
In a letter to a blood services organization in 2012, the FDA said (red highlighting by me):

During an inspection …  FDA discovered that annual competency reviews and/or QA reviews did not detect that employees were not correctly performing all steps of testing blood samples. One test was repeatedly performed incorrectly by many employees beginning 2007, and another test was repeatedly performed incorrectly by many employees since April 2008. FDA’s review of the competency assessments for those employees performing those tests found that none failed the assessment.

So in this case, employees were being tested. But they were all passing, in spite of not doing their job correctly. This highlights a key requirement when using assessments:  well designed assessments are the best way of measuring competence and understanding. But you need to make sure that your assessments are well designed, valid and reliable. A poor quality assessment can give false comfort.
A sensible regulator will not only want to see that you assess your employees’ competence, but also that you use professional techniques to create, deliver and report on those assessments to ensure that they match the requirements for the job.  It is of course desirable that all your employees pass competency assessments, but only if those assessments truly measure the key requirements for the job.
For more background on good practice using assessments in the health care industry, read the previous To Your Health posts listed below. You can also read Questionmark’s best practice white papers for advice on providing assessments that get effective results.