An operator competency assessment should answer one question: can this person run the job the way your best operator runs it? Most forms answer a smaller one. They prove the training happened.

Seventeen Ticks. Six Line Stops.

Tomasz passed his sign-off on a Friday. Seventeen boxes, seventeen ticks, supervisor initials at the bottom.

By Wednesday his line had stopped six times. The form had no box for any of them.

Nothing on that form was wrong. He could name the parts, find the stop button and complete the changeover steps in order. What he couldn’t do was the reset the experienced operator performs by feel when the last case of the old product clears the sealer. It wasn’t on the form, so nobody had checked it.

This Isn’t a Tomasz Problem

Most plants have a sign-off form, and most forms share the same blind spot. It was written from the SOP. The SOP was written from the spec. The method your best operator actually uses is in neither.

The survey is not about sign-offs, but it shows how little confidence workers have that new hires are trained past the basics. In a September 2024 Morning Consult survey published by GE Aerospace, 1,000 people working in manufacturing were asked whether new hires, those new to the industry in the last two to four years, are getting the training they need. 55% answered “do the basics of the job” and 14% “just get in the door,” a combined 69%. The PDF does not say who commissioned it, and it measures what workers believe, not what new hires can do.

What Does an Operator Competency Assessment Actually Measure?

Most measure one of three things, and none is competence.

• Attendance. The person completed the training module or shadow shifts.
• Knowledge. The person can answer questions about the job.
• Coverage. The person was seen performing each task once.

Each of these is a reasonable thing to check. But a person can pass all three and still not be ready, because none of them compares what the trainee does to what the best operator does. That comparison is the only measure that matters, and it’s the one the form skips.

Why Do Sign-Offs Pass People Who Aren’t Ready?

Because three habits creep into almost every sign-off process.

First, the form tests the document. If the SOP is missing a step, the form can’t ask about it. We covered the same trap in how to measure SOP compliance in manufacturing: auditing against the document tells you nothing about the steps the document left out.

Second, one clean run counts as a pass. A trainee who completes a changeover on a quiet shift with the trainer standing beside them has shown the best case. The job also includes the bad case.

Third, the trainer is also the assessor. People rarely mark their own teaching as incomplete.

How Do You Assess Operator Competency Without Another Quiz?

Compare the trainee to your best operator on the real job. Don’t compare the trainee to the form.

Here is a sign-off that works, in four moves:

1. Build the criteria from the best method. Watch your best operator run the job and list every action, including the ones no document mentions.
2. Assess under variation. Include at least one changeover, one abnormal condition and one shift other than first.
3. Separate the assessor from the trainer. A second supervisor or lead marks the run.
4. Sign off in tiers. “Runs with support,” “runs alone” and “handles abnormal conditions” are three different sign-offs.

This ties directly to the idea behind operator to operator variation: the best method is often the one nobody wrote down, so that’s the method the criteria must come from.

What the Market Sells Instead

The usual answer is tracking. Learning platforms that log completed modules. Certification dashboards that turn a column green when a date is filled in. Skills matrices with a heat map. All of it records that training occurred, and none of it checks whether the trainee’s hands match the best operator’s.

A skills matrix is useful for knowing who is trained on what. We looked at what a manufacturing skills matrix missesfor that reason. But the matrix records a level. It doesn’t create the level. The thing that sets the bar is the criteria, and the criteria need a source.

When the Method Changes, the Sign-Off Should Too

This is the part most plants miss. The best operator’s method moves. A new material, a new supplier or a better trick changes the job, and a sign-off built last year tests last year’s job.

At SenseiLab, the sign-off criteria come from the Smart Living SOP. When the floor changes a method and the SOP is updated and validated, the criteria a trainee is signed off against change with it, and drift between the document and the work gets flagged instead of discovered on Wednesday. That’s the thing a static form can’t do.

Run the Divergence Check This Week

You can start without us. It takes about two hours:

1. Pick one operator signed off in the last 90 days, on one job.
2. Watch them run the job and watch your best operator run the same job right after, with a second person writing down every difference.
3. List each difference.
4. Hold the sign-off form beside the list. Mark every difference that appears on the form as covered and every one that doesn’t as missing.
5. Mark every box on the form that matched no observed difference as dead.

Count the missing ones. If the missing pile is bigger than the covered one, the form is certifying the document, not the job. Our guide to how to train new operators faster shows how to fix what you teach once you know what’s missing.

Frequently Asked Questions About Operator Competency Assessment

What is an operator competency assessment?

An operator competency assessment checks whether a person can perform a job to the standard of your best operators. Many plants use a sign-off form, but a form only covers what the written procedure already describes.

How do you know when a new operator is ready to run alone?

Check them under variation, not on a single clean run. They should handle a changeover and an abnormal condition, assessed by someone other than their trainer, and their method should match the best operator’s on the steps that affect quality.

Who should sign off a new operator?

Someone other than the trainer. A second supervisor or lead who watched the run, not someone who taught it, should mark the assessment.

How do we train new operators faster?

Fix what you teach before you speed up how you teach it. Our article on how to train new operators faster shows how to build training from the best method rather than the written one.

Forward this to one supervisor who has signed off someone they weren’t sure about.

About the author: Diego Echenique, MBA, is CEO & co-founder of SenseiLab, with 20+ years in manufacturing operations.