Part 2 of 4
The manual path Start with a path people can operate without the AI (the manual path). Switching automatically to another AI supplier does not, by itself, establish one.
Is there a documented way for people to do this work without the AI? People must be able to take responsibility for completing the work, using the tools available to them.
No. There is no manual path; we would wait for AI or rely only on automatic failover. Yes. People have a procedure, even if its documentation or testing is incomplete. Not sure; I need to check the evidence.
How complete are the instructions? They should be followable by a competent person who did not write them. L1 does not require a completed drill.
There are no written instructions. The steps are followable, and limitations and the volume ceiling are stated. The steps are followable, but limitations or the volume ceiling are missing. Some steps are missing or depend on knowledge the reader would not have. Not sure; I need to check the evidence.
Does any step still need the AI system this procedure replaces? Include API calls and information the system would have to generate at the time of use.
Yes. At least one step still needs it. Not sure; I need to check the evidence. No. The procedure can run without that system.
What can people access without the supplier cooperating? Consider inputs, previous outputs, reference data and the business knowledge needed to make decisions.
The required data and working knowledge are both available independently. The data is available, but some necessary knowledge exists only inside the AI. Some required data is available only through the supplier. Not sure; I need to check the evidence.
Are the people responsible for the manual path identified by role? This asks who would do the work. Evidence of their performance comes later.
Not sure; I need to check the evidence. Yes. The responsible roles are recorded. No. It has not been assigned.
Does the manual path use a different external AI supplier? A shared cloud, jurisdiction or access restriction may stop both suppliers. This check concerns a substitute used inside the manual path.
Yes, but there is no recorded assessment of failures affecting both suppliers. No. The manual path does not substitute another external AI supplier. Yes, and an assessment of shared failure risks is recorded. Not sure; I need to check the evidence.
Part 3 of 4
Evidence from practice Use the records for this capability. A drill is a practice run on real work with the AI genuinely unavailable (specification section 4.4). A tabletop exercise (talking through the procedure) or a rehearsal on sample cases is not a drill. Training, planned exercises and estimates do not establish actual performance. Only relevant follow-up questions appear.
What drill evidence can you rely on for this procedure? Use the latest drill being relied on, and account for any later failed drill. Extra tabletop or rehearsal exercises do not invalidate an otherwise current drill.
There is no drill evidence; at most we have tabletop or rehearsal practice. A drill failed and has not yet been repeated successfully. A successful drill is recorded, with no later unresolved failed drill. Not sure; I need to check the evidence.
How long ago was that successful drill? This checks the 90-day L2 schedule. L3 uses a separate routine-performance review, while still requiring current measurements.
90 days ago or less. 91 to 119 days ago. 120 days ago or more. Not sure; I need to check the evidence.
Was the replaced AI actually unavailable during that drill? For example, access was blocked or the integration was disabled for the participants.
No. It remained accessible, even if people were told not to use it. Yes. Participants could not access it. Not sure; I need to check the evidence.
What work did participants perform? Use the actual workload within an authorised, safe scope. A rehearsal can prepare a team but does not satisfy the real-work requirement.
Real work from this capability. Only sample cases, simulated work or a tabletop discussion. Not sure; I need to check the evidence.
How often are drills unannounced to participants? The requirement is at least one in every four. You do not need four completed drills before a first assessment.
A completed group of four drills had no unannounced drill. Fewer than four drills have taken place; the first group is not complete. At least one in every completed group of four was unannounced. Not sure; I need to check the evidence.
What does the latest usable drill record contain? Time to Manual is time to the first correct human result. Capacity records the share of normal work people can sustain and for how long. The record also needs its date and what went wrong.
Only estimates or projections, rather than measurements. Measurements exist, but a figure, duration, date or record of problems is missing. All of those details, measured in a drill within the last twelve months. All of those details, but the measurements are more than twelve months old. No measured figures are recorded. Not sure; I need to check the evidence.
How many current operators have demonstrated this work without supervision in a drill? Training records alone do not establish this. Count demonstrated performance of this capability.
One person. At least two people. None; we have only training records or expectations. Not sure; I need to check the evidence.
Have the people who last performed this work unaided left the organisation? If they have left, a repeat drill is required before reasserting the level.
Yes, and the capability has since been re-drilled. Yes, and it has not been re-drilled since. No. Not sure; I need to check the evidence.
Outside drills, do people routinely complete this work independently of AI? Completing real work counts, including a regular share of the volume. Merely reviewing AI output does not.
People review AI output but do not independently complete the work. Yes. Independent human work is part of normal operation. No. The manual path is used only for exercises or outages. Not sure; routine human performance has not been established.
When did a review last confirm routine human performance? This is the L3 review, separate from the register review and from drill dates.
No confirming review is recorded. Within the last twelve months. More than twelve months ago. Not sure; I need to check the evidence.
Your result identifies gaps and the evidence needed for your next improvement.
Calculate my result Clear answers