1. Scope
Define the data type, intended model or workflow use, volume, Hebrew variant, source permissions, file formats, metadata, quality metrics, timeline, and acceptance criteria.
2. Guidelines
Create collection, annotation, evaluation, formatting, redaction, and reviewer instructions. Guidelines include positive examples, negative examples, edge cases, escalation rules, and defect severity definitions.
3. Pilot
Produce a small representative batch or evaluation sample. Measure defects before scale and verify that the output schema, reviewer instructions, and acceptance process are usable.
4. Review
Compare outputs with the agreed rules, resolve ambiguity, revise instructions, and document decisions. High-severity failures and recurring defects are prioritized.
5. Production
Deliver recurring batches with tracking, QA evidence, rejection reasons, acceptance summaries, and change control when guidelines or model behavior evolve.
Acceptance evidence
| Control | Evidence |
|---|---|
| Scope compliance | Required fields, formats, volume, language, and source status |
| Quality metrics | Measured thresholds relevant to the task |
| Reviewer calibration | Calibration results, disagreement rate, and adjudication notes |
| Defect management | Error taxonomy, severity, root cause, and corrective action |
| Batch acceptance | Accepted, rejected, repaired, and pending counts |
Quality is task-specific
No single accuracy percentage proves that a dataset or model is production-ready. Metrics are selected according to the business use, failure cost, source data, and required decision reliability.