Learn by opening one retained evidence package
An evidence package is a recorded selection of source passages, concepts and relationships for one question. It also records what was missing. It contains no AI answer. This project is an independent experiment, not official DWP guidance or an individual benefits decision.
1. Start with the question
Read the question exactly as supplied. A question about a permanent care-home move does not tell us whether the person has a partner, which benefit components are present, or which date applies. Those missing facts must stay visible.
2. Check what was recorded
A retained example preserves a previous selection. Opening it does not rerun Ask OKF or fetch the latest law. Its source version identifies the captured material; its engine identifies the software that selected it. An unknown historical engine is labelled as unknown.
A hash is a fingerprint of exact bytes. Matching hashes show that you received the recorded file. They do not establish that the source is current or that an interpretation is correct.
3. Read the evidence in small pieces
A manifest is a catalogue of the selected records. Choose a record to read its complete source text and metadata. Metadata describes its origin, locator, dates and authority. The reader checks every required part before presenting a complete value.
A link labelled “first exact part” may contain only part of the text. Use the verified reader for the complete passage. Smaller deliveries solve a transport problem; they do not remove the need to read qualifications and exceptions.
4. Follow the relationships
A directed relationship joins one identified record to another. Its direction matters: a summary can require a supporting passage, while merely mentioning a benefit does not establish that a rule applies to every claimant.
An endpoint outside the selected package is a reference. This reader does not silently fetch it or fill the gap. Check the missing-evidence list before drawing a conclusion.
5. Inspect dates and authority
A source date describes the material. A capture date describes when this project obtained it. Neither is automatically the date on which a legal rule applies.
Official guidance, machine-extracted text and project-authored interpretation are different layers. The package preserves their labels. A technically verified download does not upgrade any layer to specialist-approved advice.
6. Check the complete package
The complete machine-readable JSON preserves the original question, selected records, relationships, provenance and diagnostics. JSON is a structured text format that software can read. The small resource files must reconstruct exactly the same package.
An insufficient result means the package cannot establish the complete requested answer within its recorded evidence and scope. A truncated result means a retrieval or output limit was reached. These are useful findings to show in a demonstration.
7. Compare an AI answer separately
An AI trial gives a model the fixed package and a fixed instruction. A citation check can establish that a quoted passage exists at the stated locator. It cannot by itself establish that the conclusion follows, that all exceptions were considered, or that the rule applies to the case.
Read the model trial guide and retain unsuccessful attempts alongside successes. The unknown-term control tests whether a model abstains when the package contains no evidence. Benefits specialists still need to assess meaning and applicability.
Reuse this method for another department
Begin with source, terminology, ontology, regulation and legislation discovery. Preserve the original acquisition; then build explicit concepts, relationships and task requirements. Measure candidate discovery, required-path retention, delivered bytes and answer quality separately. Add neutral control questions so a small set of demonstration questions cannot become the only measure of success.
Follow the portable methodology, retrospective and publication guide. Do not copy the DWP benefit vocabulary into a new department's model without its own discovery and review.