Extractive Question Answering

What Is Extractive Question Answering?

Extractive question answering is a natural language processing method that finds and returns the exact words in provided text that answer a question. Instead of writing a new response, it selects a continuous piece of source text, called an answer span, so readers can check the evidence.

It is commonly used with defined document sets, such as manuals, policies, contracts, or help articles. Unlike web search, extractive QA returns a precise answer rather than links. Unlike conversational AI, it does not normally compose a new explanation. It extracts what the source says.

How Extractive Question Answering Works

An extractive QA system usually combines document retrieval with a model that reads a selected passage. Its dependability relies on both stages.

1. Prepare source documents by collecting approved text, removing duplicates, and preserving titles, dates, permissions, and metadata.
2. Retrieve a relevant document or passage when the collection is large. This may not be necessary for a single short passage.
3. Split long documents into manageable passages, often called chunks, while retaining enough context to preserve meaning.
4. Send the user’s question and selected context to a reader model. The model considers how the wording relates.
5. Convert text into tokens, which are units a model can process. A word may contain one or more tokens.
6. Score possible start and end token positions. The highest-scoring valid pair identifies the answer span, such as “within 30 calendar days.”
7. Apply a confidence threshold and no-answer option. If the source does not support an answer, the system should say so rather than force a span.
8. Display the extracted text with its document title, location, and nearby evidence so users can verify it.

Key Components of an Extractive QA System

The document corpus is the approved source material. For a small, fixed context, a reader model can search the passage directly. For a larger corpus, a retrieval layer identifies likely documents or chunks. Retrieval can use keyword matching, semantic similarity, or both.

Other components include passage chunking, a tokenizer, the reader model, confidence scoring, and an answerability classifier. The classifier helps distinguish “the answer is absent” from “the answer is difficult to find.” A citation or evidence window should show where the span came from. This is useful in AI glossary and knowledge-base experiences where readers need definitions with a clear source.

Retrieval quality and chunk boundaries matter as much as the reader model. If the correct paragraph is not retrieved, even an excellent reader cannot find it. If a chunk separates a condition from its exception, extracted words may be accurate but misleading.

Extractive vs. Generative and Open-Domain Question Answering

These categories describe different answer behaviors and system designs. They can overlap when an open-domain system retrieves information before answering.

Open-domain QA is not the opposite of extractive QA. It can retrieve passages from many sources and use an extractive reader to select the final answer span.

Where Extractive Question Answering Is Useful

Extractive QA works best when users need a direct answer they can check against trusted documents.

• Employee handbooks and policy portals, for questions about leave, expenses, or eligibility rules.
• Contracts and compliance documents, where reviewers need exact clauses and conditions.
• Product documentation and support knowledge bases, including self-service customer support chatbots that should point users to approved wording.
• Research repositories, where readers need a quoted definition, method, or stated finding.
• Clinical and legal document retrieval, with qualified human review and strict access controls.
• Document-review workflows that help teams find relevant language before making decisions.

Benefits of Extractive Question Answering

The main value is disciplined grounding. The system is constrained to language in its source context.

• It provides direct evidence rather than an untraceable answer.
• It lowers the chance of unsupported wording because it does not need to invent a sentence.
• It produces concise responses when users need a clause, date, name, or rule.
• It supports audits because teams can preserve the answer span, source version, and retrieval path.
• It handles controlled vocabulary well, including product names, policy terms, and regulatory wording.
• It fits stable, maintained document collections with clear ownership.

Practical Limits and Common Pitfalls

Copying words from a document does not guarantee that an answer is complete, current, or appropriate.

• Ambiguous questions can lead to a plausible but unintended passage.
• A source may not contain an answer, so reliable abstention is essential.
• Outdated, duplicate, or contradictory documents can produce conflicting answers.
• Some answers require combining non-contiguous text, which a single-span model handles poorly.
• Poor retrieval can hide correct evidence before the reader model sees it.
• Long documents and weak chunk boundaries can separate a rule from its exception.
• Confidence scores can be miscalibrated, so a high score does not always mean the answer is correct.
• Private or regulated documents require permission controls, logging, and careful data handling.
• Users may mistake quoted text for a final decision when expert interpretation is needed.

How to Evaluate Answer Quality

Benchmark evaluations often use exact match, which checks whether a predicted answer matches a reference answer, and token-level F1, which measures word overlap. The SQuAD benchmark helped establish these measures for reading-comprehension QA, but they do not capture every business risk.

Operational evaluation should also test retrieval recall, answerability accuracy, citation correctness, response time, source coverage, and real user phrasing. Review high-impact answers manually, especially where a missing condition could change a financial, legal, medical, or safety outcome. A strong benchmark score does not replace testing against actual organizational documents.

Implementation and Design Best Practices

A dependable system is designed around evidence, abstention, and source maintenance, not model accuracy alone.

• Define acceptable answer types before building, such as dates, clauses, definitions, or short policy statements.
• Clean, version, and assign owners to source documents so obsolete material can be removed quickly.
• Create meaningful chunks with modest overlap to avoid cutting context at a heading, sentence, or exception.
• Keep document metadata, including title, section, publication date, access level, and source version.
• Show the originating sentence or surrounding paragraph, not only the extracted phrase.
• Set a clear abstention path for low-confidence, missing, or conflicting evidence.
• Test paraphrases, vague wording, spelling errors, and adversarial questions that try to pull an answer out of context.
• Monitor source changes and retest important questions after updates.
• Route sensitive decisions to qualified reviewers rather than treating output as professional advice.

A Simple Extractive QA Example

Suppose a handbook states: “Employees must submit travel expense reports within 30 calendar days after returning from a business trip. Reports submitted later require manager approval.” A user asks, “When are travel expense reports due?” An extractive system may return “within 30 calendar days after returning from a business trip.”

That answer is faithful to the text, but showing the full sentence or nearby section matters. Without the next sentence, a user may miss the approval requirement for late reports. The practical rule is to return the precise span, then show enough evidence for readers to understand its conditions and limits.