ApiaryActiveLive
Try: pause · settings · learn · wipe
← Community / Reading Room
KA
craft · 24 min read

Keep a Model Test Fixture When Policy Changes

A support assistant used to give the right answer about returning borrowed display samples. On Thursday, it passed a fixture that says the person can bring a…

A support assistant used to give the right answer about returning borrowed display samples. On Thursday, it passed a fixture that says the person can bring a sample to either counter. On Friday, the shop adopts a rule requiring return to the original counter. The project owner edits the test's expected answer and reruns the report. The new result appears to show the assistant wrong yesterday as well as today. It doesn't tell the reader that the policy moved.

Another team makes the opposite edit. It notices the assistant scored poorly on three difficult questions, changes their expected answers after seeing the outputs and presents the new pass rate as a model gain. In both stories, the test loses an important separation. The historical run records how an answer matched a particular expectation. The present release needs to know whether a new run meets upcoming or active expectations. Neither goal requires treating the past as if the current answer key had always been there.

This article uses fictional decisions about ordinary displays and workshop logistics. It is not a guide to changing laws, safety rules, sensitive workforce policies or any actual commercial relationship. Apply real authority and data-handling controls independently of the testing workflow.

AI disclosure: Drafted with AI assistance. Policy versions, benchmark outputs and release decisions are synthetic teaching examples.

Decide which kind of truth each record carries

One record is the policy edition that can govern the practical answer. One is the exact material admitted to the fixture. A third names the rubric against which an output will be evaluated. Another is the captured output and its run conditions. Your dashboard shows a result derived from these things. Treating them as a single mutable file makes edits convenient and the evidence hard to reconstruct.

Policy and fixture versions need not move together. A narrow clarification might only affect two cases. A fixture can also change because its wording created ambiguity unrelated to a policy revision. A rubric could be corrected for a scoring bug. In every case, record the reason and create a version rather than overwriting wherever it hurts today's chart. A team must be able to tell a changed system result from a changed question or expected answer.

Think of three chronological statements: “Model M passed case C under policy P3 and fixture F7 on Thursday”; “P4 changes the return rule from February 1”; and “on February 1, M currently fails the equivalent updated case.” These can all be true at once. A useful closeout keeps the paths visible instead of asking which pass is the real one. Validity is always relative to a defined context and date.

The W3C provenance data model distinguishes entities, activities and agents. It gives a useful mental model. Store a policy snapshot and a rubric as distinct entities; record capture and grading as work performed with specific evidence; identify whose review authorized the assessment. A team can do this with a spreadsheet and controlled folder. It doesn't need to implement a formal standard to stop editing old observations in place.

Start with a policy change notice, not a lower score

Open the authorized source that announces the change. Identify scope, trigger, applicable dates, any transitional rule, and which source owns unresolved details. A model doesn't decide policy based on the most recently edited file. Nor does a benchmark owner gain the authority to invent a current ruling simply because an older answer started looking impractical.

For a fictional workshop, version R3 allows return at either of two staffed counters. Version R4 says “For sample loans started from February 1 onward, use the counter shown on the receipt.” A fixed start condition means a person returning on February 3 could still be under the earlier rule if they began the loan on January 29. A new test that treats every February return as R4 doesn't measure adherence to the actual policy. It measures adherence to a shortcut.

Store exact rule ownership and observation lags. A draft email might report an upcoming decision while the approved manual still applies. Ask the authorized policy owner before grading system behavior on that proposed future rule as if production had already adopted it. Still useful is a labeled preview fixture that tests readiness for the possible change. Keep its results out of the current serving score unless your definition says otherwise.

When a score drops without any known policy change, start with the evidence. Were different files supplied? Did the grader make a unit error? Did a new version of the system behave differently? A score drop does not itself prove the policy was revised, and “label was unclear” doesn't automatically justify making the model's wording correct in the test. Route the diagnosis to the correct owner.

Assign stable families and immutable fixture revisions

A stable case family helps people recognize what a test is about across rule changes. Use one family for returns to an alternate branch, one for pickup timing, one for unsupported transaction claims. Each approved revision keeps its own exact prompt, source material, rubric and expected behaviors. Family label “branch return” is not enough to reconstruct which evidence the system saw.

A lightweight registry can include family ID, immutable revision ID, originating policy references, statement of scope, evidence admitted, accepted response examples, known failure conditions, owner, and whether it is active, proposed, retired or archived. Some cases can point to one approval record rather than repeating it several times. Keep content identifiers scoped so replacing a source file under the same path cannot give a model a different fixture under the old name.

Suppose family BRANCH-07 contains revision A with R3 and revision B with R4. Revision A can remain useful for a historical release or a transition case if the policy still governs older loans. Revision B can be qualified as active for new loans. When the earlier policy stops governing any relevant production cases, its fixture may move to a historical suite without being deleted or accidentally counted in the current test. A single active Boolean per family is too weak to express these distinctions.

Define how to handle actual invalid test material. If a fixture used a draft or incomplete excerpt and no meaningful inference can be recovered, record a dated invalidation reason and restrict retention as appropriate. Do not keep unauthorized materials just to save the audit. The valid evidence you can retain might be a neutral record describing why the earlier result cannot be compared. Preserving historical accuracy has never required preserving all underlying data indiscriminately.

Test rule changes as a set of consequences

A change rarely calls for changing one answer sentence alone. If alternate counters are no longer allowed for new loans, the system may need to recognize when a receipt does not name the original counter; it may need to explain that an old loan can still use an alternate counter; and it may need to avoid inventing the actual branch or promising pickup arrangements it didn't verify. Fixtures should probe those consequences separately if they are important.

Build a synthetic three-part table. Case A starts on January 29 and asks where to return on February 3. Case B starts February 2 and shows the receipt branch. Case C starts February 2 but has no branch information. A faithful model can distinguish old eligibility, new known requirement, and missing identifying detail. Writing “always tell the person to use their original counter” as a new reference would miss at least one case and guess about another.

Ask which fixtures are genuinely unchanged. A separate case asks whether the display sample can be renewed over the phone. Neither policy edition settles it. If the new branch rule doesn't touch renewals, an answer claiming an automatic renewal should fail under both. Preserving some stable probes lets the team observe collateral regression while updating cases affected by the policy. Stable doesn't mean immune to unrelated source changes; periodically confirm its dependencies.

Record a reason for each case's status: unchanged under the update, revised for effective scope, retained as transition test, replaced because fixture was underspecified, or removed from active use after approved deprecation. A reviewer should be able to see that adding a more convenient case did not conveniently replace a harder independent one. This is useful to readers whether or not they ever care about aggregate pass rates.

Mark which release questions need side-by-side runs

A release owner might ask, “How well did the old model answer when R3 applied?” Historical results can address that. They may ask, “What happens under R4 if we leave the same system configured with old retrieval?” That is a different controlled evaluation. They may also need to compare revised and baseline systems on the same R4 fixtures. That is a direct current deployment question. A single pass rate can answer none of them unless input scope and configurations are known.

A useful comparison table gives one column for historical record, one for current frozen benchmark, and perhaps one for transition stress. On historical rows, retain the observed date and policy context. On current rows, note exactly which configurations had the same permitted evidence and which bundles intentionally differ. Most teams don't need every cross-combination. Choose the ones necessary to diagnose whether improvements arise from better source exposure, better reading, changed expectations or a different operating path.

The OpenAI evaluation best practices guide recommends task-specific testing, practical evaluation data and appropriate human judgment. In this example, narrow fixtures tied to explicit policy changes provide more diagnostic evidence than a generic thumbs-up on recent answers. A test that stops at measuring old rule recall does not verify the current tool's actual task.

Avoid treating a new model's expected performance as the reason to remove the old suite. Historical fixtures may not belong in a current release score, but results under them remain meaningful. If evaluating with old and new policies in the same suite, make the date and origin context explicit so the quoted input is part of the task, not a hidden dependency the assistant has to guess.

Hold a separate lane for the present transition

Some policies change once with no overlap. Others carry valid exceptions for actions initiated before a date. A test owner can create a time-scoped fixture without claiming accurate predictions about calendar behavior in real production. Make the test clock or scenario facts part of admissible evidence. A model reading “today” needs to know which date it refers to under the fixture; grading it using the actual date of rerun would change the case.

In the fictional return policy, include eligibility based on start date rather than current date. Another fixture could cover the exact effective boundary if the policy specifies time and location. Make the date and time basis explicit where it matters. If policy leaves the time boundary undefined, flag known ambiguity rather than silently choosing midnight in the evaluator's own time zone. The exercise is a test of grounded answers, not date inference by proximity.

Keep conditional transition answers separate from proposed corrections. An assistant can explain “for loans starting from February 1, use the receipt counter” while leaving location unknown for an individual. That is useful if the input doesn't identify the specific record. A confident statement about a person's actual transaction is not justified merely because the system knows the general rule. Graders should recognize a correct appropriate boundary, not reward only complete-looking one-line answers.

Take the opposite edge too. When the prompt states all relevant facts and a current, clear rule, excessive refusal isn't excellent. It may protect against a hypothetical error while failing the task asked. Include supported and blocked cases. A well-maintained benchmark measures appropriate confidence on both. A policy update that increases abstention is desirable on some inputs and a regression on others; the fixture set should show the distinction.

Archive a run with evidence of what was seen

One record can say run ID, configuration version, fixture version, policy evidence admitted, selected time and observed response. Store grade and adjudication in a related record with rubric version and reviewer. This prevents an updated score from appearing to modify the captured words. A simple historical page can display last verified production pass under its old context and current release decision under a separate banner.

Confirm what was actually supplied in both runs. A system connected to retrieval might have received one edition and not another because a source path remained stale. An authoritative new policy in your repository does not prove the system read it. A fair same-fixture test can expose both models to the same approved texts. A workflow realism test may intentionally compare whole retrieval paths including which documents each returned. Name the question before judging the result.

A grade can be marked old, superseded for current use, or reassessed as an historical audit. An actual score correction is possible and sometimes necessary. Keep the earlier report and explain which rows the correction affects under permitted retention. Otherwise the reader cannot tell whether tests changed, a bug was found, or someone made the answer key fit an output. Auditability is a way to correct openly, not a requirement to defend known errors forever.

Test when the source moves under the same stable web link. If the older retrieval shows the prior edition and no controlled snapshot is available, state that reproduction is incomplete. Don't replace the earlier input with today's contents and call it the same evidence. A note about known differences can still be useful, even when exact reenactment is unavailable. Capture status should not be inflated into a guarantee of permanent source availability.

Recalculate on a stable test set before claiming improvement

Suppose a hypothetical dashboard shows 18 passes out of 24 cases before an update and 16 passes out of 18 cases afterward. The displayed rate improves from 75% to about 89%. The old test included six especially difficult cases that are no longer in the active set. The new rate can accurately describe the current test population. It offers no fair standalone evidence that the model improved.

Break the report into the common cases and changed membership. If the earlier system also passed 16 of the 18 unchanged cases, then the observed result on that set did not improve. The six removed cases may have been removed for a legitimate rule change, because their scope no longer exists, or because the benchmark author found a defect. Record the actual reason. An audit can then distinguish useful current scope management from a metric that accidentally delivered good news.

Model and rubric should never be tuned and evaluated on an undisclosed moving target. A team might use failures to design help content or improve prompts, which is reasonable work. It then needs held-out cases or clearly separated evaluation stages if it wants evidence that the fix generalizes beyond the examples just edited. If all example text was chosen after seeing the answers, call it a diagnosis or regression suite iteration rather than a clean prospective score.

If membership changed for policy reasons, don't force the old benchmark to remain active merely to retain comparability. The business still needs current validation. Present the legacy and current results as different slices and compare only where both configurations saw equivalent evidence and were judged by a common rubric. Unchanged fixture definitions and changed policy cases can each earn their own short headline. That is often more informative than a blended gain.

Treat a promising future rule as future evidence

Proposed rules are frequently circulated before approval. Benchmark authors can use artificial or authorized versions to prepare for potential updates. The question is whether the tool can handle that future condition if it is adopted, not whether today's answers should already follow it. Keep preparation runs in a separate visible lane. Permit internal rehearsal under normal policy without making the output an outward operational instruction.

A fictional fixture says the office plans to consolidate sample returns at one location in spring but has not settled a start date. An assistant under this information can describe the proposal or note the missing date if asked. It should not tell visitors that the current two-counter option has already ended. A conditional scenario can hypothetically set a date and verify behavior under that premise. Label it as such and don't include it in a claim about actual policy.

When a planned revision is withdrawn, update its expected active status while retaining enough allowed history to understand what was tested. An unused test can remain a useful design exercise. It should not quietly become evidence of a failed rollout. Record which roles confirmed withdrawal and whether the team needs to check published material for stray future messaging. Comparison authors need not directly change external systems simply because they identified a risk.

Keep designated activation and deactivation triggers clear. A requirement that starts on the issue date may differ from one that starts when the customer begins a new loan, and a routine draft filename isn't proof of either. The benchmark should ask about the behavior in the appropriate window. If the window is unknown, a quality report can name it as a blocker rather than forcing a yes or no that the policy doesn't support.

Resolve a genuine key error without making history fictitious

Sometimes the old answer key was wrong even under the older policy. That is not an ordinary version change. Suppose R3 did allow returns to either staffed counter, and the old rubric erroneously required the original counter because a tester borrowed wording from another catalog. A result marked fail could have been substantively correct. A responsible audit can correct the historical grade using the same captured answer and same evidence, with a dated adjudication record.

Record original result, reason for correction, confirmed policy and reviewed replacement score. If the original comparison cannot be reconstructed, state that limitation. Do not bulk replace archived scores as if they were always correct under the old rubric. A headline may need a revision if readers rely on it, but source errors deserve visible correction rather than a disappearing change. This protects people who acted on the older number from confusion.

How do you distinguish a mistaken key from a difficult judgment call? Ask which exact requirement each version asserted, read the approved source and collect independent review without showing which model benefits. If the evidence really can't decide, use an unresolved status. Graders can document a new rule for future cases but can't decide afterward that a prior answer failed simply for not anticipating it. Apply any corrected rubric to comparable cases on both sides.

The NIST Generative AI Profile discusses risks including confidently erroneous content and overreliance. These concerns apply not only to outputs but to how people read evaluations of outputs. A plausible score can create confidence even when grounded in an unclear answer key. Independently verifying the actual policy and keeping a correction trail makes that trust more defensible, without claiming that written procedures remove all evaluation risk.

Test ambiguous policies without inventing clarity for the benchmark

An operational manual can leave reasonable uncertainty. One section says return to the named counter; another authorized section says mobile events use the welcome tent. Does a particular event count as mobile? Does the special rule supersede the general one? The test author shouldn't let a desire for automatic grading choose the answer. Depending on authority and available facts, a good output may need to ask one concise question or route the case.

Construct two versions of a synthetic ambiguity. In the first, the appropriate source explicitly classifies the event and establishes the special rule. The expected answer can use it. In the second, classification is unavailable. The expected response can state the conditional policy and avoid telling the visitor where their actual record belongs. A benchmark passes only if it recognizes why the evidence differs, not because it repeats one rule more fluently.

Separate factual uncertainty from missing action authority. A model can know where a sample belongs and still not have permission to modify its booking or send instructions on someone else's behalf. A fixture about advice shouldn't score unsolicited action as successful task completion. Conversely, the task should not ask the model to edit transactions in a testing environment just to learn whether it can explain a policy. Synthetic scenario and actual tool authority should remain visibly different.

If staff disagree about routine interpretation, use their observations to improve the source. Document the case question rather than deciding by whichever grade makes the dashboard stable. Once the right owner clarifies the intended rule, create a new fixture edition. Until then, show counts of unresolved tests. An unknown grade can be an honest representation of incomplete evidence; it does not belong in the pass column to avoid a busy release discussion.

Retire a case without killing the evidence it provided

A good fixture can become obsolete for a useful reason. The business stops offering a particular product, an old process no longer exists, or a legal source is replaced. Move the case out of current scoring when actual scope changes and mark the approved reason and date. Old runs remain interpretable as tests of old behavior. A historical failure doesn't become true or false in a new way just because nobody uses its procedure today.

Some cases need stronger handling. A test based on live customer content could later be found outside retention or use permissions. Follow the actual response and retention process; don't preserve sensitive detail just to recreate a metric. A minimal authorized retirement note may say prior case scope invalidated and not eligible for future reuse or aggregate reporting. That can be enough for an honest archive even if the exact prompt can no longer be shared.

Record whether the replacement is equivalent or introduces a new task. Rewording a synthetic label while preserving the same decision can be a comparable revised example under specified terms. Replacing a return case with a scheduling case changes coverage. Add an issue or gap if the removed case formerly represented an important edge that no current fixture covers. Retiring a bad example should not hide that the new suite has a blind spot.

Make the archive hard to confuse with production guidance. An old source might still be searchable for authorized analysis and shouldn't be retrieved as if it currently governed. Mark historical state in the record itself or separate operational and historical views. A test exercising past rules can use a clear date, edition and synthetic context. This distinction protects real product guidance without erasing inconvenient observations.

Put policy-readiness and product-readiness into two records

A team can verify that the updated assistant explains the fictional R4 rule correctly using a fixed supplied excerpt. That supports one specific claim about reasoning with that input. It doesn't prove an application has indexed the approved policy, removed superseded retrieval sources, set the correct effective date or shown the intended answer surface. Keep readiness at the fixture layer separate from verified operational state.

Think of a rehearsal in a sandbox. The rule is supplied under a documented test clock. Both baseline and candidate get it. The candidate passes the supported answer and abstention cases. Later, an end-to-end QA run in an authorized environment discovers the rendered help page has an older summary cached. The comparison is not invalidated, but its scope is clarified. The next action is to investigate the relevant serving path, not retroactively claim the reasoning test failed.

In the opposite direction, an app can serve fresh policy but the wording still overstates its scope. A deployment receipt does not replace behavior verification. Review what the assistant actually says for a representative authorized input, including transitional facts and unresolved prerequisites. Record known browser or tool limitations rather than asserting all contexts work from one passing case. Never use live sensitive records just to create realistic pressure on the system.

Give release comments two separate verbs: benchmark passed for specified case versions, and current workflow observed under named approved conditions. Sometimes only one is available. Say the other is still unchecked. This clarity prevents one capable team from translating another team's evidence into more certainty than anyone actually observed.

Run the tabletop change from end to end

Take an invented dataset with eight active cases. Four depend on the return policy, two test unrelated unsupported claims, and two test a discontinued catalog feature. The policy owner authorizes a new rule that governs only loans started on or after February 1. The catalog feature is separately retired. Your first action is to chart dependency, not rewrite all eight expected answers. Mark which current fixtures continue, which need new revisions for newly applicable policy, which need preservation for legacy cases, and which move out of active scoring with explicit reason.

Now add a case where a new loan lacks the recorded original location and a case where an older loan returns on the day of transition. Freeze the updated benchmark before running both config variants. Keep any reused scoring policy plainly named. When unusual outputs arrive, inspect the source and decide whether observed failure is model behavior, fixture ambiguity, input mismatch or unresolved. Don't insert additional exceptions into the rubric to give a sympathetic sentence the win.

Write the final comparison in stages. First, describe what changed in the policy and why the selected cases moved. Second, state how both the baseline and candidate scored on the same updated cases, if measured. Third, distinguish original archived performance, remaining differences and current-use limits. Fourth, identify which end-to-end checks remain outside the benchmark. This does not need to be a long memo, only impossible to misread as a direct improvement on an unchanging historical measure.

Ask a second person to reproduce one historical grade and one present grade using their respective receipts. If they open a new file with the same name and get the same result by coincidence, the audit still failed to establish provenance. If they cannot see one protected source under their approved role, preserve that limit and choose another verified case for the collaborative check. Don't route around a legitimate permission boundary to make the exercise feel complete.

Verify policy dependencies when a case starts passing for the wrong reason

A captured answer can match the new sentence without demonstrating that its source was current. In an invented test, a system says to return a sample to the original counter because it always recommends the original location. It now passes the usual R4 case, even though it would have answered the same under R3. Another situation where an old loan is valid at either counter exposes the shortcut. Include cases that change outcomes under the relevant policy rather than assuming a correct answer proves which rule the model applied.

You can design a positive and negative witness for the same policy rule. The positive case supplies a current loan date and matched receipt. The negative case has a date before the new policy applied or an explicitly authorized exception. Add an unresolved input lacking the date under a stated fixture. The correct behaviors differ: apply the new rule, follow an older or exception rule, or state what remains unknown. That is more informative than an all-passing set of similar questions whose expected wording doesn't tell the investigator where the answer came from.

Don't turn evidence requests into prompt riddles that no real user would pose. A very artificial fixture can be useful for checking the distinction under controlled conditions, but a decision to deploy should also consider a realistic authorized example where the caller does not use the same labels as the manual. Record whether someone asked using hypothetical words such as “say it's Tuesday” or a fully specified date. Those differences can be relevant to what the system reasonably understood.

An answer with a good destination but fabricated reasoning also needs attention. If it claims the receipt counter is required because the assistant looked at a record it never received, separate correct procedure from unsupported record access. The grading rule can require that material justifications be supported rather than insisting on one exact sentence. In a workflow where an actual assertion of record access has consequences, this may be a hard gate. Define that rule prospectively and test it on both systems.

Make a revert and a re-evaluation different operations

Imagine the team stops an R4 rollout after a help interface fails a separate verification. The live serving system may temporarily revert to an earlier version, while the authorized policy itself remains R4. Rolling back software does not restore the old commercial rule unless the responsible policy owner explicitly changed it. The benchmark needs to reflect this. A test that automatically selects R3 whenever the deployment version is rolled back would silently make the tool's current answer less reliable.

A corrective report can state one rollback observation and one ongoing policy expectation. For example: “Serving build reverted while the help panel is investigated. R4 remains the authorized rule for new loans, so the restricted assistant path is still on hold. The last R4 fixture run was unaffected but does not attest to the restored serving state.” Those words separate use, evidence and authority. The next tester can decide what to retest without assuming the old score authorizes broader use.

If the underlying policy does revert under proper authority, create a new dated active selection that may point to an earlier approved text. Avoid silently treating it as the same historical occasion. The same rule wording can apply at two different times with different transitional expectations in between. A fixture about a loan begun during the gap still needs those facts. This is why text identity and applicability are separate. Your archive can reuse reliable content without collapsing distinct release histories.

Plan the first independent verification after reversal. Read the authorized current rule and run a small current-versus-legacy witness comparison. Report what is actually observed rather than reusing the last good scoreboard. A dated receipt of the new check answers a specific present question. It complements, not replaces, the record showing what the same system got right yesterday.

Leave the dashboard with a sentence it can defend

A good visible note can say: “Current benchmark R4, 10 scored cases under active and documented transition rules. R3 history uses a different case definition and is not a comparison. Two older fixtures passed then but do not represent current policy. A separate same-R4 baseline run showed one regression on unresolved-location handling.” If all observed results are fictional, mark them as examples. If no same-rule comparison occurred, leave that sentence out rather than estimating its result.

Don't let launch incentives shape a case after scoring. If a hard failure blocks release but the rule is correct and adequately specified, improve the system or narrow deployment scope under actual authority. If the fixture genuinely needs repair, record the evidence and apply the repair consistently. A useful benchmark is an instrument for finding conditions that matter, not an accessory meant to decorate an already chosen rollout.

The next change can start simply: preserve the old run, acquire the authorized new rule, version affected expectations, and test the same scope on both alternatives before claiming progress. Mark what was removed and why. Review complete, blocked and unresolved cases as different outcomes. When the benchmark no longer rewrites its own history, a team can learn from yesterday's real result while still insisting on the right answer today.

Frequently asked
What is Keep a Model Test Fixture When Policy Changes about?
A support assistant used to give the right answer about returning borrowed display samples. On Thursday, it passed a fixture that says the person can bring a…
What should you know about decide which kind of truth each record carries?
One record is the policy edition that can govern the practical answer. One is the exact material admitted to the fixture. A third names the rubric against which an output will be evaluated. Another is the captured output and its run conditions. Your dashboard shows a result derived from these things. Treating them as…
What should you know about start with a policy change notice, not a lower score?
Open the authorized source that announces the change. Identify scope, trigger, applicable dates, any transitional rule, and which source owns unresolved details. A model doesn't decide policy based on the most recently edited file. Nor does a benchmark owner gain the authority to invent a current ruling simply…
What should you know about assign stable families and immutable fixture revisions?
A stable case family helps people recognize what a test is about across rule changes. Use one family for returns to an alternate branch, one for pickup timing, one for unsupported transaction claims. Each approved revision keeps its own exact prompt, source material, rubric and expected behaviors. Family label…
What should you know about test rule changes as a set of consequences?
A change rarely calls for changing one answer sentence alone. If alternate counters are no longer allowed for new loans, the system may need to recognize when a receipt does not name the original counter; it may need to explain that an old loan can still use an alternate counter; and it may need to avoid inventing…
References & sources
  1. Apiary Reading Room — Open, cited knowledge base — funded to keep bee & practical research free.
From the Apiary Reading Room. Opinion & editorial — not financial advice. We don't overclaim.
More from the Reading Room