ApiaryActiveLive
Try: pause · settings · learn · wipe
← Community / Reading Room
WS
NothingUnseen · 26 min read

When Surveillance Statistics Count Queries, Not People: Reading FISA Transparency Reports

A surveillance statistic can be accurate and still be used to make an inaccurate claim. A count of search terms is not automatically a count of people. A…

How to compare published surveillance numbers without turning identifiers into people, decisions into applications, or reporting bands into exact totals. All illustrative datasets and organizations below are hypothetical unless identified as a published source. This article examines dated reports and counting methods, not the current status of every surveillance authorization or the legality of an individual search. Sources checked October 30, 2026.

1. The number has a unit before it has a meaning

A surveillance statistic can be accurate and still be used to make an inaccurate claim. A count of search terms is not automatically a count of people. A count of legal orders is not necessarily a count of accounts. A report of court actions in one year need not match the number of applications first filed during that year. The arithmetic becomes useful only after those distinctions are fixed.

The April 2026 Annual Statistical Transparency Report for calendar year 2025 illustrates the problem. Its FBI table lists 7,413 U.S.-person queries for December 2024 through November 2025, compared with 5,518 for the preceding reporting period. The accompanying methodology describes unique query terms, not a headcount of Americans or every repeated search execution. The report's calendar-year title does not erase that table's different date range. ODNI's calendar-year 2025 report.

The two published values differ by 1,895. Dividing that difference by 5,518 gives an increase of approximately 34.3 percent in the stated measure. Calling it a 34.3 percent increase in people searched would add a conversion the table does not supply. Calling it a 34.3 percent increase in repeated query events would make a different unsupported conversion. A carefully worded comparison is less dramatic but more informative because it preserves what the report actually counts.

There is no implication that the underlying privacy questions disappear when terminology becomes precise. People may reasonably disagree about which surveillance powers should exist, what safeguards they need and whether published reporting is sufficient. Those disagreements are better served by evidence that does not change units halfway through an argument.

The method developed here is a reader's ledger. For each statistic, record its unit, population, period, counting rule and uncertainty. Then decide whether a proposed comparison is valid, needs adjustment or cannot be made from public information. The examples use small invented datasets so that the operations are visible. They do not reconstruct classified holdings, estimate undisclosed individuals or claim to audit an intelligence agency.

2. Separate collection, search and disclosure

A piece of information can pass through several stages: acquisition into a repository, a query against that repository, review of a returned result and disclosure in another product or process. Counts taken at different stages answer different questions. Adding them together does not produce a meaningful total merely because all of them concern surveillance.

ODNI's March 17, 2026 fact sheet describes a query as using terms to retrieve relevant information from acquired intelligence. It gives examples of U.S.-person query terms, including an address, telephone number or company name. That description helps distinguish a search operation from a fresh collection order; it does not tell a reader how many records a particular search returned or who ultimately reviewed them. ODNI's dated query explanation.

Imagine a fictional repository containing twelve messages. Analyst A runs a query using identifier T1 and receives four matches. Analyst B later runs T1 and receives the same four. The example contains twelve stored messages, two query events, one distinct query term and four distinct matched messages, assuming the results truly overlap completely. If the analysts open only three of those messages, the count of reviewed messages is different again.

Now suppose a later report incorporates information from two of the messages. That does not create two new collection events, nor does it mean only two messages were available in the repository. Each stage needs its own definition. If the public record supplies only query-term counts, the other stage counts remain unknown rather than implicitly equal to the known value.

This distinction matters when assessing safeguards. A rule about which identifiers may be used is different from a rule about retaining acquired information or distributing a report. Evidence that one stage is controlled does not prove that every other stage is controlled. Conversely, the existence of several stages does not itself establish that a safeguard was violated.

A useful explanatory graphic would draw the stages and label the metric measured at each one. It should not draw arrows with numerical conversion factors unless the source establishes those relationships. A single order can cover a different unit than a single query, and a single message may mention multiple entities. The honest diagram sometimes contains blank quantities. Those blanks identify questions the available data cannot answer.

3. Count events and distinct terms in the same small ledger

Consider six invented search events. The terms used, in order, are T1, T1, T2, T3, T1 and T2. An event counter reports six. A counter of distinct terms reports three. Neither is wrong. They measure different features of the same ledger, and the difference is not evidence that someone concealed three searches.

Add a seventh event using T1. The event count rises from six to seven while the distinct-term count remains three. Add an eighth event using T4 and the two counts become eight and four. A graph of distinct terms alone would miss the extra repetition, while a graph of events alone would not show whether activity expanded to new identifiers. Both can be relevant, but they are not substitutes.

Approval is another distinct state. The ODNI report explains that NSA's content-query statistic counts approved identifiers, while CIA and NCTC count identifiers used. An approved term need not ultimately be used. The report also distinguishes term-based content measures from event-based noncontent measures. These methodological differences should travel with any comparison rather than disappear in a simplified chart. ODNI's counting-method explanation.

In a separate hypothetical approval ledger, four terms receive approval but only three are executed. The approval count is four and the distinct-used count is three. If the fourth is used next month, the approval and execution dates may fall into different reporting periods. That timing difference is a reason to preserve event dates, not to label the original approval record an error.

Repeated use can also be concentrated. One toy system executes each of four terms once; another executes one term ninety-seven times and three others once each. Both have four distinct terms, but their event totals are four and one hundred. The second system has more repetition, not necessarily more identifiable people. Without the term-to-person relationships, even an exact event ledger cannot supply a reliable headcount.

For a public summary, write the unit explicitly: distinct approved terms, distinct used terms or executed queries. Avoid the unqualified word “searches” when the source uses a more specific definition. This is not pedantry. It determines whether the reported number changes when an analyst reruns an existing term, when a new term is approved but unused, or when two operators use the same identifier.

4. Identifiers do not come with a one-person guarantee

A person may use several accounts or telephone numbers, and an account or organizational identifier may be associated with more than one person. A name may also be ambiguous. Therefore a count of identifiers does not generally impose a simple one-to-one relationship with individual humans. The translation requires information about identity resolution and the scope of the source's definitions.

In a fictional dataset, T1 and T2 belong to Person A, while T3 belongs to Person B. Three terms then correspond to two known individuals. If T3 is instead a shared organizational address, the number of people associated with it could differ. If the dataset does not resolve these relationships, the analyst cannot choose the convenient interpretation and present it as a measured fact.

This is especially important when a term such as U.S. person has a defined scope that may include entities rather than only individual citizens. ODNI's March 2026 fact sheet lists citizens and lawful permanent residents alongside qualifying corporations and unincorporated associations. A label in that legal framework should not be casually rewritten as a count of individual Americans. ODNI's U.S.-person definition summary.

There are two different uncertainty problems. One concerns unresolved links among terms: perhaps two addresses belong to the same entity. The other concerns the meaning of the counted entity itself: perhaps the term refers to an association rather than a natural person. Combining those uncertainties into a single estimated headcount requires assumptions that should be stated and justified, not hidden behind a precise-looking total.

A hypothetical sensitivity analysis can illustrate the first problem without estimating a real population. Suppose ten terms are known to form four pairs plus two singletons, with each pair assigned to one individual and no links between groups. Under those explicit assumptions, there are six individuals. If the pairs are merely suspected rather than established, six is a scenario result, not an observed count. If shared identifiers are possible, the assumptions need further revision.

The best public explanation may therefore retain the identifier count and explain why a person count is unavailable. That limitation is meaningful: it describes the resolution of the evidence. It is not proof that the underlying population is either tiny or enormous. A serious privacy debate should distinguish a known measure from the different measure it wishes could be obtained.

5. Deduplication needs a boundary

“Duplicates removed” sounds complete, but it raises another question: duplicates within what boundary? A system might deduplicate within a day, a month, an agency or a year. The same identifier can count once in each smaller unit and more than once when those units are added together. Deduplication is an operation on a specified set, not a permanent property of a number.

Use two fictional months. January contains seventy distinct terms and February contains sixty. Thirty terms occur in both. The two-month union is one hundred: seventy plus sixty minus thirty. Adding the published monthly totals gives 130 term-months, not 130 distinct terms across the combined period. Without the overlap count, the union could range from seventy to 130 under this simple set model.

An agency boundary creates the same issue. Fictional Agency A uses seventy distinct terms and Agency B uses fifty. If twenty are shared, the cross-agency union is one hundred rather than 120. If the agencies use incompatible normalization rules, even identifying the overlap may require more than matching displayed strings. A count of distinct terms within each agency is not automatically deduplicated across agencies.

Normalization should be documented as part of the method. Imagine that a system treats two formatting variants of a telephone number as the same term, while another counts them separately. Their totals can differ without a difference in underlying intended contacts. Conversely, aggressively merging similar-looking strings could join identifiers that should remain separate. A reader should not assume that either raw string matching or more elaborate identity resolution is always correct.

The DOJ Inspector General's October 2025 querying review explicitly notes a counting distinction relevant here: its discussion of an NSD review population used a non-deduplicated figure, while FBI figures supplied for ODNI's annual transparency reporting remove repeated uses of the same term. That footnote is a reason not to divide one report's incident count by another report's denominator without checking compatibility. DOJ OIG report 26-002, counting-method footnote.

The practical rule is to preserve the original count and add a field for the deduplication boundary. If the boundary changes, show a break in the series or supply a genuinely comparable reconstruction. Do not smooth the change away because the resulting line looks inconvenient. A methodological discontinuity can be an explanation, a limitation or both, without implying that the old or new statistic was fraudulent.

6. A reporting year is not always a calendar year

A report's title, publication date and observation period are three separate dates. A document published in April can describe activity ending the previous December, or an earlier November, or a rolling review period. Comparing numbers by the year printed on the cover can accidentally align populations that overlap only partly.

The difference is easy to see with hypothetical event counts. Suppose a December-through-November reporting window contains 1,200 events. To convert that event total to January-through-December of the later year, an analyst would subtract the first December and add the later December, provided all three figures use the same event definition. If those months contain eighty and one hundred events respectively, the adjusted total is 1,220.

That simple subtraction does not generally work for distinct terms. Removing a month does not remove every term used in that month, because some may also appear later. Adding a month does not add every term used in it, because some may already be in the window. Reconstructing a unique-term total requires the relevant sets or enough overlap information, not just monthly totals.

For example, let an eleven-month core contain ninety distinct terms. The earlier December contributes twenty terms, ten of which are already in the core. The first annual window therefore contains one hundred terms. The later December contributes thirty terms, twenty-five already in the core. The shifted window contains ninety-five terms. A naive calculation of one hundred minus twenty plus thirty would produce 110, which is wrong for this invented dataset.

Publication delay creates a different issue. A late report may be the newest available source while still describing an older period. “Latest report” should not be rewritten as “current activity.” If an organization changes a rule after the observation window, the published counts cannot by themselves establish the effect of that later change.

A useful chart label therefore includes the actual start and end dates, not just a short year tag. Its notes should identify any partial-year categories and any change in the start of data collection. A blank before tracking began is not automatically zero. A category introduced halfway through a year may provide a valid partial-period count while remaining unsuitable for a full-year comparison without additional information.

7. Court applications and court actions form different ledgers

The Administrative Office of the U.S. Courts' report for 2025 states that the FISC received 287 applications and certifications. It lists 179 grants, ninety-one modifications, fifteen partial denials and four full denials, and explains that final action also occurred on two submissions from 2024. Those action categories total 289. The apparent mismatch is not resolved by silently changing a number; the report supplies a carryover explanation. AO's report on FISA court activities for 2025.

A hypothetical docket makes the timing principle clearer. A court begins a year with five pending matters, receives one hundred new matters and finishes with eight pending. If there are no other adjustments and each completed matter receives one final disposition, ninety-seven matters were completed: five plus one hundred minus eight. The receipt count and disposition count are different even in a perfectly reconciled system.

The definitions of disposition matter too. Granting a request with substantive modifications is not the same category as granting it unchanged. A partial denial is not identical to a full denial, and neither necessarily means an entire investigation ended. A reader should preserve the report's categories rather than force them into a single yes-or-no tally without explaining the transformation.

There is also a pre-filing stage. DOJ's April 29, 2026 annual FISA report describes 254 final applications for electronic surveillance and/or physical searches in 2025 and contrasts final-filed statistics with the AO's proposed-application methodology. It explains that changes or withdrawals can occur between proposed and final submissions. Those are different reporting lenses on a process, not interchangeable denominators. DOJ's calendar-year 2025 FISA report.

For the hypothetical docket, imagine that ten proposals are substantially revised before ninety final requests are filed. A statistic limited to decisions on the final requests will not, by itself, measure the amount of earlier judicial scrutiny. Nor does the mere existence of a proposal stage prove that scrutiny was adequate. The question requires evidence about the stage being evaluated.

A responsible summary can report how often a particular category of final request was granted while acknowledging what that rate excludes. It should not turn an application statistic into a count of people surveilled, or treat an order's existence as proof that every later implementation complied with it. Those are separate questions requiring different records.

8. A reporting band is an interval, not a hidden exact answer

Some provider transparency reports publish ranges rather than exact counts. A range communicates a bound under a reporting rule. It does not authorize a reader to replace the interval with its midpoint and treat the result as observed data. The midpoint may be useful in an explicitly labeled scenario, but it remains an assumption.

Apple's U.S. transparency page reports national-security requests and affected users/accounts separately. For the second half of 2025, its FISA-content row lists 500–999 requests and 77,500–77,999 users/accounts. Apple explains that its separate FISA content and noncontent reporting uses bands of five hundred and a six-month delay. These are Apple's reported categories and method, not exact counts of distinct humans. Apple's U.S. national-security reporting table and definitions.

Use invented values for a range exercise. Provider P reports between one thousand and 1,499 requests in one half-year and between 1,500 and 1,999 in the next. If these are disjoint reporting periods with a consistent event-counting rule, the annual request total lies between 2,500 and 3,498. There is no need to assume a midpoint to obtain those bounds.

If the figures instead count distinct accounts within each half-year, adding them gives account-period appearances. It does not establish the number of distinct accounts across the year. The same account may occur in both halves. Under the simplified assumption that each period's count is itself a distinct-account count, the annual union could be as low as 1,500 or as high as 3,498. The public intervals alone do not identify the overlap.

Comparing intervals also requires restraint. Suppose one period's exact count is somewhere from five hundred to 999 and the next is somewhere from one thousand to 1,499. The increase is at least one and at most 999. That establishes an increase in the measured count under the stated assumptions, but it does not establish a particular percentage increase. Choosing both midpoints would produce a convenient number that the reports did not publish.

Zero deserves special care. A published interval beginning at zero permits zero, but does not prove zero. Conversely, an interval whose lower bound is positive does establish that the reported count is nonzero, subject to the report's own definitions. A public explanation should say which of those claims the evidence supports instead of treating all broad bands as either useless or exact.

The correct response to coarse reporting is to preserve the interval and identify the precision it prevents. Uncertainty is information about the limits of the record. It should not be disguised as a fabricated exact total.

9. Purpose categories need not partition the whole

Readers often assume that the rows beneath a total are mutually exclusive categories that add up to it. Sometimes they are; sometimes they describe overlapping attributes, different periods or different exclusions. A residual calculated from incompatible rows can look like a missing population even when the table never promised an exhaustive breakdown.

The ODNI report's new FBI purpose-related estimates include footnotes explaining exclusions and the January 1, 2025 start of tracking. Some entries omit test queries or cases in which the user was uncertain about a protective purpose. Those qualifications limit what can be inferred by subtracting the displayed categories from a broader total. ODNI's purpose-category footnotes.

Consider an explicitly hypothetical set of one thousand records. Four hundred carry label A and six hundred carry label B. If 150 carry both labels, the union is 850 and 150 carry neither. Adding A and B gives one thousand, but that numerical equality does not prove that the two labels partition the population. The overlap and the unlabeled remainder happen to cancel in the sum.

Now suppose the labels refer to status at different times. A record may be classified one way when created and differently after review. A table of initial labels and a table of final labels should not be combined as if they describe simultaneous attributes. A transition matrix, if available, would answer a more precise question: how many records moved from each initial state to each final state?

For a toy transition, start with eighty records marked uncertain. Review assigns fifty to category A, twenty to category B and leaves ten unresolved. Those numbers reconcile to eighty. They do not show that the original eighty were improperly classified; uncertainty may have been the correct state before additional evidence arrived. Nor should the ten unresolved cases be silently assigned to whichever category makes a chart complete.

The public reader should look for three declarations: whether categories overlap, whether they are exhaustive and whether they share the same observation period. If any declaration is missing, label the proposed subtraction as unsupported rather than manufacturing a residual explanation. A category can be useful even when it is not part of a complete partition.

This approach also avoids moral labels unsupported by the table. A purpose category is not necessarily a finding about a person's behavior, and an uncertain classification is not proof of wrongdoing by either the subject or the analyst. The numerical task is to describe what the categories establish before drawing broader conclusions from them.

10. A compliance percentage has a scope and a rule

A compliance rate requires a numerator, a denominator, a review criterion and a period. Change any one of them and the rate can change without a contradiction. A percentage for a particular query standard is not necessarily a percentage for compliance with every procedural requirement. An inspected sample is not automatically the complete universe of activity.

The DOJ OIG's October 2025 report explains that its analysis relied on NSD's legal determinations and data rather than independently evaluating individual queries. It also distinguishes compliance with the query standard from compliance with other procedural requirements. That methodological statement is important when describing who made a determination and what the resulting rate measures. DOJ OIG's review method and compliance distinctions.

In a hypothetical review of one thousand events, suppose twenty fail a substantive criterion, thirty fail a procedural criterion and ten fail both. Forty distinct events fail at least one criterion, so overall compliance with both is 96 percent. Compliance with the substantive criterion alone is 98 percent. Both rates are arithmetically correct, but only if the labels identify the criterion. Adding twenty and thirty without subtracting the overlap would overcount failures.

A second hypothetical shows why percentages should travel with counts. One reviewed group has one failure among one hundred events, a 1 percent failure rate. Another has ninety failures among nine thousand events, also 1 percent. The rates match, but the number of affected events and the review workloads differ. Neither quantity alone describes the consequences or significance of the failures.

Review design matters as much as the arithmetic. If an oversight team deliberately selects high-risk cases, its observed failure rate should not be generalized to every case without an appropriate design. If it reviews only cases correctly labeled as belonging to a protected category, classification errors outside that category may require a separate examination. These are general measurement issues, not allegations about an undisclosed current agency practice.

A useful public statement might say that a specified reviewer found a stated number of events noncompliant with a named requirement in a defined review population. It should then explain whether the population was exhaustive, randomly sampled or selected for another reason. The statement can acknowledge serious findings without inflating their scope, and it can acknowledge improvement without treating a favorable percentage as proof that every possible problem has disappeared.

The distinction between measurement and judgment is especially important here. The count establishes how the reviewed records were classified under the stated method. Deciding whether the safeguards are adequate, whether remedies were sufficient or whether the underlying authority should continue requires additional legal and policy reasoning that the percentage does not perform by itself.

11. Identify the author and status of the report

Documents carrying official seals can still have different institutional statuses. A staff analysis, an inspector-general review, an agency explanation, a court opinion and a statute are not interchangeable. They can all contribute evidence, but their roles should remain visible when their conclusions are summarized.

The April 2, 2026 PCLOB publication is expressly a staff report. Its opening footnote says that a quorate Board approved the project's initiation in December 2024, but the resulting report was not voted on or approved by a quorate Board. That distinction does not make the staff analysis worthless; it means readers should attribute it accurately rather than describe it as a new unanimous Board judgment. PCLOB's April 2026 staff report and status footnote.

A hypothetical source ledger could therefore contain separate fields for issuing institution, document type, authoring body, approval status, publication date and period examined. Those fields make it harder to promote a proposal into law or a staff assessment into a judicial holding. They also help readers understand why two documents may emphasize different questions without necessarily contradicting each other.

Status should not be used as a shortcut for accepting or dismissing factual content. A staff report may present carefully documented facts; a formally adopted statement may still contain judgments that deserve scrutiny. The reader should inspect the evidence supporting the particular claim, while accurately identifying who is responsible for it.

Dates matter here too. A report written before a policy change cannot evaluate the change's later effects merely because it remains available on a current website. A page's recent crawl or update date is not necessarily the date of the analysis. Conversely, an older methodological explanation can remain relevant to an unchanged counting rule, provided the current source confirms that relationship.

For an article comparing surveillance statistics, the practical benefit is traceability. A reader should be able to move from a sentence to the exact report, then to the table, footnote or methodological passage supporting it. That chain is more useful than a long bibliography whose entries do not correspond to specific claims. Attribution is part of the evidence, not a decorative addition after the argument is finished.

12. Do not combine every surveillance report into one grand total

Different transparency series can describe different authorities, stages and populations. The fact that two reports concern electronic surveillance does not make their numbers additive. A grand total assembled without a common unit may double-count activity, mix incompatible processes or imply coverage of categories that neither report measures.

The federal judiciary's Wiretap Report 2025, for example, describes wiretaps concluded during 2025 that were properly reported to the Administrative Office and includes supplementary information about outcomes from earlier years. Its coverage statement and reporting limitations are part of the meaning of its numbers. That series should not be treated as another column of the FISA query table merely because both are surveillance-related. The judiciary's Wiretap Report 2025 coverage statement.

Imagine three fictional public records: a court report counts eighty orders, an agency report counts two hundred query terms and a provider report counts three hundred affected accounts. Adding them yields 580 only in the arithmetic sense. It does not produce 580 people, investigations or surveillance events. There is no common unit and no established overlap relationship.

Even reports using the same noun may differ. One provider could count accounts for which data was produced, while another counts accounts specified in received demands. The second population may include requests that were rejected or never produced data. A comparison requires the providers' definitions, not just matching column headings.

GAO's data-reliability guidance identifies accuracy, completeness and applicability to the audit's purpose as separate considerations. The last is particularly important here: a dataset may be accurate for its stated purpose yet unsuitable for the question a reader wants to answer. Reusing a valid number does not automatically validate a new conclusion. GAO's data-reliability guidance.

A better comparative display preserves the separate series and explains their relationships in words. It might say that one measures legal process, another measures activity within a repository and another measures provider responses. Where a common definition is genuinely available, calculate a comparison. Where it is not, mark the series as noncomparable for that particular operation.

This is not an argument against synthesis. It is an argument for synthesis that respects the evidence's structure. A reader can learn a great deal from the coexistence of different reporting systems without pretending that their totals form a comprehensive census of surveillance or of every person whose information may have been involved.

13. Build a small reconciliation that another reader can repeat

A transparent analysis should preserve enough intermediate work for someone else to check it. For public reports, that usually means a compact extraction ledger rather than a private claim that the tables were reviewed. The ledger should record the source location, exact unit, period, qualifiers and any calculation applied to the published value.

Consider a fictional newsroom comparing two annual releases. It copies a row labeled distinct used terms from each report and initially finds a rise from eight hundred to nine hundred. Before publishing a 12.5 percent increase, the reviewer notices that the earlier report covers ten months because tracking began late. The arithmetic is correct for the two displayed totals, but the interpretation as an annual rate increase is unsupported.

The newsroom has several honest options. It can compare the reported totals while making the unequal periods prominent. It can seek a compatible ten-month subset from the later period. Or it can decline a trend claim until comparable data is available. Multiplying the first total by twelve-tenths would create an annualized scenario, not an observed annual count, and would assume a pattern across the missing months that the evidence may not support.

Now add a revision. The earlier source is corrected from eight hundred to 840 because a reporting category was reassigned. The extraction ledger should retain the original value, the revised value, the date of the correction and the reason supplied by the publisher. The new comparison is approximately 7.1 percent, but only for the displayed totals and with the same period limitation. A silent replacement would make it difficult for readers to understand why the published analysis changed.

A minimal computational check can test the formulas without claiming to validate the underlying records. It can confirm that 900 divided by 800 minus one is 12.5 percent, and that 900 divided by 840 minus one is about 7.1 percent. It cannot prove that either source counted correctly, that the records were complete or that a legal classification was appropriate. Those require different evidence.

For the actual published reports discussed here, the same discipline means preserving the table's footnotes and observation window alongside its numbers. A screenshot or copied row without its definitions may be an incomplete extraction. When an apparent discrepancy remains unresolved, record it as unresolved and explain which document or clarification would be needed. Do not repair a public record by inventing a missing category or an undisclosed event.

14. Make the conclusion as narrow as the evidence requires

After the units, periods and methods are aligned, a useful conclusion may still be narrower than the headline a reader expected. That is not failure. It is the difference between learning something from a report and using a report as a prop for an already chosen story.

Suppose a hypothetical analysis establishes that distinct terms increased, repeated events decreased and the number of associated individuals is unavailable. The correct summary contains all three statements. It does not select one trend and let the audience assume it describes the others. The combination may suggest more breadth with less repetition, but even that interpretation depends on the definitions and coverage being comparable.

A second example concerns oversight. Suppose a defined review finds fewer failures under an unchanged rule, while a separate category has incomplete data. The improvement can be reported without pretending the incomplete category is clean. The limitation can be reported without denying the measured improvement. Evidence-based criticism and evidence-based acknowledgment use the same discipline.

The same restraint applies to intent. A statistical increase does not establish a motive, and a discrepancy between two reporting systems does not establish deception. An unexplained difference warrants examination. A documented change in scope may resolve it. If evidence later supports a stronger conclusion, that conclusion should be tied to the new evidence rather than retroactively smuggled into the original count.

For readers, the most productive questions are concrete. What exactly was counted? What period does it cover? Are repeated terms removed, and within which boundary? Does the figure describe approvals, executions, returned records or disclosures? Do categories overlap? Is a value exact, estimated, banded, missing or not applicable? Which institution made the underlying determination, and what did its review actually examine?

For publishers, the obligation is equally concrete: preserve the answers beside the numbers. A chart should not require readers to discover, several links away, that its apparent people count is a term count or that its annual series changes definitions halfway through. Where the source does not answer a question, the article should say so rather than fill the gap with a plausible story.

Transparency reporting cannot settle every dispute about surveillance. It can, however, make public reasoning more precise. The goal is not to make the numbers look better or worse. It is to keep their units, limits and provenance intact so that disagreements about power, privacy and accountability rest on what the record actually establishes.

Frequently asked
What is When Surveillance Statistics Count Queries, Not People: Reading FISA Transparency Reports about?
A surveillance statistic can be accurate and still be used to make an inaccurate claim. A count of search terms is not automatically a count of people. A…
What should you know about 1. The number has a unit before it has a meaning?
A surveillance statistic can be accurate and still be used to make an inaccurate claim. A count of search terms is not automatically a count of people. A count of legal orders is not necessarily a count of accounts. A report of court actions in one year need not match the number of applications first filed during…
What should you know about 2. Separate collection, search and disclosure?
A piece of information can pass through several stages: acquisition into a repository, a query against that repository, review of a returned result and disclosure in another product or process. Counts taken at different stages answer different questions. Adding them together does not produce a meaningful total merely…
What should you know about 3. Count events and distinct terms in the same small ledger?
Consider six invented search events. The terms used, in order, are T1, T1, T2, T3, T1 and T2. An event counter reports six. A counter of distinct terms reports three. Neither is wrong. They measure different features of the same ledger, and the difference is not evidence that someone concealed three searches.
What should you know about 4. Identifiers do not come with a one-person guarantee?
A person may use several accounts or telephone numbers, and an account or organizational identifier may be associated with more than one person. A name may also be ambiguous. Therefore a count of identifiers does not generally impose a simple one-to-one relationship with individual humans. The translation requires…
References & sources
  1. Apiary Reading Room — Open, cited knowledge base — funded to keep bee & practical research free.
From the Apiary Reading Room. Opinion & editorial — not financial advice. We don't overclaim.
More from the Reading Room