Data Management Plans

Your roadmap to FAIR‑compliant, future‑proof research data—whether you’re tracking honeybee health, training a self‑governing AI, or anything in between.

Introduction

In today’s data‑driven research ecosystem, a dataset is no longer a by‑product of an experiment; it is a first‑class research output. Funding agencies such as the NSF, the European Commission, and the UKRI now require a Data Management Plan (DMP) as part of every grant proposal. Yet many investigators treat the DMP as a bureaucratic checkbox rather than a living document that safeguards the integrity, reproducibility, and impact of their work.

For bee conservationists, this shift is especially critical. The global decline of pollinators has been quantified in over 1,300 peer‑reviewed studies since 2000, with the Food and Agriculture Organization estimating a 35 % loss of pollinator diversity in some regions. Robust, reusable datasets—ranging from hive temperature logs to landscape‑scale pollen flow maps—are the foundation for actionable policy and for AI agents that can autonomously monitor hives, predict disease outbreaks, or optimize planting strategies.

At the same time, the rise of self‑governing AI agents (e.g., autonomous swarm drones that pollinate or monitor apiaries) introduces new data stewardship challenges: provenance of training data, model versioning, and audit trails must be documented as rigorously as any field observation. A well‑crafted DMP ensures that every byte, from raw sensor streams to trained model weights, remains FAIR—Findable, Accessible, Interoperable, and Reusable—throughout its lifecycle.

This guide walks you through the essential components of a FAIR‑compliant DMP, provides concrete tools and examples, and shows how good data stewardship amplifies the impact of bee research and AI‑driven conservation.

1. Understanding the Data Management Plan (DMP)

A Data Management Plan is a concise, structured document that answers four fundamental questions about a research project’s data:

The DMP as a living contract

Unlike a static project proposal, a DMP should be revisited at least quarterly. Changes in sensor firmware, new data standards, or unexpected legal constraints (e.g., GDPR for location data) may necessitate updates. Treat the DMP as a contract between you, your institution, funders, and—crucially—future users of your data.

Aligning with funder expectations

By mapping each funder’s checklist to explicit actions in your DMP, you avoid last‑minute scrambling and demonstrate responsible stewardship.

2. The FAIR Principles in Practice

FAIR is more than a buzzword; it is a measurable set of criteria that can be evaluated with tools such as FAIRshake or FAIR Evaluator. Below we translate each principle into concrete steps for a bee‑AI project.

Findable

• Persistent identifiers (PIDs): Assign a DOI to every dataset version via Zenodo or DataCite. For model artifacts, use a Software Heritage identifier.
• Rich metadata: Include fields like taxonID, samplingProtocol, sensorModel, and modelArchitecture. Use controlled vocabularies (e.g., GBIF vocabularies for species).
• Indexing: Register the dataset with domain repositories (e.g., GBIF for biodiversity occurrences, OpenAIRE for European projects) so that it appears in search engines and data catalogs.

Accessible

• Standard protocols: Serve files over HTTPS with OAuth2 or API keys for controlled access.
• Authentication & authorization: For sensitive location data, implement tiered access—public summary statistics, restricted raw GPS logs.
• Data use statements: Clearly state the license (e.g., CC‑BY‑4.0 for open data, CC‑BY‑NC‑ND for restricted data) and any embargo periods.

Interoperable

• Standard formats: Store sensor data as Parquet or NetCDF (both columnar and self‑describing). Video should be in MP4 (H.264) with embedded metadata. Model weights in ONNX allow cross‑framework use.
• Ontologies: Link terms to the Environment Ontology (ENVO), Bee Ontology (BEO), and AI Ontology (AI-O) where applicable.
• APIs: Provide a RESTful endpoint that returns JSON‑LD, enabling machines to automatically harvest the data.

Reusable

• Clear provenance: Record the exact firmware version of each sensor, calibration dates, and any preprocessing scripts (e.g., Python notebooks).
• Licensing: Use machine‑readable licenses (RDFa) so that downstream users can programmatically verify reuse rights.
• Quality assurance: Include data quality metrics—e.g., missing‑value rate, signal‑to‑noise ratio—alongside the dataset.

Metric example: In a 2023 audit of 1,200 ecological datasets, only 42 % satisfied the “interoperable” criterion, mainly because of proprietary file formats. By committing to open standards, your project can join the top‑tier 58 % that are truly reusable.

3. Planning for Data Collection & Storage

Estimating volume

Takeaway: Even a modest network of 200 hives can generate ≈10 TB of raw data over two years. Budget accordingly and plan for redundancy.

Storage hierarchy

1. Hot storage – SSD‑based NAS for active analysis (≈1 TB).
2. Warm storage – Cloud object storage (AWS S3 Standard) for data that is accessed monthly (≈5 TB).
3. Cold storage – AWS S3 Glacier Deep Archive or Google Cloud Archive for long‑term preservation (≥10 TB, < $0.001/GB/month).

Implement checksum verification (e.g., SHA‑256) after each transfer. Tools like rclone and AWS CLI can automate nightly syncs and generate logs for audit.

Backup & disaster recovery

Maintain immutable snapshots (e.g., S3 Object Lock) to protect against tampering.

4. Documentation & Metadata Standards

Choosing a metadata schema

You can embed multiple schemas in a single JSON‑LD file, linking them via @id relationships. For example, a HiveObservation record can reference a SensorML description of the temperature logger and a ML‑Schema entry for the disease‑prediction model that consumes the same data.

Provenance with W3C PROV

Documenting who, what, when, and how a dataset was generated is essential for reproducibility. Use the PROV-O ontology to capture:

{
  "@context": "https://w3id.org/prov/v1",
  "entity": {
    "ex:hiveLog_2024_03_15": {
      "prov:label": "Temperature log for Hive #42 on 15‑Mar‑2024",
      "prov:type": "ex:SensorLog",
      "prov:wasGeneratedBy": "ex:acquisitionActivity_2024_03_15"
    }
  },
  "activity": {
    "ex:acquisitionActivity_2024_03_15": {
      "prov:startTime": "2024-03-15T00:00:00Z",
      "prov:endTime": "2024-03-15T23:59:59Z",
      "prov:used": "ex:sensorFirmware_v2.1"
    }
  },
  "agent": {
    "ex:fieldTechnician_JL": {
      "prov:type": "prov:Person",
      "prov:actedOnBehalfOf": "ex:UniversityOfX"
    }
  }
}

Such provenance graphs can be visualized with tools like ProvVis and are searchable in repositories that support PROV queries.

Documentation best practices

• README.md – one‑page overview, data dictionary, and contact information.
• CODE_OF_CONDUCT.md – expectations for community contributions.
• LICENSE – plain‑text license file; link to SPDX identifier (e.g., CC-BY-4.0).
• Data processing scripts – store in a version‑controlled repository (GitHub, GitLab) and reference the commit hash in the metadata.
• Data quality report – generate automatically with Great Expectations or Pandera, summarizing missing values, outliers, and validation failures.

5. Legal, Ethical, and Licensing Considerations

GDPR and location privacy

Bee‑related GPS data can inadvertently reveal private land ownership. To stay GDPR‑compliant:

1. Pseudonymize exact coordinates to a 100 m grid for public release.
2. Store the raw coordinates in a restricted-access repository, with access granted only to approved collaborators under a Data Use Agreement (DUA).
3. Provide a data protection impact assessment (DPIA) as an appendix to the DMP.

Intellectual property for AI models

When a model is trained on a mix of open and proprietary data, the resulting weights inherit the most restrictive license. A common approach:

• Open‑source code under MIT or Apache‑2.0.
• Model weights under CC‑BY‑4.0 if all training data are open; otherwise, use a custom license that restricts commercial use (e.g., CC‑BY‑NC‑ND).

Document the provenance of each training subset in the metadata so downstream users can assess licensing compatibility.

Ethical AI for bee health

Self‑governing AI agents that make decisions about hive treatment (e.g., applying miticides) must be auditable. Include in the DMP:

• Decision‑log schema – timestamp, sensor input, model confidence, action taken.
• Human‑in‑the‑loop policy – thresholds where the system must request beekeeper approval (e.g., confidence < 80 %).
• Bias assessment – evaluate whether training data over‑represent certain climates, which could lead to misdiagnosis in under‑sampled regions.

6. Sharing Strategies & Repository Choices

General‑purpose vs. domain‑specific repositories

Rule of thumb: Deposit raw, unprocessed data in a domain‑specific repository (e.g., GBIF for occurrence data) and derived products (aggregated statistics, model weights) in a general‑purpose or AI‑focused repository. This dual‑deposit strategy maximizes discoverability and ensures compliance with both biodiversity and AI community standards.

Embargoes and controlled access

A typical timeline:

Document this schedule in the DMP and include the embargo expiration date in the metadata (prov:endedAtTime).

Machine‑readable data packages

Package datasets using BagIt or RO-Crate to bundle data files, metadata, and checksums. Example command:

# Create a BagIt bag
bagit.py create my_hive_dataset/
# Add metadata
echo '{"@context":"https://w3id.org/ro/crate/1.1/context","@graph":[...]}'> my_hive_dataset/metadata.json

These packages are ingestible by most repositories and enable automated validation pipelines.

7. Long‑Term Preservation & Sustainability

Bit‑rot and format migration

Even well‑preserved data can become unreadable if the file format falls out of support. Mitigation steps:

1. Choose archival formats: CSV for tabular data (UTF‑8), Parquet for columnar, NetCDF‑4 for multidimensional arrays, MP4/H.264 for video, ONNX for models.
2. Periodic validation: Run a checksum audit annually and verify that the file can be opened with the latest software version.
3. Migration plan: Allocate a 5 % annual budget line for format migration (e.g., converting legacy .txt logs to Parquet).

Institutional commitment

Many universities have a Digital Preservation Office that guarantees a minimum of 10 years of preservation for datasets with a DOI. Include the institution’s policy link (e.g., [[institution-preservation-policy]]) in the DMP and note any cost‑sharing arrangement.

Funding for preservation

If the total projected storage cost is $0.023 per GB per month (AWS Glacier Deep Archive 2024 pricing), a 10‑year preservation of 10 TB costs roughly:

10,000 GB × $0.023/GB/mo × 12 mo × 10 yr = $27,600

Factor this amount into the budget justification section of the DMP. Many funders allow a sustainability line item up to 15 % of the total grant budget.

8. Managing Interdisciplinary Projects: Bees + AI Agents

Data governance matrix

A RACI chart (Responsible, Accountable, Consulted, Informed) can be embedded in the DMP to clarify who updates which metadata fields.

Example workflow

1. Data ingestion – Sensors push JSON payloads to an MQTT broker; a Lambda function writes them to S3 (hot tier).
2. Pre‑processing – A Jupyter notebook (tracked in Git) resamples to 1‑minute intervals, flags outliers, and stores results in Parquet (warm tier).
3. Model training – The AI team pulls the Parquet files, trains a CNN, logs hyperparameters with MLflow, and registers the model in the Hugging Face Hub.
4. Decision support – An edge device at the hive runs the model locally, logs decisions to a PROV‑compatible file, and sends alerts to the beekeeper’s mobile app.
5. Archival – After 30 days, raw JSON logs are moved to Glacier; processed Parquet files and model artifacts are deposited in Zenodo with a DOI.

Each step is described in the DMP with responsible party, timeline, and quality checks.

9. Tools, Templates, and Automation

Sample DMP template (Markdown excerpt)

# Project Overview
- **Title:**