# Build and update datasets Populate fixed historical dataset rows using asynchronous, credit-funded research. Source: Build Dataset populates a list of rows you supply. Update Dataset refreshes explicit historical periods using your existing observations as context. Both accept and return portable JSON: row IDs, units, fiscal periods, values and evidence. They do not create row outlines, forecasts, Excel formulas or workbook layouts. ## Agent skill The **finbar-datasets** skill guides an assistant through preparing inputs, starting paid research, waiting for completion, retrieving all result pages and explaining unresolved or retained values. It supports both MCP and REST. For an agent that supports Agent Skills, save these files in a `finbar-datasets` folder, preserving the paths shown, then install the folder using your client's skill installation process: - [SKILL.md](https://docs.finbar.com/skills/finbar-datasets/SKILL.md) - [references/inputs.md](https://docs.finbar.com/skills/finbar-datasets/references/inputs.md) - [references/rest.md](https://docs.finbar.com/skills/finbar-datasets/references/rest.md) The skill requires a connected Finbar MCP server or authenticated REST access. Connecting the MCP server alone does not install or activate it. The same dataset tools remain usable without a skill. ## Access Dataset runs require `datasets:read`, `entities:read` and `documents:read` permissions. Existing tokens carrying these scopes can be reused. Starting a dataset run consumes AI credits. Availability is controlled by your deployment; a disabled deployment returns `service_unavailable` when starting work. ## Define the dataset First use `search_entity` to obtain the company's `entity_id`. Supply stable row IDs and enough detail to distinguish the intended series: label, description, reporting basis and scope. Each row requires a unit and a positive scale. For example, `unit: "USD"` with `scale: 1000000` returns millions of dollars. For percentages, state whether you want a fraction or a number from 0 to 100. This example uses an illustrative entity ID; replace it with a discovered ID. ```json { "dataset": { "entity_id": "O-11111111111111111111111111111111", "rows": [ { "id": "revenue", "label": "Revenue", "basis": "GAAP", "scope": "Consolidated", "unit": "USD", "scale": 1000000 } ], "observations": [] }, "target_periods": [ {"year": 2025, "period_type": "Y", "period": 0} ], "value_version": "original" } ``` Periods use Finbar's shared [reporting-period definition](https://docs.finbar.com/api-mcp/periods.md): `year`, `period_type` and `period`. For example, Q/2 means the second fiscal quarter, QC/3 means the first nine months, and Y/0 means the full fiscal year. The same target supports balance and flow rows: balances are measured at period-end, while flows cover the specified interval. `value_version` defaults to `original`; choose `latest_restated` explicitly when appropriate. Optional `instructions` can clarify matching and reporting conventions. Build accepts empty observations. Update requires at least one existing numeric observation. Pass your saved dataset and the periods you want researched; existing observations guide matching but are not treated as verified source evidence. Use `status: "provided"` with a numeric value for your own historical inputs. Saved `reported` observations require values and sources; saved `derived` observations require values and calculations. `unresolved` observations require null values and reasons. Missing statuses or contradictory combinations are rejected before work starts. ## Start, poll and retrieve | Action | MCP tool | REST request | | ---------------------------- | -------------------- | ---------------------------------------- | | Populate supplied rows | `build_dataset` | `POST /v1/datasets/build` | | Update existing observations | `update_dataset` | `POST /v1/datasets/update` | | Check progress and usage | `get_dataset_run` | `GET /v1/datasets/runs/{run_id}` | | Request cancellation | `cancel_dataset_run` | `POST /v1/datasets/runs/{run_id}/cancel` | | Retrieve a result page | `get_dataset_result` | `POST /v1/datasets/runs/{run_id}/result` | | Download a JSON bundle | Use result pages | `GET /v1/datasets/runs/{run_id}/bundle` | For REST creation, send the JSON body with `Content-Type: application/json` and an `Idempotency-Key` header. For MCP, add `idempotency_key` to the tool arguments. Build and Update share a key namespace for the same actor across both transports. Use a distinct key for each intended run. Retry the same input with the same key when a start response is lost: it returns the same run and does not start another paid job. Reusing a key with different input returns `idempotency_conflict`. Creation returns promptly (`202` for REST) with a `run_id`. Work may take several minutes. Poll at the returned `poll_after_seconds` interval, normally 10 seconds. Follow `next_actions` and inspect `messages` for progress and recovery guidance. Execution moves through `queued`, `running`, optionally `cancelling`, and a terminal state: `completed`, `failed` or `cancelled`. Execution and billing have separate states; keep polling while billing is `pending_settlement`. When `result_available` is true, retrieve pages. REST accepts `{}` for the first page; MCP requires `run_id`. Optional `row_id`, exact `period` and `limit` narrow the response. The default limit is 25 observations and the maximum is 100; serialized pages are also bounded to 48 KiB, so a page may contain fewer. Pass `next_cursor` back as `cursor` with the same run and filters until it is absent. A partial result can change while the job runs, invalidating its cursor; restart retrieval or wait for the terminal result. MCP run responses link to `finbar://datasets/{run_id}/results`, an authenticated resource containing the first bounded page. Use `get_dataset_result` for subsequent pages. Large dataset data is kept out of status responses and the assistant's context until requested. **POST /v1/datasets/runs/{run\_id}/result — 200 response** Example response. IDs and values are illustrative. ```json { "dataset": { "entity_id": "string", "rows": [ { "id": "string", "label": "string", "description": "string", "section": "string", "basis": "string", "scope": "string", "unit": "string", "scale": 0 } ], "observations": [ { "row_id": "revenue", "period": { "year": 2025, "period_type": "Y", "period": 0 }, "value": 1250, "status": "reported", "sources": [ { "kind": "finbar", "source_id": "T-example", "table_row_index": 3, "table_column_index": 1 } ], "report_period": { "year": 2025, "period_type": "Y", "period": 0 } } ] }, "changes": [ { "row_id": "string", "period": { "year": 2025, "period_type": "Y", "period": 0 }, "previous": { "row_id": "revenue", "period": { "year": 2025, "period_type": "Y", "period": 0 }, "value": 1250, "status": "reported", "sources": [ { "kind": "finbar", "source_id": "T-example", "table_row_index": 3, "table_column_index": 1 } ], "report_period": { "year": 2025, "period_type": "Y", "period": 0 } }, "proposed": { "row_id": "revenue", "period": { "year": 2025, "period_type": "Y", "period": 0 }, "value": 1250, "status": "reported", "sources": [ { "kind": "finbar", "source_id": "T-example", "table_row_index": 3, "table_column_index": 1 } ], "report_period": { "year": 2025, "period_type": "Y", "period": 0 } }, "applied": true } ], "messages": [ { "code": "string", "severity": "string", "message": "string", "row_id": "string", "period": { "year": 2025, "period_type": "Y", "period": 0 }, "retryable": true } ], "next_cursor": "string", "expires_at": "string" } ``` REST bundle downloads support up to 2 MiB. Larger bundles return `dataset_bundle_too_large`; retrieve pages instead. Stored results have an 8 MiB ceiling. Split rows or periods into smaller runs if the result reaches its limit. ## Interpret and save results Each page contains `dataset`, `changes`, `messages` and `expires_at`. Merge rows by `id`, and observations and changes by `(row_id, period)`, rather than array position. Deduplicate messages repeated across pages. Assemble all pages before using the resulting dataset as Update Dataset input. | Observation status | Meaning | | ------------------ | --------------------------------------------------------------------------------- | | `provided` | A caller-supplied historical value preserved from the input; not newly verified | | `reported` | A finite historical value with citations to source content read during the run | | `derived` | A historical value calculated by the server from structured arithmetic references | | `unresolved` | No supported value; `value` is null and `reason` explains the gap | Derived observations contain a `calculation` using `add`, `subtract`, `multiply` or `divide`, evaluated left to right. Inputs reference other verified observations in the batch or earlier completed periods, or supply numeric constants for arithmetic. A calculation using only constants must cite source content for its financial inputs, such as full-year and nine-month values used to derive a fourth quarter. Cycles, unavailable inputs and division by zero are rejected. The service does not use prior caller-supplied values as evidence for a new calculation. Inspect referenced observations for the underlying sources. `changes` records the prior observation, the proposed observation and whether it was applied. If a refresh is unresolved, the resulting dataset retains the old observation, with `applied: false` and an explicit warning. Non-target observations and your row definitions are preserved. A completed run can still contain unresolved observations; always inspect the warnings. Save the complete result locally before `expires_at`. Content expires seven days after termination; run and billing metadata remain available. Expired retrieval returns `dataset_result_expired`. A new run needs your saved input and a new idempotency key. Each request accepts at most 200 rows, 40 target periods, 2,000 row/period targets, 8,000 prior observations and 2 MiB of JSON. Individual observations are limited to 8 KiB. Split larger datasets by rows or periods while keeping any calculation dependencies together. ## Source verification Both operations review proposed reported and calculated values against their source content, row definition, period, reporting basis, units and scale. Values needing correction receive at most one research retry and are reviewed again. If a value remains unsupported, it is returned as `unresolved` with a reason; calculations depending on it are also unresolved. An unresolved update preserves the caller's prior value and includes a warning. If the review cannot complete, the run fails without applying unverified values from that period. Retrieve any completed periods, inspect the run message and usage, then retry unresolved targets with a new idempotency key. Review and correction can extend execution time to several minutes. ## Charges, cancellation and recovery Both operations consume your account's AI credits for actual provider usage, including research, a second AI review of proposed values, corrective retries, and work done before failure or cancellation. Polling and result retrieval use no AI credits, but remain subject to API/MCP request limits. There are at most two active dataset runs per actor. `dataset_concurrency_limit` means you must wait for an existing run to finish or cancel it. Credit availability is checked before admission and between periods; this is not a fixed-price quote or spending cap. `usage` reports provider calls, tokens, `credits_charged_micros` (millionths of a credit) and `billing_state`. Cancellation before work starts incurs no AI charge. Cancellation during work is cooperative: poll until it finishes, then retrieve any completed observations. Failed and cancelled runs can retain useful partial results. Retry unresolved targets with a new key after correcting the cause. An interrupted worker never automatically repeats uncertain paid research. Recorded usage is settled idempotently. If a provider call's usage could not be confirmed, `billing_state: "attention_required"` and a message identify the need for support review; the service does not invent a charge for unknown usage. Common errors include `validation_failed` (correct the supplied fields), `insufficient_ai_credits` (check the account balance), `dataset_result_not_ready` (poll before retrieval), and `rate_limit_exceeded` (follow the retry delay). Errors include request IDs; contact [support](https://docs.finbar.com/web/support.md) with the request and run IDs if recovery guidance does not resolve the problem.