Flow tasks¶

Every task type in the Flows task catalogue, grouped as the catalogue groups them. For how tasks fit together, see Flows; for statuses, variables, permissions and limits, the Flows reference.

Fields every task has¶

Field	Meaning
Task label	The name on the task's card. At most 40 characters.
Timeout	Where offered, under the advanced fields. The task fails if it runs longer than this. Unset, a task has no time limit.

Any field can hold a flow variable instead of a fixed value. The fields marked text below also accept variables written inside the text — ?name or $name — which are replaced with their values when the task runs. An embedded name that matches no variable is left in the text as written.

Inputs and outputs¶

A task does not receive data from the task before it. It reads from and writes to TriplyDB itself: each input and output names a dataset, graph, query, story, group or service.

Kind	Picks out
Dataset	One dataset, by account and name.
Dataset, by tags	Every dataset of an account that carries all of the given tags. No tags means every dataset of the account.
Graph	One graph, by account, dataset and graph name.
Query, story	One saved query or story by name, or every one of an account that carries all of the given tags.
Group, service	One group or service, by name.

A dataset or graph that a task writes to is created if it does not exist yet. Writing into an existing graph adds to it: tasks that produce data never replace what a graph already holds. To start a run from empty, delete the graph first with a Delete task.

Data import¶

Import from web¶

Downloads RDF from a URL into a dataset or graph.

Field	Meaning
Data URL	The address to download. It must return RDF in one of the supported formats, possibly compressed; an HTML page is rejected.
Request headers	Advanced. Headers sent with the request, for example an `Authorization` header holding a masked variable. Text.

Output: a dataset or a graph. With a graph as output, every statement goes into that graph. With a dataset, triples go into the dataset's default graph and quads keep the graph they name.

Import from OAI-PMH¶

Harvests the records of an OAI-PMH repository into a graph, following the repository's pages until there are none left.

Field	Meaning
API URL	The repository's base endpoint, without `?verb=` or any other query parameters.
Set	Optional. Harvest only this set.
From, until	Optional. Harvest only records changed in this date range.
Metadata Prefix	Optional. The metadata format to request, such as `oai_dc`.

Output: a graph.

Import via connector script¶

Runs a script on a connector agent and imports the data files it writes. See Run custom code in a flow.

Field	Meaning
Connector agent name	The agent to run the script on.
Connector script name	The script, as the agent reports it.
Live artifacts	Advanced. File names the script writes that should be attached to the run as soon as they appear, instead of when the script ends.

Output: a dataset or a graph, as for Import from web. Nothing is imported if the script fails.

Execute via connector script¶

Runs a script on a connector agent without importing anything: for work that happens outside TriplyDB. Same fields as Import via connector script; no output.

Data processing¶

Construct query¶

Runs a SPARQL CONSTRUCT query against a dataset and writes the result to a graph.

Field	Meaning
Query location	Inline, to type the query into the task, or Saved query.
Query	Inline only. At most 10,000 characters.
Saved Query	Saved query only. A saved query by name — or every CONSTRUCT query of an account with the given tags — optionally at a fixed version.
Query variables	Saved query only. A value for each of the saved query's API variables, fixed or taken from a flow variable.

Input: one dataset. Output: a graph.

In an inline query, every SPARQL variable that has the same name as a flow variable is replaced with that flow variable's value, as a string literal. ?dataset in the query becomes "my-dataset" if the flow has a variable called dataset.

Select query¶

Runs a SPARQL SELECT query against a dataset. Same fields as Construct query, except that a saved query is always picked by name.

Input: one dataset. Output: a table of results, kept with the run as a downloadable file.

SPARQL updateCheck your contract¶

Runs a SPARQL UPDATE request against a dataset, changing it in place. Same fields as Select query. This task is available only on instances where SPARQL Update is part of the agreement.

Input: one dataset, which is changed. No output.

Set variables¶

Runs a SPARQL SELECT query and turns its first result row into flow variables for the tasks after it: each variable the query selects becomes a flow variable of the same name. select (count(*) as ?count) { … } sets ?count.

Field	Meaning
Query	A SELECT query, typed into the task. It always runs with `LIMIT 1`: only the first row is used, or the one `OFFSET` picks.

Input: at most one dataset; a query that reads none needs no input. Output: none. The run's page lists the variables the task set, under Variables.

Every value becomes plain text, like every other flow variable: an IRI becomes its IRI string, and a literal its value without its datatype or language tag. A query that needs the IRI or a number back converts it, for example with iri(?thing) or xsd:integer(?count).
A variable the row leaves unbound — every variable, when the query matches nothing — is not set. The task still succeeds; a later task that needs the value fails.
A blank node cannot be a variable's value, and fails the task.

A flow cannot run while a variable set here has the name of one of the flow's own variables, while two tasks on one path set the same variable, or while a task reads it without an edge making sure it runs after the task that sets it. To use every row of a result, use Select query.

Generation with SHACL¶

Applies the SHACL rules in a dataset to that dataset's data and writes the statements they derive to a graph.

Field	Meaning
Max iterations	Advanced. The number of times all rules are applied. Stops earlier when an iteration derives nothing new. Default 10.

Input: one dataset. Output: a graph.

Data management¶

Task	What it does	Inputs	Output	Fields
Create dataset	Creates a dataset if it does not exist; does nothing if it does.	—	A dataset	—
Merge graphs	Combines graphs, or all graphs of datasets, into one graph. Duplicate statements appear once.	One or more datasets or graphs	A graph	Timeout
Copy item	Copies a dataset, story or graph.	One dataset, story or graph	Its copy	Overwrite item if it already exists, in the output's dialog
Copy items to account	Copies datasets to another account, keeping their names.	Datasets	—	Account to copy resources to, Overwrite items when they already exists
Move	Moves or renames a dataset, query, story, group or graph.	One item	Its new location	Overwrite item if it already exists, in the output's dialog
Set access level	Sets the access level of datasets, queries or stories.	Datasets, queries or stories	—	Access level
Set description	Replaces the description of datasets or queries.	Datasets or queries	—	Description, at most 1,000 characters
(Re)create service	Creates a service for a dataset, or recreates it so it serves the dataset's current data.	Datasets	A service	See below
Delete	Permanently deletes datasets, queries, stories, graphs, groups or services. Deleting something that does not exist is not an error.	One or more items	—	—

Moving or copying into a group fails, before anything is overwritten, when the item's access level is more open than the group's.

(Re)create service is configured by service type:

Service type	Extra fields
Virtuoso	—
Jena	Reasoner type: `OWL`, `RDFS` or `None` (advanced)
Elasticsearch	Component templates and Index templates, as JSON (advanced)

Validation¶

SHACL validation¶

Validates a dataset or graph against SHACL shapes and writes the validation report, as RDF, to a graph.

Field	Meaning
Shapes graph	The dataset or graph holding the shapes. Left empty, the input is used as its own shapes graph.

Input: at most one dataset or graph. Output: a graph holding the report.

The task finishes successfully when the report has been written, whether or not the data conforms, so the tasks after it run either way. Read the report graph to see the result.

Communication¶

Send request¶

Sends an HTTP request — to call a webhook, or tell another system that data is ready.

Field	Meaning
Request URL	Where to send it.
HTTP method	`GET`, `HEAD`, `DELETE`, `POST`, `PUT` or `PATCH`. Default `GET`.
Request headers	Advanced. Text.
Request body	Advanced. For `POST`, `PUT` and `PATCH`. Text.

The task fails when the response status is not in the 2xx range. The response status, headers and body are shown with the run.

Send email¶

Field	Meaning
Recipient email address, CC email address, BCC email address	At least one recipient; at most 10 across the three.
Email subject	Text.
Email body	At most 10,000 characters. Text.

A flow can contain at most five Send email tasks.

Control flow¶

Task	What it does	Fields
Decision	Chooses one outgoing branch: the first condition that holds, or else when none does. Tasks on the branches not taken are skipped.	Conditions: each compares two values with `=` or `!=`
Or	Continues once every task in any one of its incoming groups has finished. Use it to join branches after a Decision.	—
Wait	Waits before the next task starts.	Duration, in seconds. Default 10
Stop run	Ends the run at once.	Status: `aborted` (default) or `failed`. Message: shown with the run. Text