Flows¶
Look-up material for Flows. The task types have a page of their own: Flow tasks. For how flows work, see Flows; to build one, Build your first flow.
Where things are¶
An account's flows are listed on its page, beside its datasets, stories and queries, and new ones are made with New → Flow. Inside a flow, the left-hand navigation has these entries:
| Entry | Holds |
|---|---|
| Flows overview | Back to the account's list of flows. |
| Flow builder | The canvas. The Flow elements panel on the left has a Tasks tab (the task catalogue, with a search box) and a Variables tab. Clicking a task's card opens its settings on the right. |
| Schedules | Times at which the flow runs by itself. |
| Hooks | Events in TriplyDB that start the flow: an upload to a chosen dataset. See Hooks. |
| API | The address for starting the flow over HTTP, with code snippets. |
| Presets | Saved sets of variable values. |
| Masked values | Secrets the flow uses, such as API keys. |
| Runs | Every run of the flow, with its trigger, status, who started it, and when it started and finished. A run's page shows each task's status; clicking a task shows its details. |
The builder's top bar has Run and Save, and a menu (⋮) with Rename, Access control, Version history, Copy and Delete.
| Key | On the canvas |
|---|---|
| Ctrl/⌘+A | Select every task. |
| Shift + drag | Select the tasks inside a box. |
| Ctrl/⌘+C | Copy the selected tasks, and the connections between them, to the clipboard. |
| Ctrl/⌘+V | Paste copied tasks — from this flow, another flow, or another instance. |
| Delete | Remove the selected tasks or connection. |
| bottom-right corner shows a green check when the flow can run, or the number of problems that | |
| stop it. |
Access levels¶
Set under ⋮ → Access control. A flow is Private or Internal; the Public option is shown but cannot be chosen.
| Level | Who can see the flow and its runs, and copy it | Who can run it |
|---|---|---|
| Private | The owner — for a group, its members — and instance administrators. | The owner; for a group, its members, directly or through a parent group, whose role grants Run flows; and instance administrators. |
| Internal | Anyone logged in to the instance. | The same as for a private flow. |
The access level never lets anyone run a flow: that always takes the Run flows permission below, which only the owning account's members can have. Copying a flow you can see takes Create flows where the copy goes. See Access control.
Permissions¶
Granted through a group member's role; see Custom roles. The labels are the ones in the role editor.
| Permission | Allows |
|---|---|
| Read flows | Seeing flows, their runs and their results. Part of every role. |
| Create flows | Creating flows in the group, and copying a flow into it. |
| Run flows | Starting and aborting runs. |
| Manage flow blueprints | Editing, moving and deleting flows, and managing presets and schedules. |
| Manage flow masked variables | Creating, changing and deleting masked values, and seeing them. |
A run acts with the permissions of whoever started it — for a scheduled run, the schedule's owner. A flow can only read and change what that account may read and change.
Run statuses¶
A run's status is the most severe status among its tasks, in this order:
| Status | Meaning |
|---|---|
failed |
A task went wrong. The run log shows the error. |
timeout |
A task ran longer than its Timeout. |
aborted |
Someone clicked Stop run on the run's page, or a Stop run task ended it. |
running |
At least one task is running, or some have finished and others are still to start. |
pending |
Waiting to start. |
waiting |
Started by a data upload hook, and waiting for that upload to finish. |
finished |
Every task that was to run has finished. |
A task can also show as skipped, when a Decision chose a different branch, or as never started, when the run ended before reaching it.
How a run was started¶
A run's startedFrom.type in the HTTP API:
| Value | Started by |
|---|---|
manual |
Someone clicking Run in the console, or Execute flow on the API page. |
schedule |
A schedule. |
api |
A request to the HTTP API made with an API token. |
hook-asset |
An asset upload hook. |
hook-index-job |
A data upload hook. |
Variables¶
Defined on the Variables tab of the builder's Flow elements panel, with New variable. Each variable has:
| Property | Meaning |
|---|---|
| Name | How tasks refer to it: a variable called dataset is written ?dataset. Names starting with system_ are reserved. |
| Description | Optional. Shown to whoever runs the flow. |
| Required | A run cannot start without a value. |
| Allowed options | Optional. The only values a run may give it. |
Every value is text. Where a field expects something else — a number of seconds, an IRI, a dataset name — the text must be a valid one.
System variables¶
Set by TriplyDB on every run. They cannot be defined or overridden.
| Variable | Value |
|---|---|
?system_triggererAccountName |
The account name of whoever started the run. |
?system_triplyToken |
A TriplyDB API token for that account, valid only while the run lasts. |
A run that a hook started also has these:
| Variable | Value | Hook |
|---|---|---|
?system_uploadedToAccountName |
The account owning the dataset that was uploaded to. | Both |
?system_uploadedToDatasetName |
The name of that dataset. | Both |
?system_uploadedGraphIRI |
The name of the graph the upload added. | Data upload |
?system_uploadedAssetName |
The name of the uploaded asset. | Asset upload |
?system_uploadedAssetURL |
The asset's address in the console. | Asset upload |
Task output variables¶
Set during the run by a Set variables task, and listed on the builder's Variables tab under Task output variables. They are not given when a run starts, and cannot share a name with one of the flow's own variables.
Masked values¶
A masked value is a named secret — an API key, a password — used in tasks like a variable, as
?name. Its value is not shown to anyone without the Manage flow masked variables
permission. A run fails if it uses a masked value that has no value set.
Where a run's values come from¶
Later sources win:
- The system variables.
- The preset the run was started with, if any; a scheduled run uses its schedule's values, and a run started by a hook uses the hook's.
- Values given when starting the run: in the run dialog, or as parameters to the HTTP API. An empty value unsets the variable.
Presets¶
A named set of variable values; the name is at most 80 characters. Presets can be made only once the flow has variables. One preset can be marked Default preset: its values are filled in when someone clicks Run, and when a schedule is created. An API request that names no preset does not use it.
Schedules¶
| Property | Meaning |
|---|---|
| Label | At most 80 characters. |
| Schedule | Every so many minutes or hours, or every day, week or month at a set time. The dialog lists the next five runs, in your own time zone. |
| Timezone | The time zone the schedule's times are in. |
| Variables | The values the scheduled runs use. |
| Enabled | A disabled schedule keeps its settings and does not run. |
A schedule belongs to the person who created it, and its runs act as that person. Removing that user removes their schedules.
Hooks¶
A hook starts the flow when something happens in a dataset. Hooks are listed on the flow's Hooks page, and made with Add Hook, which takes the Manage flow blueprints permission.
| Hook type | Starts a run |
|---|---|
| Asset is uploaded to a dataset | Each time an asset, or a new version of one, is uploaded to the chosen dataset. |
| Data is uploaded to a dataset | For each graph that an upload on the dataset's Graphs page adds. The first run is created as soon as the upload starts and stays waiting until it finishes. If the upload adds no graph, that run is aborted. |
| Property | Meaning |
|---|---|
| Label | Between 2 and 80 characters. |
| Enabled | A disabled hook keeps its settings and does not fire. |
| Hook type | One of the two above. |
| Dataset | The dataset to watch. It must be a specific dataset, not a variable. |
| Variables | The values the hook's runs use, which can be filled in from a preset. |
Like a schedule, a hook belongs to the person who created it, and its runs act as that person. The run's system variables say what was uploaded.
HTTP API¶
The API runs flows; it does not build them. Creating or changing a flow over the API is not supported: build flows in the console, and copy them between instances with copy and paste.
Authenticate with an API token in an Authorization: Bearer
header. The token's account needs the Run flows permission on the flow. ACCOUNT is the
flow's owner, FLOW the flow's ID, and API the instance's API address. The flow's API page
shows the full address to start a run, with ready-made cURL, Python and JavaScript snippets, and
an Execute flow button that starts a run from the browser.
| Purpose | Request |
|---|---|
| Start a run | POST https://API/flows/blueprints/ACCOUNT/FLOW/run |
| Start a run with a preset's values | POST https://API/flows/blueprints/ACCOUNT/FLOW/triggers/PRESET/run |
| Abort a run | POST https://API/flows/runs/RUN/abort |
| One run | GET https://API/flows/runs/RUN |
| The tasks of a run, with their details and files | GET https://API/flows/taskInstances?runId=RUN |
| Download a file a task produced | GET https://API/flows/taskInstances/TASK/artifacts/ARTIFACT |
| A flow's runs | GET https://API/flows/runs?blueprintId=FLOW |
| Download a flow's definition | GET https://API/flows/blueprints/ACCOUNT/FLOW/download.ttl |
Variables go in the query string or in a JSON body, one value each: ?dataset=iris. A name the
flow does not define is rejected, and so is a value outside a variable's allowed options. The
response is the new run, including its id.
The runs listing accepts status, startedFromType (a value from
How a run was started), startedBy, scheduleId, hookId, assetId,
createdAtFrom, createdAtTo, sortBy (createdAt, finishedAt), sortDirection and limit
(at most 10,000).
Failure emails¶
When a run fails or times out, the account that started it gets an email; an aborted run sends none. To stop receiving them, switch off Email notifications under General in your user settings.
Limits¶
| What | Limit |
|---|---|
| Flow name | 80 characters |
| Task label | 40 characters |
| Inline SPARQL query | 10,000 characters |
| Send email tasks per flow | 5 |
| Recipients per Send email task | 10, across To, CC and BCC |
| Email body | 10,000 characters |
| Description set by Set description | 1,000 characters |
| Task timeout | None, unless the task's Timeout is set |
Availability¶
Flows is switched on per TriplyDB instance, by its administrator. The SPARQL update task has a switch of its own. An instance administrator sees every flow, run and task on the instance under Admin settings, and registers connector agents there.