On this page:

Paginate SPARQL results

This page explains how to retrieve all results from a SPARQL query using pagination.

Often SPARQL queries can return more than 10,000 results, but a single result set carries only the first 10,000. This is a transport limit rather than a query limit — the query itself matched everything — so to get the rest you page through the result set.

TriplyDB supports two ways of doing that. Any HTTP client can use the saved query API; TriplyDB.js needs TypeScript but handles the paging itself, so there is less to get wrong.

Pagination with the saved query API

  1. Request the saved query's run endpoint with a pageSize, starting at page=1:

    https://api.triplydb.com/queries/academy/pokemon-color/run?page=1&pageSize=100

  2. Follow the next URL in the response's link header, and repeat until a page comes back empty. The header returned with page 3, for example, is:

    Link:
    <https://api.triplydb.com/queries/academy/pokemon-color/run?page=4&pageSize=100>; rel="next", <https://api.triplydb.com/queries/academy/pokemon-color/run?page=2&pageSize=100>; rel="prev", <https://api.triplydb.com/queries/academy/pokemon-color/run?page=1&pageSize=100>; rel="first"

pageSize can go up to 10,000. See Pagination arguments for the defaults and limits of both arguments.

Pagination with TriplyDB.js

TriplyDB.js is the official programming library for interacting with TriplyDB. TriplyDB.js allows the user to connect to a TriplyDB instance via the TypeScript language. TriplyDB.js has the advantage that it can handle pagination internally so it can reliably retrieve a large number of results.

To get the output for a construct or select query, follow these steps:

1. Import the triplyDB.js library and set your parameters, regarding the TriplyDB instance and the account in which you have saved the query as well as the name of the query. Do not forget that we perform TriplyDB.js requests within an async context.

import Client from '@triply/triplydb'
async function run() {
  // Your code goes here.
  const client = Client.get({token: process.env.TRIPLYDB_TOKEN})
  const account = await client.getAccount('account-name')
  const query = await account.getQuery('name-of-some-query')
}
run()

2. Get the results of a query by setting a results variable. More specifically, for construct queries you use the statements() call:

const query = await account.getQuery('name-of-some-query')
const results = query.results().statements()

For select queries you use the bindings() call:

const query = await account.getQuery('name-of-some-query')
const results = query.results().bindings()

Additionally, saved queries can have 'API variables' that allow you to specify variables that are used in the query. Thus, if you have query parameters, pass their values as the first argument to results as follows:

// For SPARQL construct queries.
const results = query.results({
  someVariable: 'value of someVariable',
  anotherVariable: 'value of anotherVariable'
}).statements()
// For SPARQL select queries.
const results = query.results({
  someVariable: 'value of someVariable',
  anotherVariable: 'value of anotherVariable'
}).bindings()

3. To iterate the results of your SPARQL query you have three options:

3.1. Iterate through the results per row in a for-loop:

// Iterating over the results.
for await (const row of results) {
  // execute something
}
Note: For select queries the for-loop iterates over the rows of the result set. For construct queries the for-loop iterates over the statements in the result set.

3.2. Save the results to a file. This is only supported for SPARQL construct queries:

// Saving the results of a SPARQL construct query to a file.
await results.toFile('my-file.nt')

3.3. Load all results into memory in the form of an Array. Usually, if you want to process results, then option 3.1 suits better; if you want to persist results, then choose option 3.2.

// Loading results for a SPARQL construct or SPARQL select query into memory.
const array = await results.toArray()