
Query Data in Bulk with Pagination
Developers can use "offset" and "limit" parameters to pull large datasets from the Wood Mackenzie Data API with multiple queries.
To help you get started, here is an example of a query in the M language that could be used in Power BI or Excel:
let GetData = (URL as text, genApiKey as text) =>
let
PageSize = 5000,
GetPage = (Offset as number) =>
let Response =
Web.Contents(URL,[Headers = [#"Gen-Api-Key" = genApiKey],
Query = [limit = Number.ToText(PageSize),
offset = Number.ToText(Offset),
format = "json"]]),
Json = Json.Document(Response),
Data = try Json[data] otherwise {}
in
Data,
Pages =
List.Generate(
() => [Offset = 0, Data = GetPage(0)],
each List.Count([Data]) > 0,
each
let
NextOffset = [Offset] + PageSize
in [ Offset = NextOffset,
Data = Function.InvokeAfter(
() => GetPage(NextOffset),
#duration(0, 0, 0, 2))
],
each [Data]),
Result = Table.FromRecords(List.Combine(Pages))
in
Result
in
GetData
This query has a few features that we recommend for best performance:
No default 200‑row limit
Automatically retrieves all pages of data
Designed for large data volumes
Safe for production use