Query Data in Bulk with Pagination

Developers can use "offset" and "limit" parameters to pull large datasets from the Wood Mackenzie Data API with multiple queries.
To help you get started, here is an example of a query in the M language that could be used in Power BI or Excel:

let GetData = (URL as text, genApiKey as text) =>
let
PageSize = 5000,
GetPage = (Offset as number) =>
let Response =
Web.Contents(URL,[Headers = [#"Gen-Api-Key" = genApiKey],
Query = [limit = Number.ToText(PageSize),
offset = Number.ToText(Offset),
format = "json"]]),
Json = Json.Document(Response),
Data = try Json[data] otherwise {}
in
Data,
Pages =
List.Generate(
() => [Offset = 0, Data = GetPage(0)],
each List.Count([Data]) > 0,
each
let
NextOffset = [Offset] + PageSize
in [ Offset = NextOffset,
Data = Function.InvokeAfter(
() => GetPage(NextOffset),
#duration(0, 0, 0, 2))
],
each [Data]),
Result = Table.FromRecords(List.Combine(Pages))
in
Result
in
GetData