Sep 13, 2019 07:59 AM
I am connected the API of Airtable to Power BI but at the time of displaying the data it only shows me 100 records when the base has more than 1000 records.
Solved! Go to Solution.
Sep 18, 2019 10:13 PM
Sep 13, 2019 08:33 AM
The Airtable API uses the concept of pagination.
The response will contain an offset if there are more than 100 records. Include this offset in the next request’s parameters
Sep 13, 2019 08:37 AM
How can I add these parameters?
Sep 13, 2019 08:39 AM
Are you using the Airtable Javascript library or accessing the it via their REST API?
If you are using the REST API you need to pass another parameter “offset”
Sep 13, 2019 08:43 AM
I am using REST API. With what value should I pass the “offset” parameter?
Sep 13, 2019 08:46 AM
The “offset” value is returned in the response of your first call to the Airtable API under the key “offset”
Take the above returned “offset” value and pass it in your new request.
Sep 13, 2019 08:46 AM
This is what you want - API Offset Parameter
Sep 13, 2019 09:08 AM
Reloading the data source sending as an offset parameter and still showing 100 records. The image I send is using POSTMAN to return the json.
Sep 13, 2019 09:41 AM
In the screenshot shown, there is no value shown for the offset parameter, therefore, the API will default to the first 100 records. It is responding exactly how it should.
Sep 13, 2019 09:48 AM
@Bill.French That the point, what is the parameter that the offset must pass?
Sep 13, 2019 10:32 AM
As mentioned by @Andrew_Johnson1 above, the offset value is returned by the first request (which has no offset). Each ensuing request must contain the offset value returned by the recent [previous] successful response.
@Chinara_James makes this abundantly clear in her blog post which was referenced in the forum thread I shared above.
Sep 13, 2019 10:48 AM
I understand that, what happens is that the visualization of the data I am working with Power BI. If I was working with NodeJs I understand the recommendations but this is not the case.
Example:
Sep 13, 2019 11:20 AM
Oscar,
I’m, almost certain PowerBI doesn’t have the internal capacity to automatically understand and perform the multiple requests into Airtable required to get all of the data you want to visualize. As such, you are responsible for creating a middle-ware process that does know how to iterate across all the record sets that the Airtable API is capable of exposing.
There are a few ways to do this - I would check out the Zapier/PowerBI integration recipe.
I wasn’t suggesting you needed to use NodeJS - simply demonstrating how it is done using the Airtable API was my intent recommended this article.
Sep 13, 2019 11:40 AM
My fault for not seeing this at the outset - I scanned the threads quickly and when I read this I assumed you were building middleware integration code to use with PowerBI. I should have asked for deeper details.
PowerBI is a wonderful data visualization platform and I have used it often for many clients including this one featured by Microsoft a few years ago when PowerBI started supporting real-time streams. As you can see, I have deep expertise using PowerBI’s API and Azure streaming protocol, so if you need professional help, give me a shout.
Sep 18, 2019 10:13 PM
Sep 18, 2019 10:56 PM
Great article Oscar!
That’s the middleware you needed and it’s ideal that you did this in PowerBI script - simplifies the moving parts probably fast as well.
Oct 15, 2019 07:03 AM
Thank you @Oscar_Diaz and @Bill.French
I have almost got there. :slightly_smiling_face: I wrote out the code from the screenshot in the Medium article as below, replacing the [APP_ID] [TABLE_ID] [KEY_ID] with the appropriate entries, but PowerBI coughed at the last line:
let
Pagination = List.Skip(List.Generate( () => [Last_Key = "init", Counter=0], // Start Value
each [Last_Key] <> null, // Condition under which the next execution will happen
each [ Last_Key = try if [Counter]<1 then "" else [Quelle][Value][offset] otherwise null, // determine the LastKey for the next execution
Quelle= try if [Counter]<1 then
Json.Document(Web.Contents("https://api.airtable.com/v0/[APP_ID]/[TABLE_ID]", [Headers=[Authorization="Bearer [KEY_ID]"]]))
else
Json.Document(Web.Contents("https://api.airtable.com/v0/AIRTABLE_BASEID/TABLE_NAME&offset="&[WebCall][Value][offset] , [Headers=[Authorization="Bearer key????????????"]])),// retrieve results per call
Counter = [Counter]+1 // internal counter
],
each [Quelle]
),1),
#"ConvertedTable"= Table.FromList(Pagination, Splitter.SplitByNothing(), null, null, ExtraValues.Error)
in
#"Tipo cambiado"
I closed with the Spanish translation of “tipo cambiado” which is “type changed” and got this error . . .
Oct 21, 2019 05:27 AM
Try replacing
“tipo cambiado”
with
#“ConvertedTable”
Jun 11, 2020 11:16 AM
Hi Bill,
RE: Power BI
have a QQ if you don’t mind. i see that you mention experience with Power BI (and you undoubtedly are a Pro with Airtable), so i’m curious if you’ve ran into this issue.
i’m using a slight variant on the web.contents() script listed in this thread and an issue thats popped up is that ‘published’ PowerBI reports have trouble ‘auto-refreshing’ reports/data-sets utilizing web.contents. the tables i’m using in my reports have between 300-25k rows across ~50 columns. i’m trying to deploy the PowerBI App for internal usage without having to do a daily/manual refresh (or something similar).
if you have any suggestions, help would be greatly appreciated. i’ve also checked out [Using The RelativePath And Query Options With Web.Contents() In Power Query And Power BI M Code] but it seems to have some sync issues as well, which concerns me deploying using this method.
Best regards,
Matt
Jun 12, 2020 12:42 PM
Hi Matt - I have a few thoughts but slammed recently. Back shortly.