Skip to main content

I get served a list of 500 company numbers every weekday at 5 am, i need to have available information for my sales agents by 7.30 am in the morning.

This i solve by using an API that collects the information from a government website where i come from. It's the only source available.

The API is limited to just one company per request.

I have solved this code wise using a for of loop which loops over my table, sends an api fetch request based on the number, then inserts the information, then starts the next loop.

The issue I'm facing is the limit of 50 fetch requests pr automation.

The only solution i can come up with myself at my current code/situation is to make the automation limited to 50 runs pr automation, then make 10+ scheduled weekday runs.

I have access to 50.000 monthly automation runs so that's no issue, but there has to be a better/ More elegant solution than this?

Here's an example API link:

https://data.brreg.no/enhetsregisteret/api/enheter/929135180/roller?

The number marked in red is the company number i mentioned, it gives a result of who owns the company, which in this case is me, Ruben Rossvold.

You can have up to 25 actions per automation run. With 50 fetches per scripting action, you can do this all in one automation run. The trick is dividing up the different companies across the different scripting actions. Note that depending on how long each fetch takes, you might not be able to complete all 50 fetches in 30 seconds. 

 

One option is to have the first action be a script that marks a checkbox for all company records as needing fresh data. Then each subsequent script can process as any companies as it can before reaching one of the limits (50 fetches or 30 seconds.) As the script processed each company, it clears that company’s  checkbox. Additional copies of that same script keep processing company records until they are all done. 

You can have up to 25 actions per automation run. With 50 fetches per scripting action, you can do this all in one automation run. The trick is dividing up the different companies across the different scripting actions. Note that depending on how long each fetch takes, you might not be able to complete all 50 fetches in 30 seconds. 

 

One option is to have the first action be a script that marks a checkbox for all company records as needing fresh data. Then each subsequent script can process as any companies as it can before reaching one of the limits (50 fetches or 30 seconds.) As the script processed each company, it clears that company’s  checkbox. Additional copies of that same script keep processing company records until they are all done. 


Each company without the fresh info is stored in a filtered view showing the companies lacking the info, not needing the checkbox.

Each fetch is really fast so that's not an issue.

You can have up to 25 actions per automation run. With 50 fetches per scripting action, you can do this all in one automation run. The trick is dividing up the different companies across the different scripting actions. Note that depending on how long each fetch takes, you might not be able to complete all 50 fetches in 30 seconds. 

 

One option is to have the first action be a script that marks a checkbox for all company records as needing fresh data. Then each subsequent script can process as any companies as it can before reaching one of the limits (50 fetches or 30 seconds.) As the script processed each company, it clears that company’s  checkbox. Additional copies of that same script keep processing company records until they are all done. 


I read your answer again and i just realized what you meant with 25 actions.

So if i make one automation with 10 scripting actions using the same script, it would actually fix my issue.

I tested my script and it's able to go for 50 records in 21 seconds.

this is my code:

for (let record of records ) {
//This is the API call
    let brregRawData = await fetch(`Some url${record.name});
    let brregJson = await brregRawData.json()
    let personNavn = brregJson.rollegruppern0].rollern0].person.navn.fornavn
    let personEtternavn = brregJson.rollegruppern0].rollern0].person.navn.etternavn
    if (brregJson.rollegruppern0].rollern0].person.navn.mellomnavn == undefined) { 
        var personMellomNavn = ""
    } else {
        let personMellomNavn = brregJson.rollegruppern0].rollern0].person.navn.mellomnavn + " "
    }

Is there any way i could introduce a max limit of 50 iterations through the for of loop?

Declaring a count value outside of the for-loop and then making an IF 40 then break statement fixed my issue.

let count = 0; // initialize count as 0 outside of the for of loop
  for (let record of records ) {
  if(count >= 50) break; // break out of loop if count is equal to or greater than 40 (since count starts from 0)
count++; // Increment the count after each iteration
}

 

Reply


Terms & Conditions