Skip to main content
Solved

Daily automation only for modified values

  • February 9, 2024
  • 5 replies
  • 58 views
L
Forum|alt.badge.img+2

Hello everyone, I hope you can help me with this. I have a table with many records related to Spotify listeners or Instagram followers of various profiles. I usually review and update a lot of data daily. I have created an automation to generate a copy of all records that are modified in the source table and move them to another table so that I can track the evolution, for example, if the monthly listeners of an artist grow or decrease.

The problem is that in the destination table, it creates a new line every time there is the slightest modification in any value of the source table. I would need to modify the automation so that, for example, it creates new records in the destination table only once a day, with the lines that have been modified in the source table during the previous 24 hours. This way, I would have in the destination table only one line with the most updated results of everything I have reviewed that day.

Of course, if you can think of any other way to keep track of how the values evolve, I would be happy to hear it. Thank you very much in advance.

Best answer by Dan_Montoya

The reason all of the artists are showing up in 1 record is probably because you are creating the record with the list of recordIDs found in your 'where PROCESAR is not empty'.

Problem:  the find will only return 100 records.  So when you change more than 100 artists, your find approach will not process 101, 102, etc.

I would use a new field to track hours since change. Then create a view that is filtered where hours since change is > 23 and PROCESS is set.  Then change your automation to be triggered by when the record enters the view create the record in table 2.  This will process all of the records that are changed regardless of more or less than 100.

5 replies

Dan_Montoya
Forum|alt.badge.img+17
  • Dan_MontoyaEmployee
  • Employee
  • 275 replies
  • February 9, 2024

You could create a modified time field that only watches the fields that you want to monitor.  Then create a formula that calculates the difference between now (the time your automation runs) and the time of the modified time.  If it is less than 24 hours, process the change (assuming you run the automation once per day).

From your description of following how things change over time, I would suggest you create a history or reporting table that gathers the key metrics over time.  For example, every day create an entry for every artist with their total number of followers/listeners.  Using this type of model you could create a history of any attribute you are tracking: artist, genre, release year, device type, etc.

L
Forum|alt.badge.img+2
  • Lovemonk_Asesor
  • Author
  • New Participant
  • 2 replies
  • February 14, 2024

You could create a modified time field that only watches the fields that you want to monitor.  Then create a formula that calculates the difference between now (the time your automation runs) and the time of the modified time.  If it is less than 24 hours, process the change (assuming you run the automation once per day).

From your description of following how things change over time, I would suggest you create a history or reporting table that gathers the key metrics over time.  For example, every day create an entry for every artist with their total number of followers/listeners.  Using this type of model you could create a history of any attribute you are tracking: artist, genre, release year, device type, etc.


Hello Dan_Montoya, and thank you very much for your response. Do you have any suggestions on how to apply it? Following your advice, I have incorporated a formula that adds the text "PROCESS" to the artists I have reviewed in the last 24 hours.

Then in the automation, I add a first step to run an automation every 24 hours, and then I include a filter to select only those artists with the "PROCESS" label.

So far, so good, but in the last step of the automation, when I ask it to generate entries in the new tracking table, I can't get it to do it with one line for each artist. It writes them all together, separated by commas, on the same line.

Any suggestions to solve this?

Thanks again for your help.

 

Dan_Montoya
Forum|alt.badge.img+17
  • Dan_MontoyaEmployee
  • Employee
  • 275 replies
  • Answer
  • February 14, 2024

The reason all of the artists are showing up in 1 record is probably because you are creating the record with the list of recordIDs found in your 'where PROCESAR is not empty'.

Problem:  the find will only return 100 records.  So when you change more than 100 artists, your find approach will not process 101, 102, etc.

I would use a new field to track hours since change. Then create a view that is filtered where hours since change is > 23 and PROCESS is set.  Then change your automation to be triggered by when the record enters the view create the record in table 2.  This will process all of the records that are changed regardless of more or less than 100.

L
Forum|alt.badge.img+2
  • Lovemonk_Asesor
  • Author
  • New Participant
  • 2 replies
  • February 16, 2024

The reason all of the artists are showing up in 1 record is probably because you are creating the record with the list of recordIDs found in your 'where PROCESAR is not empty'.

Problem:  the find will only return 100 records.  So when you change more than 100 artists, your find approach will not process 101, 102, etc.

I would use a new field to track hours since change. Then create a view that is filtered where hours since change is > 23 and PROCESS is set.  Then change your automation to be triggered by when the record enters the view create the record in table 2.  This will process all of the records that are changed regardless of more or less than 100.


Hello, it seems to be working!
I see that it only adds a single record per artist and day to the tracking table.
I need to fine-tune the fields that the filter has to monitor a bit more, but it looks like everything is working fine.
Thank you very much for your help, and greetings.

M
Forum|alt.badge.img+2
  • ManikyaVarma
  • New Participant
  • 2 replies
  • September 12, 2024

Great Work


Terms & ConditionsAccessibility statement