Skip to main content

I am trying to search for research applications on district websites and have all the website URLs in one column and keywords for research applications in another. Is there a way to modify the existing script for searching the web for keywords to only search specific websites?

Hi Ariel and welcome to the community!



Probably, but without seeing “the existing script” there would be no way to know for sure.


let districtTable = base.getTable('Districts/Education Agencies');
let resultsField = districtTable.getField('Results');

let recordQuery = await districtTable.selectRecordsAsync();
let record = recordQuery.records[0];
let searchQuery = record.getCellValueAsString("Keywords")
output.text ("Compiling search terms...")


async function searchAsync(searchTerm) {
let response = await fetch(
`https://api.apify.com/v2/actor-tasks/EFvVjhi1CSuOfd09L/run-sync-get-dataset-items?token=SsEpDxFwqJ7FgXL2TpN3EmicG`,
{
method: "POST",
headers: {"content-type": "application/json"},
body: JSON.stringify({queries: searchTerm}),
},
);
let result = await response.json();
let resultLines = [];
for (let {searchQuery, organicResults} of result) {
resultLines.push(`## ${searchQuery.term}`);
for (let {title, url, description} of organicResults) {
resultLines.push(`- [**${title}**:](${url}) ${description}`);
}
}
return resultLines.join('\n');
}
if (searchQuery) {
let searchResults = await searchAsync(searchQuery.split(',').map(term => term.trim()).join('\n'));
await districtTable.updateRecordAsync(record, {[resultsField.id]: searchResults});
output.text ("Sync Complete")
}

Hi Ariel and welcome to the community!



Probably, but without seeing “the existing script” there would be no way to know for sure.


Hi Bill, I have replied with the existing script below. Do you think this looks like something that could be adjusted to serve the purpose I mentioned in the first message?


Hi Bill, I have replied with the existing script below. Do you think this looks like something that could be adjusted to serve the purpose I mentioned in the first message?



Yes, but anything’s possible, right? :winking_face: Certainly this has some of the logic that I imagine might be helpful based on the stated requirements.



These requirements are a bit vague.



  1. Define “district website”; please provide an example or two.

  2. Define “research application” and how one would be detected on a given “district website”. Examples are helpful.

  3. When “searching the web for keywords” you need to decide if you are crawling the web or leaning on a search engine that has already crawled the web. What’s your strategy?


Reply