Help

Re: Script to Fetch Wikipedia Summary from Keyword

1238 0
cancel
Showing results for 
Search instead for 
Did you mean: 
Anders_Solberg
4 - Data Explorer
4 - Data Explorer

Description

I wrote this little scriptlet for personal use and thought maybe someone else might find it useful.
It fetches the summary of a Wikipedia article in one cell based on a query in another using the public MediaWiki API.

example

airtable

code

I’m just starting out programming and I don’t know JS so there’s probably alot of bad practice. I wanted to write it in a more functional way but I couldn’t figure it out how when it’s asynchronous.
Feedback welcomed with a smile :grinning_face_with_big_eyes:

/* CONFIG */
let queryColumn = 'Query'
let resultColumn = 'Wiki'
let language = 'en'
let table = base.getTable('table');
let view = table.getView('view');


/* SCRIPT */

/* list of all records in view */
let result = await view.selectRecordsAsync();
for (let record of result.records) {
    let query = record.getCellValueAsString(queryColumn)
    let seachQuery = encodeURIComponent(query)
    /* skips wiki request if airtable record has no query */
    if (query == null) {continue}
    /* skips request if airtable record already has */
    if (record.getCellValue(resultColumn) == null) {
        let wikiURL = 
            'https://' + language +
            '.wikipedia.org/w/api.php?origin=*' + 
            /* specify content of wiki request (https://en.wikipedia.org/w/api.php?) */
            '&action=query&prop=extracts&titles=' + seachQuery + '&exintro' +
            '&format=json';
        let wikiResponse = await fetch(wikiURL, {mode:"cors", headers:{origin:"*"}});
        let wikiJSON = await wikiResponse.json();
        let wikiIntroHTML = wikiJSON.query.pages[Object.keys(wikiJSON.query.pages)].extract;
        try {
            /* removes everything but raw text */
            var wikiIntroText = wikiIntroHTML.replace(/\n|<.*?>/gm,''); 
        }
        catch(e) {
            console.error(
            `Wiki search for "${query}" returned an undefined object.
            This is likely because no such Wikipedia article exists,
            or because your query is not formatted correctly. (it is case sensitive)`
            )
        }
        await table.updateRecordAsync(record, {[resultColumn]: wikiIntroText,});
    }
}
output.markdown('# done')
2 Replies 2

Very cool, thanks for sharing!

This is awesome, thanks so much for posting. Am wondering if you’ve revised it since originally posting?