Strange Characters in CSV Export File

[Mens_Shed]

When I export values from a table I get three weird characters at the start of only the first line which contains the field names.

My first line looks like this

Member,Forename,Middle Names,Surname,DOB,Address 1,Address 2,Town,County,Postcode,Home Phone,Mobile Phone,ICE Contact,ICE Number,Email Address,Occupation,Qualifications,Interests,Disabilities

Is this a bug?

[W_Vann_Hall]

Mens_Shed:



You don’t say how you’re saving the export or under what OS you’re running, but that’s a byte order mark. Seemingly, you’re viewing a file saved as UTF-8 in an ISO-8559-1 encoded browser or editor. How to eliminate it depends on how you’re saving it and how you’re examining it: You may have an option to save the export without BOM, or you may be able to change the encoding for the saving application to ISO-8559-1.

[Mens_Shed]

Good morning and thanks for the reply.

  To create the file I simply use Airtable's 'Download CSV' menu choice which has no options and store the resulting CSV file on my Windows laptop.

  When I open the file in Notetab the characters are not visible but when I process the file using a local server side scripting language, the characters appear. (All I am doing at this stage is to isolate the field names and then parse out the corresponding field values whilst removing the " characters as my scripting language has a problem interpreting fields enclosed by that character.)

I can probably strip out these characters but just wondered why they are there as this has not happened before in all the years I have been working with CSV export files.

I have checked the process on three different browsers and all give the same result.

Thanks again.

[W_Vann_Hall]

The standard encoding for HTML 4.01 was ISO-8859-1, while the standard for HTML 5 is UTF-8; that’s likely the reason you’re only now seeing BOMs.

You should check to see if the reason you’re not seeing the BOM in NoteTab is that they don’t actually exist. I think Windows Notepad expects to see a BOM at the beginning of UTF-8 files and, if one doesn’t exist, actually will add the mark at the beginning of the file. Could your scripting language be doing something similar? Does the language support encoding configuration?

[Mens_Shed]

Hello again

  I renamed  a fresh copy of the downloaded CSV file to test.txt and opened it using Open Office Writer and the characters are there so must have been written by the Airtable download process?

Member,Forename,Middle Names,Surname,DOB,Address 1,Address 2,Town,County,Postcode,Home Phone,Mobile Phone,ICE Contact,ICE Number,Email Address,Occupation,Qualifications,Interests,Disabilities

John Tuach

[W_Vann_Hall]

Sounds most like an artifact of OS/browser or OS/editor (or OS/browser/editor) inconsistencies. When I open my test exports from Airtable using a hex editor, the BOM character isn’t present. What OS are you using?

[Mens_Shed]

I am on Windows 7.

The process I used was to (a) download a CSV file from Airtable (b) rename it to test.txt and then © opened it with Open Office Writer so no browser was involved.

I would offer to attach the raw file for your perusal but not allowed.

Appreciate your help but so that I can carry on with writing the required script, I have just worked round the issue by chopping first three characters off the start of the first line in the file.

Risky I know but hey, what’s life without any risk!

[W_Vann_Hall]

Oh, are you using the Airtable Windows app?

[Mens_Shed]

Yes - I access Airtable via

Airtable

Airtable: Organize anything you can imagine

Airtable works like a spreadsheet but gives you the power of a database to organize anything. Sign up for free.

through my laptop and Windows 7.

[W_Vann_Hall]

OK, now I’ve managed to confuse myself.

Where before I seemingly couldn’t get Airtable to preface a CSV export with a BOM, now I can’t seem to get it to stop doing that. This is true for the Windows app and the webapp via a browser. Even more mysteriously, this appears to have been standard export behavior for at least the past year; I have Airtable-exported CSV files from the time I was developing my Wardrobe Manager base that contain embedded BOMs.

The strangest thing, though, is that I also have a few exported CSVs without BOMs, including a few from earlier today. I don’t think the files were saved a second time after the initial export, but I’m at a loss as to how they could be missing the BOM. (Could this be a Win 7/Win 10 thing? Perhaps the BOM-less ones were exported using my other PC.)

I’m going to pursue this last possibility some more once I’m in arms’ reach of my Win 10 machine. Your solution will probably have to involve your scripting language. It might be worth doing some web searches on “[your scripting language]” and either “UTF-8” or “BOM”. (For instance, there are a number of sites addressing PowerShell and unwanted BOMs.)

Sorry to confuse everyone involved…