Welcome to the community @Marcus_Lindberg!
Unfortunately, this feature request (without response) suggest there is no apparent support for this but I would still file a support ticket with Airtable to confirm this is the case.
What this issue suggests is that utf8_general_ci is being used by Airtable which will sort Ö as O. In contrast, the utf8_swedish_ci will instead properly sort items with umlauts. The result is that the correct sorting order of things like ÄAÖ will go to the end of the alphabet.
Lacking such support in Airtable, you could use the new Script Block feature to read the table’s desired index field, sort it properly in javascript (applying the proper character encoding), and then mimic a sort feature by updating a separate numeric index field with values that result in the sort as you require.
A tool like this could be used across many tables and bases and would require only a single click to update the sort index.
