This is one of the many bugs in Airtable, but they are uninterested in fixing this bug, according to conversations I have had with the support team about this bug. You can try your luck with them by sending an email to email@example.com to tell them how important it is to you that they fix this bug. At the very least, they should always be hearing from people about how their bugs are hurting people’s usage of the platform.
You might be able to find relief by using a rollup field instead of a lookup field, but I haven’t tested this myself yet. In your original table, create a rollup field using the formula values and see if that works in your destination table.