Airtable keeps attachments after their original records are deleted. This is to preserve your history. If you ever want to go back and restore a previous snapshot of a base, you would expect the attachment to still be included. Attachments are not automatically deleted until after your revision history expires.
If you need to free up attachment space before the attachments your revision history expires, I recommend contacting customer support.