Absolutely! The CSV download is restricted to the records visible in the active view. Use filters to control the displayed records, then the download will match.
Looking at the permissions overview page (linked below), I don’t see any mention of disabling CSV downloads at any level, but it may be related to another option and just not mentioned specifically. You might want to write directly to Airtable support to inquire further.
