When a view is specified during an API request, all returning records will be contained in that view and any filterByFormula parameters will be applied/considered on records contained in that view.
However I’m still a little confused about what you’re trying to accomplish—it sounds like you want to fetch records using the manual sort order defined in by a view yet you want to retrieve records that are NOT contained in that view (by “overriding” the view filters with a custom filterByFormula). Even if you were to accomplish that, how would you anticipate the records from outside the view would be sorted in the results?
The only way I know to have the “best of both worlds” (i.e. a manual sort order from a view with all non-view records appended to then end) would be with two requests:
You could 1) request all manually sorted records from the view and 2) request all records from the table where AND(RECORD_ID() != ‘recXXXX’, RECORD_ID() != ‘recXXXX’, etc) …
What language are you using to connect with the API?