I'm wondering if it can't be a problem to have all files stored in the same directory (archive/files). I'm using the Bookreader plugin made by Julien Sicot, and for every book in the library I upload one file per page.
In some months I should have about 80 000 pages in my digital library and I wonder if at some point, it won't become a problem.
The performance of a directory with a very large number of files is really dependent on the underlying filesystem, which is a bit beyond the Omeka level.
Most servers lately use the "ext3" filesystem, which is basically the standard for Linux. ext3 contains code that means files can be found and retrieved very quickly, even from directories with a very large number of files.
Omeka pretty much always acts on files one at a time, but operations you perform yourself, like listing the files in the archive directory, or backing up that directory, will naturally take longer as you start to get more files.