Adding multiple files to Item functionality is buggy

I'm having repeated, reproducible difficulty adding multiple files to an item. This is the case when I try to batch add name filtered items in the dropbox, and worse, when I try to add these items one by one. After a while (the last time on the 3rd file) Omeka returns an error. At that point the Item in question can no longer be deleted. Attempting this causes another error. Omeka 1.4.1. I can provide more details if needed.

So, are you saying that you encounter the same problem whether or not you add the files one at a time or all at once?

Yes, at least from my experience that appears to be the case. If you like, I can set-up a separate instance of Omeka in MAMP or with my web host (Dreamhost) to reproduce this in a controlled environment.

Please email me at arno dot bosse at gmail if you want to pursue this. I know I've already taken up a bunch of the Omeka team's time with this issue (and with the problem of rendering thumbnails from multi-page TIFF or PDF files) so I'd like to help out in some way if possible.

I'm seeing this now as well when adding a single new file to an item which previously had a PDF added to it. This causes an error. When I look at the database record for the item in omeka_files I see that it has an authentication ID and a generated archive filename but that it is not recorded as stored. The file has not been uploaded and thumbnails have not been created.

I can work around this by setting has_derivatives and stored to 1 and uploadng the necessary files to archives myself but this is a real PITA process since I have lots of items with PDF files and often need to add additional image files to these items. I do hope this can be fixed.

I've confirmed that it will not show an error if the previously added file was properly processed (i.e. thumbnail could be generated).

Can you check to see if you have a bunch of extra files sitting around in your temp directory (probably /tmp)?

The current version of Omeka doesn't deal well with multi-page TIFFs and PDFs, and tends to leave behind extra unused thumbnails, which could be screwing with your subsequent upload attempts.

Yes, lots. fullsize_ sess_ square-thumbnail_ and thumbnail_ jpeg files plus a bunch of PNGs with no recognizable prefix.

I'm beginning to think this may be a specific problem with Dreamhost (http://dreamhost.com/). I've been following a lot of false trails in terms of possible causes. For example, in the past, I would regularly get errors when trying to add more than three files (via Browse button or Dropbox). I thought that this was due to their number. But my files are all roughly the same type and now I think it may be due to the size of the files since I am also seeing this with individual, large files (e.g. 1.7MB PNG). In the database this results in a row being written for the file but sometimes with the has_derivatives and/or stored attributes set to 0.

Given that there are so many temp files in my /tmp directory (see above) even while there is still enough space in /tmp makes me think perhaps outdated versions of ImageMagick and Ghostscript are responsible. Dreamhost offers Ghostscript 8.62 (2008-02-29) and ImageMagick 6.3.7 while the current versions are closer to 9.0.2 for Ghostscript and 6.7.1 for ImageMagick. In the case of the 1.7MB PNG file I was able to successfully add it once I deleted its row in the omeka_files table and reduced its file size.

I'll see if I can reproduce the errors in MAMP on my Mac which has current versions of both. If not, I'll file a request with Dreamhost for an update or look into whether I can build my own versions on Dreamhost (I don't use my own VPS). Dreamhost also offers 1-click install of Omeka (1.3.2) so it would be a pity if their tools aren't up to the job.

I was finally able to resolve this by building my own version of imagemagick and ghostscript on dreamhost.

Whew! Well done chasing all that down. Sorry it was so much, and thanks for the info about Dreamhost.