I've grown very frustrated with csv import and I am open to any suggestions that will allow me to do a batch import of files with their metadata.
When I try to import with csv import, the metadata imports fine, but the file does not. I've tried multiple csv files including the test files that come with the plugin. I've sought help from my hosting site (namecheap): nothing is amiss on that end. I've reinstalled the plugin. I've reinstalled omeka. I posted on the fourm about this a few weeks ago but none of the suggestions fixed the problem. Nothing works. I'm about to admit defeat and accept that I'll never get the files to import with csv import. So now I need to figure out another way to do the bulk import of files and metadata.
I had an idea and I was hoping someone could tell me if it could work: could I do a bulk upload of files using the dropbox plugin and then do a bulk upload of the metadata with csv import? Or will the files uploaded by dropbox not tie to the metadata uploaded by csvimport? I need both, not one or the other. If there's some way to do this, can someone share the steps?
I'm also open to any other suggestions: I have 500+ pdf files of scanned letters and metadata I need uploaded and I'd rather not do them one by one.
Thanks,
Amber
Hi,
You should set the full url of each file, like that:
http://thefamilypapers.info/corresp_1911-20_030.pdf
Sincerely,
Daniel Berthereau
Infodoc & Knowledge management
Hi Daniel,
Thanks for responding. I tried that with a newly revised csv file, it still doesn't work :(. Here's the file: thefamilypapers.info/csvimport2.csv
Hi Amber,
When I download your csv file to a Mac using http from your link, I note that there are ^M (control-m) characters immediately preceding each instance of the http:// link, eg:
[...] Fanny."^Mhttp://thefamilypap [...]
These are Carriage Return (CR) symbols, to the best of my limited knowledge, and may be an artefact of the editor within which you are creating or from which you are exporting the .csv file.
It's worth considering that these CR charactes are interfering with the effectiveness of the file as read by the plugin. We are currently having success uploading files with the metadata, as Daniel says above using the complete url of the file (which has to be readable by Omeka as well as by you).
We are currently using the free openoffice.org suite spreadsheet (Excel clone) and exporting as csv, and that's working for us.
Good luck with it,
Fergus
Hi Fergus,
Thanks for the response. I created the csv file using excel. I'll take a look at what you suggest.
Hi Again Fergus,
What you say makes perfect sense: if the url is incorrect, it will not upload properly. However, I've tried viewing the csv file in open office on a mac (as you state you did) as well as a text editor on a Windows 7 computer and I can't see the carriage return you cite above. Any suggestions?
Thanks,
Amber
Your file works fine as-is for me.
Do things work correctly when you try to manually upload just one of the PDFs? I'm not sure CSV Import is the culprit here. In your last thread, you posted the following log lines:
[22-Jun-2014 18:35:04 America/New_York] PHP Warning: exec() has been disabled for security reasons in /home/theftciw/public_html/clara/application/models/File.php on line 315
[22-Jun-2014 18:35:04 America/New_York] PHP Warning: shell_exec() has been disabled for security reasons in /home/theftciw/public_html/clara/plugins/PdfText/PdfTextPlugin.php on line 134
Are you sure the files aren't actually uploading at all? The warnings you posted would indicate that Omeka would have a problem making thumbnails for files on your server, but the files themselves should still upload.
Hi John,
Thanks for responding. Since the last thread, I reinstalled Omeka and I'm not getting that error message any more within the Omeka error log.
As for the files uploading: when I upload the csv file and the metadata appears, I then check each item and it gives me a message that no file has been attached.
If I upload the files one by one, they usually work, but not all the time. I can't get the dropbox plugin to work properly either, it can't handle more than one or two files at a time.
I'm using cpanel and if I click on the error log within cpanel, I get a lot of error messages:
[Mon Jul 07 15:06:57 2014] [error] [client 193.111.60.1] File does not exist: /home/theftciw/public_html/403.shtml
[Mon Jul 07 15:06:57 2014] [error] [client 193.111.60.1] client denied by server configuration: /home/theftciw/public_html/test/.htaccess
[Mon Jul 07 15:06:34 2014] [error] [client 193.111.60.1] File does not exist: /home/theftciw/public_html/404.shtml
[Mon Jul 07 15:06:34 2014] [error] [client 193.111.60.1] File does not exist: /home/theftciw/public_html/favicon.ico
[Mon Jul 07 15:06:34 2014] [error] [client 193.111.60.1] File does not exist: /home/theftciw/public_html/404.shtml
[Mon Jul 07 15:06:34 2014] [error] [client 193.111.60.1] File does not exist: /home/theftciw/public_html/favicon.ico
[Mon Jul 07 15:06:34 2014] [error] [client 193.111.60.1] File does not exist: /home/theftciw/public_html/403.shtml
[Mon Jul 07 15:06:34 2014] [error] [client 193.111.60.1] client denied by server configuration: /home/theftciw/public_html/.htaccess
[Mon Jul 07 15:04:22 2014] [error] [client 74.92.14.130] File does not exist: /home/theftciw/public_html/404.shtml, referer: http://thefamilypapers.info/
[Mon Jul 07 15:04:22 2014] [error] [client 74.92.14.130] File does not exist: /home/theftciw/public_html/favicon.ico, referer: http://thefamilypapers.info/
[Mon Jul 07 15:04:22 2014] [error] [client 74.92.14.130] client denied by server configuration: /home/theftciw/public_html/.htaccess
[Mon Jul 07 15:02:50 2014] [error] [client 193.111.60.1] File does not exist: /home/theftciw/public_html/403.shtml
[Mon Jul 07 15:02:50 2014] [error] [client 193.111.60.1] client denied by server configuration: /home/theftciw/public_html/.htaccess
[Mon Jul 07 15:01:48 2014] [error] [client 193.111.60.1] File does not exist: /home/theftciw/public_html/403.shtml
[Mon Jul 07 15:01:48 2014] [error] [client 193.111.60.1] client denied by server configuration: /home/theftciw/public_html/clara/db.ini
[Mon Jul 07 14:57:24 2014] [error] [client 193.111.60.1] File does not exist: /home/theftciw/public_html/404.shtml
[Mon Jul 07 14:57:24 2014] [error] [client 193.111.60.1] File does not exist: /home/theftciw/public_html/favicon.ico
[Mon Jul 07 14:57:24 2014] [error] [client 193.111.60.1] File does not exist: /home/theftciw/public_html/404.shtml
[Mon Jul 07 14:57:24 2014] [error] [client 193.111.60.1] File does not exist: /home/theftciw/public_html/favicon.ico
[Mon Jul 07 14:33:15 2014] [error] [client 66.31.252.74] client denied by server configuration: /home/theftciw/public_html/.htaccess
I'd appreciate any help you can provide. Thanks.
The last time I attempted to upload an abbreviated version of the csv file it worked!!!! I cut it down to 3 rows. It's been 20 minutes and it's still working on the last 2, but the item in the first row uploaded correctly. Is it supposed to be this slow? I tested my Internet speed and it's running at an expected level for Broadband.
Hi,
Main part of your log are not related to Omeka, but to the configuration of your server, public directory and right access to files.
If you use a shared host server, this can be very slow. If you use your own server, import of pdf shouldn't be so slow, except if you have other installed plugins.
When the process will be ended, if some rows are imported correctly but not all, that means you may need to check your Omeka logs and perhaps php.ini on the server (to increase max upload size, that seems limited to 32MB, because the first three pdfs are just lower). You can increase upload time too.
Sincerely,
Daniel Berthereau
Infodoc & Knowledge management