I have been toying around with the Omeka dropbox plugin and would like to thank the support staff for providing it. The tool provides an easy way to batch import files; something very useful. :) That said, I was wondering if there is a way to quickly associate metadata with batch uploads? I know that Omeka associates unique item id keys to new uploaded items (I assume this has to do with the underlying database) so associating files and information should be as simple as mapping the correct information. But, this doesn't necessarily make it easy.
Anyone have a clue on this?
Hi AJMillion. There is currently no way to associate metadata with individual items during batch uploads, but it's a good idea and one that should be implemented. We're currently working on a CSV Import plugin that will import items from a CSV file into Omeka (although for the moment it's taking a back seat to our upcoming OAI-PMH harvester). Maybe we can somehow tie in the CSV plugin to the Dropbox plugin, which would allow simultaneous file and metadata import.
In the meantime I think you're limited to using the Dropbox plugin to batch upload files and separately editing the individual items. Not quite an efficient process, but It'll get the job done. Do you have any other ideas?
Jsafley,
Thanks for responding. The CSV support would be a good step, because anyone exporting/importing metadata will likely be able to convert other formats into this one. Another idea would be to enable the use of XML, but as with all of these formats I suspect the issue of precise mapping might come up.
The CSV / batch idea is a good one. I used something similar last year while working on an archival project but it was much less flexible. A tech rigged it up so we could import a bunch of old files. We used the .jpg file names to associate the two prior to our uploading. It worked fine.
I'm just interested in finding ways to batch process large amounts of images (to see how Omeka handles). I know it's designed to be "museum like," but if the style sheets are configured correctly and images are properly arranged, then I don't see how it couldn't start to cross over into "archive-land."
I too am interested in the possibility of batch uploading at least image files and associated metadata. So, I would second Jsafley's comments. In my case, I am already a CONTENTdm user, but the presentation interface for Omeka makes is very attractive. However, the process of adding metadata, one field, one item at a time would really be a painful method, IMHO.
Is there documentation on how the OAI-PMH harvester is designed to function? I'm thinking that might make some sort of viable way to use my Cdm collections and frontend them with Omeka. (maybe I'm all wet here.
Hi bethnicol --
In addition to a CSV importer and the generalized OAI-PMH harvester (we're still too deep in research and development to provide any documentation), we are working towards a specialized a set of CONTENTdm migration tools. We cannot provide a firm date for when these tools will be ready, but they are high on our list of priorities for the spring and early summer. We will make an announcement as soon as a beta version is ready.
Thanks for using Omeka and letting us know how you're making out.
Tom
I read Tom's post here, and I'm wondering what sort of movement there has been on the CONTENTdm migration tool. I have several CDM collections that I would like to keep in their legacy system while allowing users to access the collections via Omeka.
jpeg2000--
We're working very hard towards the 0.10.0 release which will feature a new data architecture more amenable to CONTENTdm import and export. Once that's finished sometime early in the new semester, our first priority will be a suite of CONTENTdm migration tools. I can't give a precise date, but sometime before Thanksgiving would be a safe bet.
Take care,
Tom
Thanks, Tom! That sounds pretty great. I'll keep an eye out for it, and look forward to giving those migration tools a spin.
jason (jpeg2000)
jkucsma_at_metro.org
What happened to the CONTENTdm migration tools project? I want to use it for the first time to enhance a special collection where the metadata and images are stored in CONTENTdm.
Thanks, Lyn
Lyn--
It is underway. In the next few days we will be releasing version 0.10.0 alpha to the dev list for developer community testing and refinement. This new version of Omeka includes a completely new metadata architecture, prerequisite to CONTENTdm interoperability. (Don't worry, version 0.10.0 comes with a set of migration tools for current users of version 0.9.x.) When version 0.10.0 is released as a public beta later in the fall, it will also include the CONTENTdm migration tools.
Watch this space.
Take care,
Tom Scheinfeldt
Hello Tom,
I haven't seen much mentioned about it with the new release, and I'm wondering if the CONTENTdm migration tools are included in .10 release.
Jason
We are going to do a batch load of metadata to Omeka via csv file. I am just wondering whether there is any instruction on the format of csv file. We are transferring data from ContentDM to here. Do you have any documentation on that?
gwcdc,
I would defer to Omeka staff or check on the Omeka Dev site for better and more qualified advice, but here's what I did awhile back.
I kept these notes for internal use, but maybe they'll be helpful to you.
http://docs.google.com/Doc?id=dhb8httt_169f7msc3g2
These notes are great!
A couple of other quick things I will emphasize:
1. You can only add one item type (image, documents, et al) per import.
2. In your CSV file, be sure that you include the table headings as your first row. The plugin assumes that the first row includes headings, so if they are not present you will lose the first record in that table.
Also, including the headings will help you map the fields in your CSV file to the appropriate ones in Omeka.
3. As currently designed (although this is getting fixed), once you have mapped a CSV field to the Omeka one, you must click the "add element" button. If you don't, it won't be added.
We are fixing this for the 1.0 release because it is quite confusing for users.
Good luck with the ingest and let us know how it goes.
Sheila
Thanks a lot for the document. That really helps. I got a question while trying to transfer my csv file to Omeka. How to do de-import csv file on Omeka? The status tab shows the importing is in progress. However, it has been two days since it is like that. I wanted to de-import the process and do some test to see whether the Chinese characters in my csv file caused the problem, but I have no idea of how to de-import it. Can you pls. give me some hint.
Thanks a lot.
Hi,
We are working on some bugs with the CSV importer so it might not be you. I also had trouble while testing it and the import process was stuck "in progress" also.
Once the import has finished you have the opportunity to delete all of the records you have just imported.
Have you imported many of the records? I found that I didn't import any of the records when it was perpetually "in progress." So there wasn't anything to stop.
Have you tried deleting the CSV file from the plugin directory? Not quite sure if this will stop the importing process, but it might be worth a try.
Sheila
There were some issues with the document I posted here. You might want to check out the thread at Omeka Dev in case that is part of the problem (http://groups.google.com/group/omeka-dev/browse_thread/thread/29e82370d9a8362a).
Any update on the ContentDM migration tools?
No, not specifically for ContentDM.
But your ContentDm installation might be OAI-PMH compliant which would allow you harvest the data and then import it into Omeka using the OAI-PMH plugin.
Sheila
We have over 150,000 records in ContentDM and I've tested OAI-PMH and it works fine... But, then I have to associate the images manually? Are the images stored physically on disk on the server are and binary bitstrings in the database? We're trying to write our own utility.
rzscher,
Omeka's OAI-PMH Harvester plugin does not normally associate files with the imported items. To do so, you must edit the Dublin Core mapping class, found here:
plugins/OaipmhHarvester/libraries/OaipmhHarvester/Harvest/OaiDc.php
It's a little tricky, but map the file URL to the "fileMetadata" portion of the returned array in OaipmhHarvester_Harvest_OaiDc::harvestRecord(), following the documentation for the insert_item() function in the following file:
application/libraries/globals.php
Hi,
The CSV import plugin allows to import items metadata and files, that's great. Is there a way to also load files metadata ? I mean, I've got items made of 20-30 files, and each file has DC metadata. How could I manage to load those metadata too ?
Thanks for your answer !
Hi everybody,
I'm currently trying to figure out a problem that the dropbox plugin or CSV plugin could maybe resolve.
This is our situation : we are creating many items via the OAI-PMH plugin. For us the next step is to link some medias to those items (at least part of them). Would it be possible to import a batch of medias and have them linked directly to the right item, via their id for example ?
Thanks again for your help,
Pauline Ziserman.
Never Lost Images.
There's not really a good way to do that with CSV plugin--it will always look to create a new item.
Would it be ok with the dropbox plugin ?
It sounds like Dropbox has the same problem. It doesn't really automatically add files to items. Usually it is used the same way, to create lots of items.
Dropbox will let you add files in the drop box to individual items, but that still means going to each item edit page
One option is to use the OAI data to create a CSV instead of using it to import directly into Omeka. This would allow you to add a column with the location of the files for each of your items. If your metadata isn't too complex, you can probably just import the OAI-DC XML into OpenRefine and then export the result to CSV. More complex data would need to be handled via XSLT or some other method (I use Ruby), but it's still a viable option.
Hi,
Because I have the same needs for some projects, I use a fork of CsvImport to import or to update metadata of items or files,
Sincerely,
Daniel Berthereau
Infodoc & Knowledge management