CSV Import jobs failing to complete

I am looking into possible server issues on this, but would like to know if anyone has other ideas. I have never had any problems using CSV Import to bring data into Omeka, but in the last few days all my imports show the number of rows and Queued, but never complete. No items are imported. I checked and the data is being inserted into the imports table. It just does not complete the process of creating items.

Steve

I checked and Omeka is not exceeding the execution time limit. I haven't a clue why the import jobs are staying Queued and never completing.

I'm fairly certain that there is a problem with the CSV data being exported. There must have been a change in the database. I'll look into that next.

Do the imports ever get past the "Queued" state?

If not, that usually means the import isn't even starting. Are you getting any other messages (an Omeka error page, messages in the detailed status page for the imports, messages in Omeka's error log if you have error logging enabled)?

No. They stay in the Queued state.

An Omeka error page is not displayed.

I have checked the PHP error log, the Omeka error log without finding anything suggestive of an issue. Some warnings (from SimpleVocab) because of the debug level, but no errors.

I've confirmed the CSV file I'm testing with has issues. It has data in the wrong columns when I open it in a spreadsheet (Open Office). I need to correct these issues before attempting another import.

I created a test CSV file, which is available here

https://docs.google.com/file/d/0B85KOU2S8m0mNnMxcG1RV1h0aW8/edit?usp=sharing

I have successfully opened this file in Open Office spreadsheet. Note that the field separator is the pipe character. This is a two line file. I started an import at 8/6/13 4:54 PM and it still says Queued.

omeka_csv_import_imported_items shows zero rows
omeka_csv_import_imports shows the one import I started

I deactivated and reactivated the CSV Import plugin and attempted another import, which failed, becoming stuck at the Queued status permanently.

I uninstalled and reinstalled CSV Import hoping that might fix anything that's confusing it. So far with the same result. Stuck at Queued.

I did some digging in the dev list about processes and found this

https://groups.google.com/forum/#!searchin/omeka-dev/_log/omeka-dev/DC75QM0sulI/254p1WVO_QMJ

mention of process logging. I looked at CSV Import Import.php and sure enough it appears to support logging. But I am not seeing anything in the Omeka logs or a process.log

Any help getting process logging to work on CSV Import would be great!

I just need to find out what Omeka is doing or not doing. So I can do something about it.

I've poked around in CSV Import and discovered it has logging of key points in a process lifetime. Through _log(), which I believe is a Zend function. But I'm not seeing these in the Omeka error log. How do I get these log messages to show up?

I'd like to see a message when Import.php starts a process because I'm fairly certain the jobs never start.

Do you have Omeka's error log enabled (in config.ini)?

Mentions of "process.log" or "processes.log" are old. There's no longer a separate log for processes.

Though, because they're all still saying Queued, it does seem most likely that they're simply not being started at all. One of the first things a started import does is set the status to "In Progress."

Yes.

I checked the logs carefully and the debug messages created by CSV Import are getting through to the Omeka log. I only see the queued message:
[CsvImport][time:Aug 8, 2013 12:47:36 PM][id:1] Queued import.

I have replaced all Omeka files with files from the archive file. And replaced all the CSV Import files from the archive file as well. Just in case a file was damaged or mixed up.

I am still not seeing any of these processes start. There should be a message in the log "Started import." right after the status is set to IN_PROGRESS. I can confirm the process is being inserted into the queue table. The temp file is recorded there. I can see the browser uploading the CSV file, although it goes by quick.

This was working before. Something on the server or
something in php configuration may have changed, but I have no idea what it could be.

I am using the test CSV files as well as my own, so I know I am working with a good file.

I moved the existing Omeka files, backed up the database and dropped all the tables, got a fresh set of Omeka files, installed Omeka, installed the CSV Import plugin, tried a test import and its showing In Progress.

I just wish I knew what broke. This shows it is not the server. I can't believe it was a damaged file because I refreshed them straight from the archive, but its possible. It could still be something to do with damage to the database content, something getting deleted or mixed up. I just can't think what it would be.

I'm seeing 3 imported items and the status is Complete. Yay!

I am using the latest Omeka 2 and I have seen this problem on CSV imports I have attempted and while I don't know the precise reason the file got stuck in the queued phase, I found that the best strategy for importing was to deactivate all other plugins. Once I did that, the imports went fine. With your reinstall and other discussions I have seen about CSV import and different plugins, I think your import worked because you had not installed a plugin that was interfering with CSV import.

Yes, it was a conflict between CSV Import and PBCoreElementSet plugin when the 'Add url as identifier' box was checked.

This has happened again. For no obvious reason, CSV Import suddenly began returning "Queued" status for an import, which never shows any activity after queuing.

I deactivated all other plugins without success.

This means I will have to enter data manually for 200 entries.

What happened is that for some reason when I made the supposedly final import of all films including image files, somehow the Vimeo embed code was not imported into Embeded Video field (custom metadata element added to Item Type Moving Picture). I planned to undo the import, then remap it, and run the import again, but since I can't import anything, each embed code will have to be copied and pasted.