We've installed Neatline 2.0.1 with Omeka 2.0.3. I'm unable to successfully query any Omeka items into Neatline (using ranges, tags, etc.) I get a lovely green message saying that the query has been successful, but there are no records in Neatline.
Additionally, when in Neatline, if I type into the Omeka ID box to try to connect a Neatline record to an Omeka record, nothing happens (i.e., no autocomplete, nothing).
I've tried this in Chrome and Firefox with the same results.
Ideas?
Thanks!
JenB
Having the same problem with imports, though it looks like I was able to connect Neatline with an Omeka item while creating a Neatline record. (At first no, then yes, after saving an unconnected record first and then connecting it.) Is there a Neatline-specific error log? Nothing's going into the Omeka log. Same versions of each.
Hi folks,
This seems to be a problem for a couple of people, and we're working on a solution. When items are imported into a Neatline exhibit, Neatline spins up a "background process" behind the scenes to do the computationally-expensive work of creating all the new records. On some configurations, it seems that this process isn't getting dispatched properly. It's a tricky bug, because it seems to be tied to the OS - things work fine on Ubuntu, but apparently don't work on CentOS, etc.
Triplingual, what flavor of linux are you working with? I'm going to spin up a bunch of boxes on AWS and see if I can figure out what's going on.
For the folks working on the Omeka core - have you ever run into similar problems with long-running background jobs? As far as I can tell, Neatline is dispatching the jobs in a pretty standard way (see https://github.com/scholarslab/Neatline/blob/436594f2112ea62ffd29ccb75f696f3907bf79e4/controllers/ExhibitsController.php#L86...L92). But it's possible that we're doing something weird in the code that's tripping things up.
Either way, this is high priority, and I'll try to get a fix out in the next couple days. One easy solution would be to move this stuff out of a background process and just run it inside of a regular web request. That would fix the problem, but it would make it difficult/impossible to import really large batches of Omeka items (eg, many thousands). But that may also be so rare that it's not worth accommodating.
Trip and Jen - how many items are you working with? What's the largest number that you could imagine ever needing to import?
Thanks for the info,
David
Funny that you mention CentOS, 'cause I'm on 5.8 of that. Our total number of items is a bit TBD, as we are going to be working with students over the course of a semester. To pull out one possibly obvious implication of that, we will be loading in N items at the start and then incrementing. N is almost certainly < 200. Largest number to import ever (for certain values of 'ever') is not substantively different, since we are very much in pilot mode.
If you start getting bogged down in the details of CentOS, I can probably get you in touch with our sysadmin so you could get the finer grains of our environment.
Just wanting to pull in a bit more info here; anyone using selinux? If so, what mode are you running? Also, what version of php is running? So you don't have to look it up...
$ sestatus
$ php -v
Hey Jen and Trip,
Ok, after talking to John and Wayne, let's try this. Open up the Omeka config.ini file (under application/config/config.ini) and find the jobs.dispatcher.longRunning key at line 166. Change the value to Omeka_Job_Dispatcher_Adapter_Synchronous. So, the line should read:
jobs.dispatcher.longRunning = "Omeka_Job_Dispatcher_Adapter_Synchronous"
Save the file, and try running the import again. Does it work now?
-David
My result:
Omeka has encountered an error
Zend_Controller_Exception
Could not find file css/exhibits.css!
(and much more, can post the dump if needed)
I deleted my initial test exhibit, created a new one, and ran the import with _Synchronous to get this error. Flopping the config.ini entry back to _BackgroundProcess left me with no imported items, but also no Zend error. Tried this with ID range 1-100 as well as single ID of 1.
And to answer Wayne's questions:
Not using selinux, PHP version is 5.3.3 (Zend engine 2.3.0, FWIW)
Hey Trip,
Ok, that's actually a good sign - since this code was designed to run inside of a separate process, we'll need to change some things to make it work inside a regular web request.
I'll try to fix this and cut out a new release ASAP. Thanks for bringing this to our attention!
-David
Happy to help, and happy to help ongoing as this gets sorted.
Oh, and would the whole error trace be useful to you?
Hey Trip,
Yeah, that would be great. Send it to david.mcclure<at>virginia.edu.
Thanks!
David
Hi Trip and Jen,
Alright, I just built out two completely fresh Centos machines (5.8 and 6.4) and tried this out. Jen, I can reproduce the second problem you were encountering:
----
Additionally, when in Neatline, if I type into the Omeka ID box to try to connect a Neatline record to an Omeka record, nothing happens (i.e., no autocomplete, nothing).
----
This is because the php-xml extension isn't installed on the system. Turns out that there's some code in Omeka's search API that makes use of the DOMDocument class, which isn't present unless php-xml is installed. This is an easy fix - just install the package and restart apache:
sudo yum install php-xml
sudo service httpd restart
As for the bigger problem with the batch-imports - I just can't reproduce this. It works fine for me on really barebones installs of both 5.8 and 6.4. Let's try these steps:
1. Open up the .htaccess file in the Omeka root. Uncomment line 14 (SetEnv APPLICATION_ENV development). Then, go back and try to run an import. Do you get an error message? If so, what?
2. Open up the Omeka config file (application/config/config.ini) and change line 87 to log.errors = true. Then, try to run the import a couple times, and send me log files under application/logs.
We'll get to the bottom of this yet.
-David
I'll try to get a request in for adding the package, since unfortunately I don't have root on our box. It may be a bit, as I am on vacation right now (in Scholars' Lab metro area, in fact) and as such am spending most of my time on other things. The error traces should be easier, though.
David..
Hello I have the same problem....
Can work under ubuntu root and install php.xml and change the lines!?
regards
gus
www.koluel.org
Gus,
On Ubuntu, I believe the package is php-xsl, but the rest should world. For the Ubuntu packages needed, you can check out this:
http://www.scholarslab.org/research-and-development/installing-omeka-through-amazon-web-services/#step19
Wayne
it is ok now..
by the way the command in ubunt for php-xsl is
sudo apt-get install php5-xsl
another questions..
Why when I search items of Neatline....I can not find anything!!
Please if you search in
http://www.koluel.org/neatline/show/huelgas-personal-ferrocarril
the item "Huelgas personal ferrocarril" in search omeka .... I don't see the exibition
gus
Hi Gus,
We should start a new topic on the search, but basically there's not an API that allows plugins to hook in to the search for Omeka. If you want/need to create items that are searchable, you'll need to create Omeka items (and make them public), then import them in to Neatline.
Wayne
Hi David--
Thanks so much for following up.
1) re the autocomplete issue: we added the php.xml file and it now appears to work... however... we only get results if we type the entire string [eg., if we search for 'College Hall' you need to type all of 'College Hall'; no results are returned if you just type 'College'.] Is that expected? Plus, the results that are returned are correct, but the drop-down list is populated by the Collection Name only... not terribly useful?! (It's hard to determine which one you want when all the selections are named the same!)
2)We edited the .htaccess file as instructed and got no error message-- we got the same green success message. The error log I will send via email so as not to clog up this page.
Also, we don't have huge collections-- Neatline exhibits won't be larger than a few dozen objects?
Thanks very much,
Jen
Hi folks,
Alright, thanks to a stack trace that JenB sent over, I think I see what's going on here. Basically, the default template that Neatline uses to generate the metadata for imported items fires off a "hook" that allows other plugins to inject new information about the item. If other plugins are installed that both (a) make use of that hook but (b) don't work when run inside the context of the background job, errors occur when Neatline fires the hook, causing the import to fail. For example, in Jen's case, the Coins plugin trips things up.
JenB and triplingual - before I push out a new release, could you do a little experiment, just to make sure this is the problem?
1. Open up the default item.php template in Neatline - plugins/Neatline/views/shared/exhibits/item.php.
2. Delete lines 22-24, under the <!-- Plugins. --> header.
3. Save the file, and try to run an item import. Does it work?
Let me know. If this is the problem, it's trivial to fix.
Thanks,
David
Fantastic! It works! Thank you! [Is that all we need to do at this point? no updates needed?]
Now for a follow-up question on the finer points of querying: I can query the collection based on tag, range, etc. just fine. If I want to go back and edit the query, however, (eg, adding an additional tag) it doesn't seem to pick it up. I know in the previous (1.x) version of Neatline, once an item was queried into Neatline, it stayed there (ie, any new search would not "remove" that item from the list of results). Is that the case with 2.x? and is it also the case that once a query has been performed, you can't expand it either? (I guess at that point you would just drag add'l things in from the Neatline side... yes?)
Thanks supermuch!
JenB
Back from vacation today, and the fix is also fantastic here. What are the next steps? (Would the error trace from the ASYNC trial be useful still? My guess is no, but I can replicate it again if needed. We've got a metric ton of plugins installed, or at least it feels like it.)
I'll go back and look into the php-xml package, but don't keep this thread open for me on that account.
Hey Jen and Trip,
Great, I just wanted to make sure that this was actually getting at the problem on your systems. I'm going to fix one other small thing this morning, and then I'll cut off a point release with the change.
Jen, about the querying behavior - basically, you can run the import as many times as you want, and Neatline will always guarantee that exactly one instance of all the matching items is imported into the exhibit. For example, say you have a collection with 100 items in it, and import it into Neatline. As you make changes to those individual items, the instantiations of those items are updated in the exhibits. Then, later on, you add 50 new items to the collection. If you wanted to "synchronize" an exhibit with the new items in the collection, you could just go to "Import Omeka Items" and run the import again, and Neatline would just add the 50 _new_ items in the collection, and wouldn't duplicate the original 100 items. Think of importing items as "pulling" from the collection - you can do it as many times as needed, and you don't have to worry about duplicating content.
As for changing the query altogether - that's fine, but Neatline won't automatically delete items that were imported according to the original query. Eg, if you import 100 items from Collection 1, and then change the query to Collection 2, you'll have all the original Collection 1 and the Collection 2 items. Collection 1 won't be deleted. This is in contrast to the way it worked in the first version, where Collection 1 records _would_ have been removed from the Neatline exhibit. This was problematic because it means that you could have "orphan" records that had Neatline-specific data, but weren't listed in the editor because they were excluded from the query.
I'll post the new version shortly. Thanks for the help with this!
-David
Hi David,
I am also experiencing this issue. Have you released the updated code yet?
Thanks!
Brianna
Hi Brianna,
Yep, we just released 2.0.2, which includes the fix!
http://www.scholarslab.org/geospatial-and-temporal/announcing-neatline-2-0-2/
-David
Folks,
Alright, I just posted a second hotfix release (2.0.3) that repairs another problem with the item-import workflow, this time related to a difference in the code environment between the synchronous/asynchronous job adapters, which can be swapped in and out in the config.ini. (Thanks Caleb Derven for bringing this to my attention.)
This should definitively resolve the problem with all combinations of plugins, and with all configurations of the Omeka installation. Again, thanks all for helping us track this down. Let us know if you run into any other problems!
-David