EEA & Eionet documentation hub

Browse documentation for IT-systems used by the European Environment Agency and the Eionet network.

EEA Similarity

eea.similarity

http://ci.eionet.europa.eu/job/eea/job/eea.similarity/job/master/badge/icon

Introduction

EEA Similarity is a Plone add-on which provides content suggestions based on similarity scores to a search string. It uses NLP algorithms like TF-IDF (frequency-inverse document frequency) and LSI (Latent Semantic Indexing).

Initial use case is to provide a list of possible duplicates when adding content (based on the entered title).

Contents

  • Introduction
  • Installation
    • zc.buildout
  • Getting started
  • Dependencies
    • Possible issues with scipy and numpy:
  • Source code
  • Copyright and license
  • Funding

Installation

zc.buildout

If you are using zc.buildout and the plone.recipe.zope2instance recipe to manage your project, you can do this:

  • Update your buildout.cfg file:

    • Add eea.similarity to the list of eggs to install
    • Tell the plone.recipe.zope2instance recipe to install a ZCML slug

      [instance] … eggs = … eea.similarity

    zcml = … eea.similarity

  • Re-run buildout, e.g. with:

    $ ./bin/buildout
    

You can skip the ZCML slug if you are going to explicitly include the package from another package’s configure.zcml file.

Getting started

  1. Go to Site Setup > Add-ons and install EEA Similarity
  2. Create IDF index by calling @@create_idf_index Browser view on site root. If you have plone.app.async installed, it will add a daily job to async queue, otherwise you’ll have to setup an external cron job to call this periodically.
  3. Customize settings via Site Setup > EEA Similarity Settings

Dependencies

EEA Similarity has the following dependencies:

  • stemming
  • gensim (which in turn depends on numpy and scipy)

On CentOS you need to install:

  • blas-devel
  • lapack-devel
  • gcc-fortran

On Debian/Ubuntu you need to install:

  • libblas-dev
  • liblapack
  • gfortran

Possible issues with scipy and numpy:

Due to a bug in scipy, it is possible that the packages will not install when running buildout. There are two solutions/options:

  1. Install them with easy_install pip before running the buildout.
  2. Update zc.buildout to version 2.9.0 and activate the wheel support via buildout.wheel and the dependencies will be installed normally.

Source code

The Initial Owner of the Original Code is European Environment Agency (EEA). All Rights Reserved.

The EEA Similarity (the Original Code) is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

More details in License.txt

Funding

EEA - European Environment Agency (EU)

Edit this page