--------------------------------------------------------------------
CDS Invenio v0.99.0 is released
March 27, 2008
http://cdsware.cern.ch/invenio/news.html
--------------------------------------------------------------------
CDS Invenio v0.99.0 was released on March 27, 2008.
What's new:
-----------
*) new Invenio configuration language, new inveniocfg configuration
tool permitting more runtime changes and enabling separate local
customizations (MiscUtil)
*) phased out WML dependency everywhere (all modules)
*) new common RSS cache implementation (WebSearch)
*) improved access control to the detailed record pages (WebSearch)
*) when searching non-existing collections, do not revert to
searching in public Home anymore (WebSearch)
*) strict calculation of number of hits per multiple collections
(WebSearch)
*) propagate properly language environment in browse pages, thanks to
Ferran Jorba (WebSearch)
*) search results sorting made accentless, thanks to Ferran Jorba
(WebSearch)
*) new OpenURL interface (WebSearch)
*) added new search engine API argument to limit searches to record
creation/modification dates and times instead of hitherto creation
dates only (WebSearch)
*) do not allow HTTP POST method for searches to prevent hidden
mining (WebSearch)
*) added alert and RSS teaser for search engine queries (WebSearch)
*) new optimized index structure for fast integer bit vector
operations, leading to significant indexing time improvements
(MiscUtil, BibIndex, WebSearch)
*) new tab-based organisation of detailed record pages, with new URL
schema (/record/1/usage) and related CSS changes (BibFormat,
MiscUtil, WebComment, WebSearch, WebStyle, WebSubmit)
*) phased out old PHP based code; migration to Python-based output
formats recommended (BibFormat, WebSubmit)
*) new configurability to show/hide specific output formats for
specific collections (BibFormat, WebSearch)
*) new configurability to have specific stemming settings for
specific indexes (BibIndex, WebSearch)
*) optional removal of LaTeX markup for indexer (BibIndex, WebSearch)
*) performance optimization for webcoll and optional arguments to
refresh only parts of collection cache (WebSearch)
*) optional verbosity argument propagation to the output formatter
(BibFormat, WebSearch)
*) new convenient reindex option to the indexer (BibIndex)
*) fixed problem with indexing of some lengthy UTF-8 accented names,
thanks to Theodoros Theodoropoulos for reporting the problem
(BibIndex)
*) fixed full-text indexing of HTML pages (BibIndex)
*) new Stemmer module dependency, fixes issues on 64-bit systems
(BibIndex)
*) fixed download history graph display (BibRank)
*) improved citation ranking and history graphs, introduced
self-citation distinction, added new demo records (BibRank)
*) fixed range redefinition and output message printing problems in
the ranking indexer, thanks to Mike Marino (BibRank)
*) new XSLT output formatter support; phased out old BFX formats
(BibFormat)
*) I18N output messages are now translated in the output formatter
templates (BibFormat)
*) formats fixed to allow multiple author affiliations (BibFormat)
*) improved speed of the record output reformatter in case of large
sets (BibFormat)
*) support for displaying LaTeX forumas via JavaScript (BibFormat)
*) new and improved output formatter elements (BibFormat)
*) new escaping modes for format elements (BibFormat)
*) output format template editor cache and element dependency
checker improvements (BibFormat)
*) output formatter speed improvements in PHP-compatible mode
(BibFormat)
*) new demo submission configuration and approval workflow examples
(WebSubmit)
*) new submission full-text file stamper utility (WebSubmit)
*) new submission icon-creation utility (WebSubmit)
*) separated submission engine and database layer (WebSubmit)
*) submission functions can now access user information (WebSubmit)
*) implemented support for restricted icons (WebSubmit, WebAccess)
*) new full-text file URL and cleaner storage facility; requires file
names to be unique within a given record (WebSearch, WebSubmit)
*) experimental release of the complex approval and refereeing
workflow (WebSubmit)
*) new end-submission functions to move files to storage space
(WebSubmit)
*) added support for MD5 checking of full-text files (WebSubmit)
*) improved behaviour of the submission system with respect to the
browser "back" button (WebSubmit)
*) removed support for submission "cookies" (WebSubmit)
*) flexible report number generation during submission (WebSubmit)
*) added support for optional filtering step in the OAI harvesting
chain (BibHarvest)
*) new text-oriented converter functions IFDEFP, JOINMULTILINES
(BibConvert)
*) selective harvesting improvements, sets, non-standard responses,
safer resumption token handling (BibHarvest)
*) OAI archive configuration improvements: collections retrieval,
multiple set definitions, new clean mode, timezones, and more
(BibHarvest)
*) OAI gateway improvements: XSLT used to produce configurable output
(BibHarvest)
*) added support for "strong tags" that can resist metadata replace
mode (BibUpload)
*) added external OAI ID tag support to the uploader (BibUpload)
*) added support for full-text file transfer during uploading
(BibUpload)
*) preserving full history of all MARCXML versions of a record
(BibEdit, BibUpload)
*) XMLMARC to TextMarc improvements: empty indicators and more
(BibEdit)
*) numerous reference extraction tool improvements: year handling,
LaTeX handling, URLs, journal titles, output methods, and more
(BibEdit)
*) new classification daemon (BibClassify)
*) classification taxonomy caching resulting in speed optimization
(BibClassify)
*) new possibility to define more than one keyword taxonomy per
collection (BibClassify)
*) fixed non-standalone keyword detection, thanks to Annette Holtkamp
(BibClassify)
*) new embedded page generation profiler (WebStyle)
*) new /help pages layout and webdoc formatting tool (WebStyle)
*) new custom style template verification tool (WebStyle)
*) added support for the XML page() output format, suitable for AJAX
interfaces (WebStyle)
*) introduction of navigation menus (WebStyle)
*) general move from HTML to XHTML markup (all modules)
*) fixed alert deletion tool vulnerability (WebAlert)
*) do not advertise baskets/alerts much for guest users; show only
the login link (WebSession)
*) password reset interface improvements (WebSession)
*) new permanent "remember login" mechanism (WebSession, WebAccess)
*) local user passwords are now encrypted (WebSession, WebAccess)
*) new LDAP external authentication plugin (WebAccess)
*) new password reset mechanism using new secure mail cookies and
temporary role membership facilities (WebAccess, WebSession)
*) added support for Single Sign-On Shibboleth based authentication
method (WebAccess)
*) new firewall-like based role definition language, new demo
examples (WebAccess)
*) external authentication and groups improvements: nicknames,
account switching, and more (WebSession, WebAccess)
*) task log viewer integrated in the task monitor (BibSched)
*) new journal creation module (WebJournal)
*) new generic statistic gathering and display facility (WebStat)
*) deployed new common email sending facility (MiscUtil, WebAlert,
WebComment, WebSession, WebSubmit)
*) dropped support for MySQL-4.0, permitting to use clean and strict
UTF-8 storage methods; upgrade of MySQLdb to at least 1.2.1_p2
required (MiscUtil)
*) uncatched exceptions are now being sent by email to the
administrator (MiscUtil, WebStyle)
*) new general garbage collector with a possibility to run via the
task scheduler and a possibility to clean unreferenced
bibliographic values (MiscUtil)
*) new generic SQL and data cacher (MiscUtil)
*) new HTML page validator plugin (MiscUtil)
*) new web test suite running in a real browser (MiscUtil)
*) improved code kwalitee checker (MiscUtil)
*) translation updates: Spanish and Catalan (thanks to Ferran Jorba),
Japanese (Toru Tsuboyama), German (Benedikt Koeppel), Polish
(Zbigniew Szklarz and Zbigniew Leonowicz), Greek (Theodoros
Theodoropoulos), Russian (Yana Osborne), Swedish, Italian, French
*) new translations: Chinese traditional and Chinese simplified
(thanks to Kam-ming Ku)
*) ... plus many other minor bug fixes and improvements
Download:
---------
<http://cdsware.cern.ch/download/cds-invenio-0.99.0.tar.gz>
<http://cdsware.cern.ch/download/cds-invenio-0.99.0.tar.gz.md5>
<http://cdsware.cern.ch/download/cds-invenio-0.99.0.tar.gz.sig>
Installation notes:
-------------------
Please follow the INSTALL file bundled in the distribution tarball.
Upgrade notes:
--------------
If you are upgrading from CDS Invenio v0.92.1, then we advise you to
try the upgrade first on a development server. There was a number of
important changes since the last stable release that might make the
upgrade possibly time-consuming in case you have to dump and reload
your bibliographic tables. Please follow the following steps:
- Launch the bibsched monitor and wait until all active bibsched
tasks are finished. Then put bibsched daemon into manual mode.
- Stop all submission procedures and other write operations. For
example, you may want to stop Apache, edit httpd.conf to
introduce a global site redirect to a temporary splash page
saying that upgrade is in progress, and restart Apache.
- Take a backup of your current MySQL database and your CDS Invenio
installation directory (usually /opt/cds-invenio). Remove your
old /opt/cds-invenio/lib/python/invenio/magic and
/opt/cds-invenio/var/www/help directories.
- First of all, note that CDS Invenio v0.99.0 must use MySQL server
at least 4.1 and the database must be running in UTF-8 mode. If
you have been running older MySQL server, or you have not created
your database and data in default charset UTF-8 but say in
Latin-1, then you must dump and reload all your tables. In order
to check which version and charset you have been using, you can
run:
$ echo "SELECT VERSION()" | /opt/cds-invenio/bin/dbexec
$ echo "SHOW CREATE DATABASE cdsinvenio" | /opt/cds-invenio/bin/dbexec
$ echo "SHOW CREATE TABLE bib10x" | /opt/cds-invenio/bin/dbexec
If you need to convert your tables to UTF-8, then you basically
have to dump the tables, change encoding of the file, and load
the content back into new table definitions. This process may
take some time and carefulness. See for example this blog post:
<http://www.oreillynet.com/onlamp/blog/2006/01/turning_mysql_data_in_latin1_t.html>.
- Second of all, you must upgrade some Python modules, namely your
MySQLdb module as indicated in the INSTALL file. Note that this
will make your current site unusable, but we have stopped Apache
already anyway. If you have been using Stemmer, you must upgrade
its version as indicated in the INSTALL file.
- Untar new sources and rerun configure with old prefix argument
(your old configure line is available at
/opt/cds-invenio-BACKUP/etc/build/config.nice) and install CDS
Invenio (see INSTALL file, part 1a).
- CDS Invenio v0.99.0 uses a new INI-style configuration file
invenio.conf instead of the old config.wml. Please see INSTALL
file, part 1b on how to customize your installation. You will
have to create a 'invenio-local.conf' file where you have to
merge your old config.wml values (they are available at
/opt/cds-invenio-BACKUP/lib/wml/invenio/config.wml). Beware of
variable name updates, e.g. 'filedirsize' now became
'CFG_WEBSUBMIT_FILESYSTEM_BIBDOC_GROUP_LIMIT'.
- If you have previously customized your page header and footer via
WML, then you should now put your page customizations in a new
template skin. Please see the WebStyle Admin Guide for more
information. Also, if you have edited v0.92.1 CSS style sheet,
you may want to merge your changes into new elements of the
0.99.0 style sheet.
- If you have customized your /opt/cds-invenio/etc/ files in the
last installation (for example to edit the stopwords list or
configure language stemmers), then you have to restore your
changes into corresponding /opt/cds-invenio/etc files.
- Update your database table structure:
$ make update-v0.92.1-tables
If you are upgrading from previous CDSware releases such as
v0.7.1, then you may need to run more update statements of this
kind, as indicated in previous RELEASE-NOTES files. (You could
also start afresh, copying your old data into a fresh v0.99.0
installation.)
- If you have previously customized your page templates
(e.g. webbasket_templates.py), then check for changes the new
version brings:
$ make check-custom-templates
This script will check if your customized page templates
(see the WebStyle Admin Guide) still conform to the default
templates. This gives you a chance to fix your templates
before running 'make install', by providing a list of
incompatibilities of your custom templates with the new
versions of the templates.
This step is only useful if you are upgrading and if you
have previously customized default page templates.
- You have also to run manually several migration scripts:
$ python ./modules/webaccess/lib/collection_restrictions_migration_kit.py
This script will migrate restricted collections that used
Apache user groups to a new firewall-like role definition
language.
$ python ./modules/websubmit/lib/fulltext_files_migration_kit.py
This script will check and update your fulltext file
storage system to the new style (e.g. unique document
names).
$ python ./modules/websession/lib/password_migration_kit.py
This script will update your local user table in order to
use encrypted passwords for more security.
- CDS Invenio v0.99.0 uses new, faster word indexer. You will have
to reindex all your indexes by running:
$ /opt/cds-invenio/bin/bibindex -u admin -R
- You have to edit your httpd.conf in order to make Apache aware of
new URLs. Please run:
$ /opt/cds-invenio/bin/inveniocfg --create-apache-conf
and check differences with your current setup and edit as
appropriate.
- Restart Apache and check whether everything is alright with your
system.
Note that the detailed record pages (/record/10) have now a
tab-style look by default. You may want to update your formats
to this style or else to disable this feature. Please see the
WebStyle Admin Guide for more information.
- Put the bibsched daemon back into the automatic mode. You are
done.
Further notes and issues:
-------------------------
*) Some modules of this release (e.g. mail submission system) are
still experimental and not yet activated. You may have a peek at
what is planned, but please do not rely on them.
*) The admin-level functionality of several modules is not fully
developed or documented yet.
What's next:
------------
*) Improving the known issues mentioned above. Strengthening the
documentation towards the v1.0.0 release.
*) Improving the inveniocfg tool with automated upgrade capabilities.
*) Deploying the new URL schema for all pages (admin).
- end of file -