These notes describe the Metadata Editor (MDE), a generic Web 2.0 based
service for maintaining high quality, schema-based metadata developed at
the University of Queensland eResearch Unit (UQERU). It requires
a web services container (see dependencies) that provides the launching
point for an interactive, browser based metadata editor. It works in
conjunction with the UQERU's Service Provider Interface (SPI) jar
which must be deployed and configured as described in the documentation
README for that component.
A working MDE installation consists of (at least) the following:
- The MDE package - provides the MDE servlet and all client-side MDE code.
- The SPI package - provides common interfaces for metadata repository access.
- The org.json package - a third-party open source library used by the MDE package.
- A working SPI implementation; e.g. the supplied "Reference Implementation".
- Shared library archives used by the MDE package and implementation.
The server-side components of MDE require a Java 5 JRE and Tomcat 5.5 installation as described
below.
MDE requires some form of repository to hold metadata schemas. Currently there are two alternatives:
- You can use an installation of the UQERU Metadata Schema Repository (MDSR), which is
distributed as part of the UQERU "MSF" tarball. This can be installed into the same
Tomcat server as MDE, or a different one. (MDSR in turn requires a Fedora repository
installation.)
- You can configure and use a static "flat file" schema repository as part of MDE itself.
Metadata schemas for use by MDE are expressed as MSS files. These are prepared using the
Metadata Schema Editor (MSE) that is distributed as part of the UQERU "MSF" tarball.
Copyright and Licensing
The MDE software and documentation are Copyrighted by the University of Queensland, and licensed
under the GPL 3.0. A copy of the GPL 3.0 license may be found in GPL_LICENSE.txt.
The "org.json" library is covered by an open source license that places minimal restrictions
on use, modification and redistribution. The relevant license may be found in JSON_LICENSE.html file.
The MDE package also includes a copy of Ext JS version 2.0.2. This version of Ext JS is
dual licensed under LGPL 3.0 or a commercial license. The relevant license information may be
found in the embedded copy of Ext JS in the mde.zip file; look in the zip file for "/lib/ext/LICENSE.txt".
(Note: later versions of Ext JS replace LGPL 3.0 with GPL 3.0.)
Validated Platforms
This component has been validated for the following platforms, and 3rd-party
components.
Server side:
- Red Hat Enterprise Linux (REHL) 5, Fedora Core 7 & 8
- Tomcat 5.5.27; see limitations for important information about 5.5.26 and earlier.
- (Compatibility with Apache 2.x via Mod-proxy ... see below)
- Sun JRE 1.5.x
Client side:
- Firefox 2.x and 3.0 on Windows XP/XP Pro (SP2), Linux and Mac OSX
- Internet Explorer 7 on Windows XP/XP Pro (SP2)
Installation and Configuration
-
Quick Start
The delivered MDE and SPI components (and Reference Implementation)
have been pre-configured to work together "out of the box" using
default JNDI names, servlet names, and relative paths. For the
simplest deployment:
- If a Tomcat instance is running, stop it.
- Ensure that the SPI jar package has been deployed and configured
as described in the SPI_README file.
- Ensure that the JSON library has been installed
as described in the JSON_README file.
- Important! If you are upgrading from a previous release, be sure
to delete any jar files from
$CATALINA_HOME/common/lib that match the
following patterns before continuing:
au.edu.archer.metadata.msf.mss_*.jar
org.eclipse.*.jar
- Run the Configure.pl script with the
--with-ff-repos option to create
a flat file based schema repository resource bundle named mde.war.
- Copy the
mde.war and mde.refimpl.war files to $CATALINA_HOME/webapps.
- Copy the
au.edu.archer.metadata.msf.mss_*.jar file to
$CATALINA_HOME/common/lib
(the "*" will match the version number in the distribution).
- Copy all the
org.eclipse.*.jar files to $CATALINA_HOME/common/lib.
- Copy the
log4j-1-2.14.jar files to $CATALINA_HOME/common/lib.
- Start Tomcat by running "$CATALINA_HOME/bin/startup.sh", or
"$CATALINA_HOME/bin/catalina.sh start", or the equivalent. (Don't
use "$CATALINA_HOME/bin/jsvc ..."; see known Bugs below.)
- Confirm
that MDE has deployed correctly. If it is able to locate its dependencies,
lines like the following will appear in the log file:
Feb 18, 2008 11:34:22 AM org.apache.catalina.core.ApplicationContext log
INFO: **** MDE repos init complete OK ****
(the logfile is located in $CATALINA_HOME/webapps/mde/logs)
- Confirm that the MDE "refimpl" application has deployed correctly by using a supported
web browser to visit "http://localhost:8080/mde.refimpl". You should see a page with
the heading "Metadata Repository" followed by a table of links for viewing and editing
a collection of sample and test records. Generally speaking, clicking on a "View" or "Edit"
link should launch MDE and load the corresponding record. (Note: some of the test links
towards the bottom of the table refer to malformed records that do not load properly.)
Configuring MDE to use an MDSR
By default metadata schemas are supplied from a simple flat file based
"repository" with the schema files included in the MDE WAR file. If
you have deployed and populated an MDSR with the relevant schemas,
you can configure MDE to use them by changing the "web.xml" file as follows:
- Make a backup copy of the "mde.war" file, then upack it using the "jar" utility.
- Locate the "WEB-INFO/web.xml" file, and open it using your favourite test editor.
- In the editor, locate the following section:
<context-param>
<param-name>registry.wrapper.class</param-name>
<param-value>au.edu.archer.metadata.mde.schema.SchemaRepositoryFileImpl</param-value>
</context-param>
<context-param>
<param-name>registry.uri</param-name>
<param-value>file:test/data/schema</param-value>
</context-param>
- Change it to the following:
<context-param>
<param-name>registry.wrapper.class</param-name>
<param-value>au.edu.archer.metadata.mde.schema.SchemaRepositoryMDSRImpl</param-value>
</context-param>
<context-param>
<param-name>registry.uri</param-name>
<param-value>http://localhost:8080/mdsr</param-value>
</context-param>
replacing "localhost" and "8080" with the MDSR hostname and port number as required.
- Save the file and exit the editor.
- Use the "jar" utility to repack the "mde.jar" file.
- Deploy the "mde.war" file and restart Tomcat.
-
Using an Apache webserver with Mod proxy
Mod proxy allows you to redirect requests from an Apache webserver to some other
server. In this case, we are going to use Mod proxy to redirect to the Tomcat server
that hosts the MDE servlets.
- In your Apach installation tree, make sure that the following line is enabled
in the "httpd.conf". (Remove the leading "#" if it is present.)
LoadModule proxy_module modules/mod_proxy.so
- If you are loading tomcat configuration file from a different location. Find your
tomcat configuration file. Then
- Add the following lines to the tomcat.conf file or to the same "httpd.conf" file
in the case that you are not using a different configuration file for tomcat.
ProxyPass /mde.refimpl/ http://[host*]:[port†]/mde.refimpl/
ProxyPassReverse /mde.refimpl/ http://[host]:[port]/mde.refimpl/
ProxyPass /mde/ http://[host]:[port]/mde/
ProxyPassReverse /mde/ http://[host]:[port]/mde/
* host example: metadata.net
† port example: 8080
- Restart both apache and tomcat.
- Other things to consider in case there are problems:
mod_proxy.so is installed in httpd/modules directory
VirtualHost configurations are correct
-
Custom MDE Installation
To change the servlet name, or JNDI name for the Record Broker Factory,
unpack the war file, edit the WEB_INF/web.xml file, then repack and deploy.
Note that the MDE package ships with a copy of the EXT library in the
WAR file to ensure that the correct, tested version is selected. This
may change in the future as the platform SOE matures.
-
Command-line Tool Installation
The MDE package includes a "bin" and a "lib" directory that contain the wrapper
script for a batch record validator utility.
You can execute the script directly from the location where you have unpacked the MDE
distribution, or you can copy the 'bin' and 'lib' trees to a more appropriate location.
In the latter case, you need to make sure that the 'lib' tree can be accessed from the
'bin' tree as "../lib". If this is not appropriate, edit the script in the 'bin' tree
and alter the "LIB=../lib" line to suit your requirements.
Changes Since Last Release
Release 1.2
- The MDE server-side has been converted to use 'log4j' logging instead of classic
Java logging. The controlling "log4j.properties" file is in the WAR file's WEB-INF
directory. (#203, #234)
- The MDE user interface now includes basic element type information in the hover-text. (#161)
- The MDE application SPI has been extended so that that an application can throw a variety
of Java exceptions to indicate problems. The MDE server-side translates these exceptions
into different HTTP response codes.(#251)
- Metadata record validation now supports complex Constraint expressions involving lists.
New features include a new kind of Operation which supports local 'variables',
and the new Operators 'Let', 'Exists', 'ForAll', 'Map', 'Reduce' and 'Range'; refer
to the MSF User Manual for more details. (Note that the changes to the MSS metamodel are
backwards compatible; existing MSS schema do not need to be converted.) (#58)
-
Release 1.1
- The editor now supports schemas with nested elements. (#197)
- The editor now supports XML date/time format 'gYear', 'gYearMonth', 'date' and 'dateTime'
in elements as specified in the schema. (#32)
- The editor now internally uses arbitrary precision representations for all calculations
involving numeric types. (#113, #171)
- MDE's server-side constraint evaluator now supports collection values, and a
number of new Operators. (#45)
- There is now a command-line tool for batch validating metadata record files. (#16)
- Installation requires deployment of additional archives (JAR files)
to
$CATALINA_HOME/common/lib. See Quickstart
items 7 and 8.
- The editor now displays the 'units' attribute from the record's schema if present. (#57)
- Records may be sent to the editor for read-only display.
- MDE's local MDSR cache lifetime can now be tuned using a servlet context parameter. (#202)
- MDE's default language (for MSS descriptions, etc) can now be set using a servlet
context parameter. (#224)
- The performance of MDE's server-side schema handling and validation has been improved.
Known Limitations
- While MDE does a lot of client-side validation during data entry, it does not do a complete job.
A lot of errors are only detected and displayed when the user invokes server-side validation
by clicking Validate or by attempting to save the current record.
- When server-side validation errors are detected, the MDE client-side adds decorations to the
displayed elements to show the errors. These decorations may be removed or overwritten before
the corresponding error has been corrected. They may be restored by clicking Validate
again. (#175)
- MDE does not support records that are not well-formed XML and do not satisfy the schema
identification and other requirements as documented in the MDE Developer Guide.
- This release of MDE does not support qualified Dublin Core; e.g. 'encoding', 'language'
and element 'refinements'.
- This release of MDE does not support Thesauri or Ontologies. Support for controlled
lists is limited to case sensitive term lists with strict compliance.
- The Settings button on the MDE user interface does nothing. MDE
currently has no user specifiable settings. (#87)
- A bug in Tomcat 5.5.26 and earlier causes truncation of metadata records > 8k bytes
long when they are sent to the MDE server-side. The bug is fixed in Tomcat 5.5.27,
so if this is causing problems for you the solution is to upgrade Tomcat. (#233)
Bugs and Workarounds
- Browser caching of Javascript may hide new or changed functionality.
It has been observed on Firefox 2.x under Linux that the MDE javascript
files referenced by the editor may be served from a stale browser-side cache.
The edit functionality then appears not to have changed, despite a
newer release having been deployed. The current workaround is the
Browser F5 key to force an actual server provided refresh. This should
not be a problem once the distribution stabilizes. However, alternative
approaches to browser-side cache management will be investigated. (#37)
- When Tomcat is launched using the "jsvc" binary using the arguments given
in the Tomcat 5.5 documentation, the container ends up with broken Java
logging. To avoid this, use "startup.sh" or "catalina.sh" to launch
Tomcat. (#140)
- MDE does not allow IEEE 754 "NaN" and "Inf" values in float and double fields.
- The MDE client-side incorrectly reports errors for 'date' and 'dateTime' values
with a negative year representing BC dates. Ignore the error indications is you need
to enter BC dates. (The MDE server-side validator allows these values, so they
should not be flagged as errors when you save a record.)
- The MDE client-side does not validate all components of Date values. For example,
it does not fully validate the minutes and seconds component of a 'dateTime' or the
hours and minutes components of timezone offsets. (MDE server-side validation will
detect and report the errors.)
Bugs Fixed in this Release
Release 1.2
- MDE was not correctly reading records with large text fields (>4Kbytes) in Firefox. (#77)
- MDE was not correctly reading records with large text fields (>4Kbytes) in Firefox. (#77)
- MDE's client-side used to reject records where the xsi:schemaLocation attribute included
(harmless) undeclared namespaces. (#133)
- MDE's Save and Cancel functions were incorrectly reporting validation
errors after previous errors had been corrected and saved. (#231)
- An element's 'dirty' flag was not consistently cleared following a successful save.
This resulted in MDE incorrectly reporting that the record needed saving when the
Close button was clicked. (#232).
- A number of MDE client-side bugs in the handling of 'unknown' elements have been fixed.
(#124, #239, #241, #242)
-
Release 1.1
- A user can now identify the schema that is used by the metadata record currently
open in the editor.
Positioning the mouse pointer over the root element of the Tree View panel will
display the schema identifier as hover-text. If any description of the schema is
available, it will be displayed following the identifier. Users should include
this information when reporting defects.
- XML elements nested within elements that supposedly represent metadata fields
caused the MDE client-side code to hang on IE7. (#97)
- MDE's server-side constraint evaluation code did not detect overflow and underflow in
integer computations. (#171)
- MDE's server-side validation now reports XML element names with incorrect casing as a
distinct error case. (#194)
- MDE previously would "gobble" invalid characters entered into numeric fields
without providing feedback to the user. It now add the characters to the
field, and highlight the field as being in error. A hover text error message is
provided. (#228)
- MDE displayed the wrong descriptions for elements in hover text help. (#210)
-oOo-
The ARCHER project is a joint development by the James Cook University, Monash University,
and the University of Queensland.
It is funded by the Department of Innovation, Industry, Science and Research (DIISR) through the
Australian Research Information Infrastructure Committee (ARIIC), 2008.