User Tools

Site Tools


knb_dev_admin_quick_reference_guide

Computing Staff Information → KNB Dev/Admin Quick Reference Guide This page serves as a central point of reference for KNB developers and

Metacat Administrator Guide

The primary souce of information on installing and configuring Metacat is the Metacat Administrator's Guide

Included are sections on:

  • Downloading and Installing Metacat
  • Configuring Metacat
  • Accessing and Submitting Metadata and Data
  • Metacat's Use of Geoserver
  • Replication
  • Harvester and Harvest List Editor
  • Event Logging
  • Enabling Web Searches: Sitemaps
  • Creating a Java Class that Implements AuthInterface

Metacat Installation And Maintenance Information

For information on building, installing and maintaining Metacat on KNB and other sites for which NCEAS has responsibility, see Metacat and KNB Web Maintenance page on this wiki.

Included are sections on:

  • Building, Installing and Publishing Metacat Release Distribution
  • Update KNB Web Service
  • Build and Install Registry Web Application
  • Site Specific Notes
    • ESA
    • SANParks (Skukuza)
    • SAEON (Pretoria)
    • SAEON (Phalaborwa)
    • SAEON (Cape Town)
  • Testing Installation
  • Common problems and issues
  • Set up JMX and run JConsole

Metacat Development Information

For information on Metacat architecture and development issues, please refer to Metacat Development Guide. Note that if you want to update or view the most current version of these docs, you can find them in the metacat project code at https://code.ecoinformatics.org/code/metacat/trunk/docs/dev.

Included in this documentation is:

  • Metacat Architecture
    • Overview
    • Query Caching and Indexing Mechanisms
    • Service Based Architecture
    • Utilities Classes
  • Setting up a Metacat Eclipse project
  • Building Metacat (dev specific)
  • Troubleshooting Metacat

Workflow Scheduler Development Information

For information on Workflow Scheduler architecture and development issues, please refer to Workflow Scheduler Page. Note that if you want to update or view the most current version of these docs, you can find them in the wfscheduler project code at https://code.ecoinformatics.org/code/wfscheduler/trunk/docs/dev.

Included in this documentation is:

  • System Overview
  • Scheduling a Workflow
  • Using the Quartz Scheduler
  • Building and Installing Workflow Scheduler

Commonly Used Links

Here are some of the more commonly used links for a KNB admin. Note that most of the metacat servers be reached via ssh using the server name in the url, even thought that may not be the physical server name, (assuming you have an account on that server).

A few Metacat comments - April 16, 2010

These are just a few random comments on existing and pressing issues with Metacat at this time.

Geoserver / Spatial files

In general, geoserver needs to be upgraded. It is out of date.

One of the primary issues we have is with the creation of spatial files. Basically, at post installation start-up time Metacat grabs spatial coordinates from all documents and re-creates the spatial files. This re-creation can take upwards of 45 minutes on KNB. The re-creation is necessary because the spatial files are located inside the webapp directory and are not persisted across installs.

A workaround for the spatial file issue is to copy in all files into /metacat/data/metacat_shps from a backup. Then set the spatial.regenerateCacheOnRestart to false in metacat.properties.

At the time of this writing, I looked into moving the data directory out of the webapp. This can be done, however, an environment variable (GEOSERVER_DATA_DIR) must be set in web.xml. This is a problem, because we have no knowledge of each system and where the data dir should go until AFTER we start tomcat. By that point, the web.xml has been read.

Another option is to explore the use of a spatial database.

Conflicting Libraries

Right now metacat pulls it's libraries from three locations:

  • metacat/lib
  • metacat/lib/lsid_lib
  • metacat/lib/spatial/geoserver/WEB-INF/lib

There are several duplications of libraries. Some of these duplications are named differently, (one jar has a version number and one doesn't). It's not easy to tell immediately if these are the same version. And then there are a few jars with multiple versions included. These don't seem to cause issues currently, but should definitely be cleaned up.

Errors in Logs

In release 1.9.2, a concerted effort was made to reduce the number of false error messages in the logs. The goal would be to only have errors in the logs that are actionable.

There are still a few errors that stem from the fact that exceptions are thrown from several layers deep and passed as general Exceptions up to the Metacat servlet layer. The servlet reports all exceptions as errors in the logs. However, there are some exceptions that are being used to pass on some business information. These are not errors per se. Some examples are when an attempt is made to read a doc id when that doc id does not exist, or an attempt to read a doc when the user does not have permissions. These should probably throw some specific exceptions that can be handled. At best, they should probably report as a warning in the logs.

KNB Server Specs

Currently the KNB server is a 32 bit RHEL machine. This means the OS is limited to ~3GB memory per process. Metacat can actually push that memory limit depending on what's going on. A 64 bit machine can allocate ~16GB per process.

One note: at one point, when we were seeing memory issues, I separated out the knb and the kepler metacat instances into completely different tomcat instances. For an entrypoint into how this is configured, see the two different startup scripts at:

  • /etc/init.d/tomcat
  • /etc/init.d/tomcat-kepler

Ecogrid and Metacat

The metacat build currently relies on a heavy dependency with ecogrid (in the seek project). When you build metacat, it does it's own build, then makes a call to the ecogrid (seek) project, which then re-builds the metacat war with the ecogrid web service api included. This is a very time consuming process. There is some talk of phasing out ecogrid, so one of the following two options should be explored:

If ecogrid is going away in a reasonable amount of time, simply remove all ecogrid dependencies from metacat.

If ecogrid will be around for a while, then separate ecogrid and metacat, possibly making the ecogrid web service stuff into a library that can be included in metacat.

knb_dev_admin_quick_reference_guide.txt · Last modified: 2012/02/07 11:32 (external edit)