User Tools

Site Tools


knb_and_esa_configuration_and_upgrading

KNB and ESA Configuration and Upgrading

Upgrade Steps

For KNB machine:

  1. Stop tomcat as root user: /etc/init.d/tomcat stop

for knb metacat:

2. Backup knb directory at /var/www/org.ecoinformatics.knb as user webuser.

3. Backup knb.lastid, nceas.lastid and nrs.lastid files in knb directory as user webuser.

4. In /home/webuser/metacat-install directory, edit build.properties file (Here is the example: /home/webuser/metacat-install/metacat-1.8.1/metacat-1.8.1/build.properties).

5. Edit build.xml file: in target “deploy-ecogrid”, change the value of property “hostname” from localhost to “knb.ecoinformatics.org”.

6. Edit lib/style/skins/nceas/nceas.cfg file: modify the password value for ldap user and password value for nceas db.

7. Edit seek configuration. In seek directory, which is parallel to metacat source directory, edit both projects/ecogrid/conf/MetacatImpl/classes/metacat.propertie and projects/ecogrid/conf/IdentifierService/classes/metacat.properties - change the value of metacatURL to http:knb.ecoinformatics.org/knb/metacat

8. Run “ant clean update” under knb metacat directory.

9. Run “ant install-ldap” to install ldap script as root.

10. Run “ant install-registry” to install registry scirpt as root.

11. Copy /var/www/cgi-knb/backup-12102007/ldapweb.cfg to overwrite /var/www/cgi-knb/ldapweb.cfg as root.

12. Copy backuped knb.lastid, nceas.lastid and nrs.lastid files back. Their locations are knb/style/skins/foo/foo.lastid (foo=knb, nceas and nrs). for kepler metacat:

13. Backup kepler directory at /var/www/org.kepler-project.library as user webuser.

14. In /home/webuser/kepler directory, edit build.properties file (Here is the example: /home/webuser/kepler/metacat-1.8.1/metacat-1.8.1/build.properties).

15. Edit build.xml file: in target “deploy-ecogrid”, change the value of property “hostname” from localhost to “library.kepler-project.org”.

16. Edit seek configuration. In seek directory, which is parallel to metacat source directory, edit both projects/ecogrid/conf/MetacatImpl/classes/metacat.properties and projects/ecogrid/conf/IdentifierService/classes/metacat.properties - change the value of metacatURL to http:/library.kepler-project.org/kepler/metacat

17. Run “ant clean update” under kepler metacat directory.

18. Restart tomcat as root: /etc/init.d/tomcat start

19. It will take a while (30 minutes and more) to build spatial cache during tomcat starting process.

20. After tomcat is up, run “ant deploy-ecogrid” under knb metacat source directory.

21. After tomcat is up, run “ant deploy-ecogrid” under kepler metacat source directory.

KNB Web Service:

The KNB web service is the web interface that provides basic KNB services. Included are:

  • A basic interface with the KNB metacat instance
  • Software documentation and installation instructions
  • Product downloads

The KNB web service is held in cvs in the knbweb module. This module must be checked out and installed via ant. The installation will copy knbweb files into the webapp/ROOT directory.

Individual software distributions are built on a product by product basis and copied into the knbweb web application as needed.

The steps to install KNB web are as follows:

  • Check Out KNB Web Code
    • log in as “webuser” user (password is available from knb admin)
    • cd to /home/webuser/project/knbweb
    • type 'cvs update' to get latest knbweb code (or 'cvs checkout knbweb' if it does not already exist)
  • Install full KNB Web site
    • type 'ant knbinstall' to do a full install of knbweb
    • all modules (products) should get checked out from cvs/svn
  • Update Application Module Code in KNB web
    • type 'ant <application>update' to upgrade just the doc files for given application.
    • i.e. to update metacat documentation in KNB web, run 'ant metacatupdate'
  • Update KNB Web Code
    • type 'ant knbsimpleinstall' to upgrade just the web files for knb web.
  • Desired Installation Results
    • KNB web code will wind up in /var/www/edu.ucsb.nceas.knb/ROOT
    • The knb web site can be seen at http:knb.ecoinformatics.org (as opposed to knb metacat instance which is at http:knb.ecoinformatics.org/knb)
  • Update software distribution Download Files
    • the distributions will need to be built for the desired applications
    • check out the source code for the application on dev.nceas.ucsb.edu.
    • in the desired application do 'ant distsrc' (and 'ant distbin' if there is a binary distribution)
    • scp the tgz and zip files to knb.nceas.ucsb.edu:/var/www/edu.ucsb.nceas.knb/ROOT/software/dist
    • make sure the links in http:knb.ecoinformatics.org/software/download.html link to the new distribution files you uploaded. If not, you will need to change the links in the knbweb code and follow the upgrade steps shown above.

For ESA machine:

1. Stop tomcat as root user: /etc/init.d/tomcat stop 2. Backup esa directory at/var/www/org.esa.data/ as user webuser. 3. Edit build.properties file under /home/webuser/metacat-1.8.1/metacat-1.8.1 directory. Here is the example: /home/webuser/metacat-1.8.1/metacat-1.8.1/build.properties 4. Edit build.xml file: in target “deploy-ecogrid”, change the value of property “hostname” from localhost to “data.esa.org”. 5. Edit lib/style/skins/esa/esa.cfg: modify the password for ldap user. 6. Edit seek configuration. In seek directory, which is parallel to metacat source directory, edit projects/ecogrid/conf/MetacatImpl/classes/metacat.properties - change the value of metacatURL to http:data.esa.org/esa/metacat 7. Run “ant clean update” under knb metacat directory. 8. Run “ant install-ldap” to install ldap script as root. 9. Run “ant install-registry” to install registry scirpt as root. 10. Copy /var/www/cgi-esa/12102007backup/ldapweb.cfg to overwrite /var/www/cgi-esa/ldapweb.cfg as root. 11. Restart tomcat as root: /etc/init.d/tomcat start 12. After tomcat is up, run “ant deploy-ecogrid” under esa metacat source directory.

Services to be tested

This is not completed test, but will cover most cases. I usually don't test insert and update since I don't want to put some garbage into production server.

For knb metacat at knb machine: 1. Login and logout in knb web site. 2. Some search in knb web site. 3. Read eml document in knb web site. 4. Login and logout in nceas skin. 5. Some search in nceas skin. 6. Read eml document in nceas skin. 7. Login, logout, search and read in obfs and nrs skin. 8. Check http:knb.ecoinformatics.org/knb/services to see if it has AuthenticatedQueryService, IdentifierService, PutService, QueryService,AuthenticationService and wsdl files work well. 9. In kepler, search data and download data.

For kepler metacat at knb machine: 1. Check http:library.kepler-project.org/kepler/services to see if it has AuthenticatedQueryService, IdentifierService, PutService, QueryService,AuthenticationService and wsdl files work well. 2. In kepler, search remote actor repository. 3. In kepler, search ecogrid service provider.

For esa machine: 1. Check http:data.esa.org/esa/services to see if it has AuthenticatedQueryService, IdentifierService, PutService, QueryService,AuthenticationService and wsdl files work well. 2. Login, logout, search and read eml document as regular user. 3. Login, logout, search and read eml document as moderator user.

Common problems and issues

1. No IdentifierService at page http:knb.ecoinformatics.org/knb/services: forget run “ant deploy-ecogrid”. 2. Get error “no such host….” when you run “ant deploy-ecogrid”: forget to modify build.xml - change hostname from “localhost” to domain name. 3. Couldn't open NCEAS registry form: forget to put correct password at nceas.cfg. 4. There is no attaching data file option at NCEAS registry form: the /tmp/knb directory is gone and we need to create one. 5. User couldn't add new ldap user or modify ldap user password: forget to replace ldap.cfg from the backup one.

Set up JMX and run jconsole

Java Management Extensions (JMX) allow for monitoring and managing applications. The jvm (and applications) can make information available via management beans (MBeans). A great deal of information is automatically available from the JVM, such as thread information, memory usage, etc. The information can be gathered at the command line and made available to a monitoring system using a program called jvmstat. A gui program can be used to view the information real time as well. Here we will discuss using JConsole, which ships with sun java. — Setting up JMX on the server — You will need to add some java arguments to the startup command for Tomcat. If you have a catalina.sh in your <tomcat_home>/bin directory, you can add the args to CATALINA_OPTS. If not, there are different startup files for different tomcat installations, so you will need to decide where the additional arguments should live. The most important consideration is that the arguments show only during start/run commands somewhere before the org.apache.catalina.startup.Bootstrap element.

For more information on JMX configuration, refer to http:java.sun.com/j2se/1.5.0/docs/guide/management/agent.html For more informtaion on JConsole client, refer to http:java.sun.com/j2se/1.5.0/docs/guide/management/jconsole.html

  • Add the following arguments to your CATALINA_OPTS in catalina.sh (or other location of your choice).
    • -Dcom.sun.management.jmxremote.port=8686 - You can choose a different port if this one conflicts
    • -Dcom.sun.management.jmxremote.ssl=false - Do not use ssl. You can change this to true if you like, but you will have to take care of exchanging ssl certificates.
    • -Djava.rmi.server.hostname=128.111.242.154 - This tells the rmi server to register itself against the server's ip. Use the phyiscal IP of the server.
  • add access - this configuration has password control. You can set the authenticate option to false and disable password protection.
    • Set up the password file
  • If it does not already exist, copy <JAVA_HOME>/lib/management/jmxremote/jmxremote.password.template to jmxremote.password.
  • Add a line at the bottom that looks like: 'webuser <web_user_password>', or whatever user password combo you want.
  • Exit and save the password file.
  • Change ownership of the jmxremote.password file to be the user that runs Tomcat: 'chown <tomcat_user> jmxremote.password'
  • Change the permissions to be read/write by the owner only: 'chmod 600 jmxremote.password'
    • Set up the access file
  • Edit <JAVA_HOME>/lib/management/jmxremote/jmxremote.access .
  • Add a line at the bottom that looks like: 'webuser readonly' or 'webuser readonly' depending on what access you want to grant. You will need a line in this file for every line you created in the password file.
  • Exit and save the access file.
  • Change ownership of the jmxremote.access file to be the user that runs Tomcat: 'chown <tomcat_user> jmxremote.access'
  • Change the permissions to be read/write by the owner only: 'chmod 600 jmxremote.access'
  • Restart tomcat
    • Check that the new parameters show up by typing: 'ps auxww | grep tomcat'
    • You should see all the parameters you entered above
    • They should all appear before org.apache.catalina.startup.Bootstrap
    • Make sure JMX is listening by typing 'netstat -an |grep 8686' (or your port). You should see that port listening.
  • Run a jconsole client against the server.
    • On a client machine, run '<JAVA_HOME>/bin/jconsole &'. You should see a gui console pop up with a 'Connect to Agent' dialog.
    • Go to the remote tab
    • Enter the host name, port, username and password for your server and hit connect.
    • You should see a summary window for the server
    • There are several things that can cause the connection to fail, but a common one is not having firewall access to the server. Fixing this is not graceful, since you have to have access to two ports: the one that JMX listens on and the RMI port that was set up dynamically by the server. Unfortunately, there is no easy way to set the port for RMI, so your client must pretty much be opened up on the server.
    • You can refer to the documentation above for what all the different tabs mean, but the memory tab is the most common section used for troubleshooting the JVM and GC.
knb_and_esa_configuration_and_upgrading.txt · Last modified: 2012/10/19 15:16 by brand