Chapter 16. HyperSQL on UNIX

This chapter explains how to quickly install, run, and use a HyperSQL Listener (aka Server) on UNIX.

Note that, unlike a traditional database server, there are many use cases where it makes sense to run HyperSQL without any listener. This type of setup is called in-process, and is not covered here, since there is no UNIX-specific setup in that case.

I intend to cover what I think is the most common UNIX setup: To run a multi-user, externally-accessible catalog with permanent data persistence. (By the latter I mean that data is stored to disk so that the catalog data will persist across process shutdowns and startups). I also cover how to run the Listener as a system daemon.

When I give sample shell commands below, I use commands which will work in Bourne-compatible shells, including Bash and Korn. Users who insist on using the inferior C-shells will need to convert.

Go to http://sourceforge.net/projects/hsqldb and click on the "files" link. You want the current version. I can't be more specific because SourceForge/Geeknet are likely to continue changing their interface. See if there's a distribution for the current HSQLDB version in the format that you want.

If you want a binary package and we either don't provide it, or you prefer somebody else's build, you should still find out the current version of HyperSQL available at SourceForge. It's very likely that you can find a binary package for your UNIX variant with your OS distributor, http://www.jpackage.org/, http://sunfreeware.com/, etc. Nowadays, most UNIXes have software package management systems which check Internet repositories. Just search the repositories for "hsqldb" and "hypersql". The challenge is to find an up-to-date package. You will get better features and support if you work with the current stable release of HyperSQL. (In particular, HyperSQL version 2.0.0 added tons of new features). Pay attention to what JVM versions your binary package supports. Our builds (version 2.0 and later) document the Java version it was built with in the file doc/index.html, but you can't depend on this if somebody else assembled your distribution. Java jar files are generally compatible with the same or greater major versions. For example,if your hsqldb.jar was built with Java 6, then it is compatible with Java versions 6 and greater.

	Note
	It could very well happen that some of the file formats which I discuss below are not in fact offered. If so, then we have not gotten around to building them.

Binary installation depends on the package format that you downloaded.

 uncompress filename.pkg.Z

 pkginfo -l -d filename.pkg

 pkgadd -d filename.pkg

 rpm -qip /path/to/file.rpm

 rpm -Uvh /path/to/file.rpm

 cd ancestor/of/new/hsqldb/home
 unzip /path/to/file.zip

Take a look at the files you installed. (Under hsqldb for zip file installations. Otherwise, use the utilities for your packaging system). The most important file of the HyperSQL system is hsqldb.jar, which resides in the subdirectory lib. Depending on who built your distribution, your file name may have a version label in it, like hsqldb-2.7.4.jar.

	Important
	For the purposes of this chapter, I define HSQLDB_HOME to be the parent directory of the lib directory that contains hsqldb.jar. E.g., if your path to hsqldb.jar is /a/b/hsqldb/lib/hsqldb.jar, then your HSQLDB_HOME is /a/b/hsqldb. Furthermore, unless I state otherwise, all local file paths that I give are relative to the HSQLDB_HOME.

If the description of your distribution says that the hsqldb.jar file will work for your Java version, then you are finished with installation. Otherwise you need to build a new hsqldb.jar file.

If you followed the instructions above and you still don't know what Java version your hsqldb.jar supports, then try reading documentation files like readme.txt, README.TXT, INSTALL.txt etc. (As I said above, our newer distributions always document the Java version for the build, in the file doc/index.html). If that still doesn't help, then you can just try your hsqldb.jar and see if it works, or build your own.

To use the supplied hsqldb.jar, just skip to the next section of this document. Otherwise build a new hsqldb.jar.

See the Building HSQLDB Jars appendix if you want to build anything other than hsqldb.jar with all default settings.

If you installed from an OS-specific package, you may already have a catalog and listener pre-configured. See if your package includes a file named server.properties (make use of your packaging utilities). If you do, then I suggest that you still read this section while you poke around, in order to understand your setup.

We're going to use SqlTool to access the database, so you will need the file sqltool.jar in addition to hsqldb.jar. If sqltool.jar isn't already sitting there beside hsqldb.jar (they both come pre-built), build it exactly as you would build hsqldb.jar, except use ant target sqltool. If your distribution came with a sqltool jar file with a version label, like sqltool-1.2.3.4.jar, that's fine-- use that file whenever I say sqltool.jar below.

Copy the file sample/sqltool.rc to the HSQLDB_OWNER's home directory. Use chmod to make the file readable and writable only to HSQLDB_OWNER.

# $Id: sqltool.rc 6381 2021-11-18 21:45:56Z unsaved $

# This is a sample RC configuration file used by SqlTool, DatabaseManager,
# and any other program that uses the org.hsqldb.lib.RCData class.
# See the documentation for SqlTool for various ways to use this file.
# This is not a Java Properties file. It uses a custom format with stanzas,
# similar to .netrc files.

# If you have the least concerns about security, then secure access to
# your RC file.

# You can run SqlTool right now by copying this file to your home directory
# and running
#    java -jar /path/to/sqltool.jar mem
# This will access the first urlid definition below in order to use a 
# personal Memory-Only database.
# "url" values may, of course, contain JDBC connection properties, delimited
# with semicolons.
# As of revision 3347 of SqlFile, you can also connect to datasources defined
# here from within an SqlTool session/file with the command "\j urlid".

# You can use Java system property values in this file like this:  ${user.home}

# Windows users are advised to use forward slashes instead of back-slashes,
# and to avoid paths containing spaces or other funny characters.  (This
# recommendation applies to any Java app, not just SqlTool).

# It is a runtime error to do a urlid lookup using RCData class and to not
# match any stanza (via urlid pattern) in this file.

# Three features added recently.  All are downward-compatible.
# 1. urlid field values in this file are now comma-separated (with optional
#    whitespace before or after the commas) regular expressions.
# 2. Each individual urlid token value (per previous bullet) is a now a regular
#    expression pattern that urlid lookups are compared to.  N.b. patterns must
#    match the entire lookup string, not just match "within" it.  E.g. pattern
#    of . would match lookup candidate "A" but not "AB".  .+ will always match.
# 3. Though it is still an error to define the same exact urlid value more
#    than once in this file, it is allowed (and useful) to have a url lookup
#    match more than one urlid pattern and stanza.  Assignments are applied
#    sequentially, so you should generally add default settings with more
#    liberal patterns, and override settings later in the file with more
#    specific (or exact) patterns.

# Since service discovery works great in all JREs for many years now, I
# have removed all 'driver' specifications here.  JRE discover will
# automatically resolve the driver class based on the JDBC URL format.
# Most people use default ports, so I have removed port specification from
# examples except for Microsoft's Sql Server driver where you can't depend
# on a default port.
# In all cases, to specify a non-default port, insert colon and port number
# after the hostname or ip address in the JDBC URL, like
# jdbc:hsqldb:hsql://localhost:9977 or
# jdbc:sqlserver://hostname.admc.com:1433;databaseName=dbname

# Amazon Aurora instances are access from JDBC exactly the same as the
# non-Aurora RDS counterpart.

# For using any database engine other than HyperSQL, you must add the
# JDBC jar file and the SqlTool jar to your CLASSPATH then run a command like:
#     java org.hsqldb.util.SqlTool...
# I.e., the "-jar" switch doesn't support modified classpath.
# (See SqlTool manual for how to do same thing using Java modules.)
# To oversimplify for non-developers, the two most common methods to set
# CLASSPATH for an executable tool like SqlTool are to either use the java
# "-cp" switch or set environmental variable CLASSPATH.
# Windows users can use graphical UI or CLI "set".  Unix shell users must
# "export" in addition to assigning.
#
# All JDBC jar files used in these examples are available from Maven
# repositories.  You can also get them from vendor web sites or with product
# bundles (especially database distributions).
# Most databases provide multiple variants.  Most people will want a type 4
# driver supporting your connection mechanism (most commonly TCP/IP service,
# but also database file access and others) and your client JRE version.
# By convention the variants are distinguished in segments of the jar file
# name before the final ".jar" .


# Global default.  .+ matches all lookups:
urlid .+ 
username SA
password

# A personal Memory-Only (non-persistent) database.
# Inherits username and password from default setting above.
urlid mem
url jdbc:hsqldb:mem:memdbid

# A personal, local, persistent database.
# Inherits username and password from default setting above.
urlid personal
url jdbc:hsqldb:file:${user.home}/db/personal;shutdown=true;ifexist=true
transiso TRANSACTION_READ_COMMITTED
# When connecting directly to a file database like this, you should 
# use the shutdown connection property like this to shut down the DB
# properly when you exit the JVM.

# This is for a hsqldb Server running with default settings on your local
# computer (and for which you have not changed the password for "SA").
# Inherits username and password from default setting above.
# Default port 9001
urlid localhost-sa
url jdbc:hsqldb:hsql://localhost



# Template for a urlid for an Oracle database.
# Driver jar files from this century have format like "ojbc*.jar".
# Default port 1521
urlid localhost-sa
# Avoid older drivers because they have quirks.
# You could use the thick driver instead of the thin, but I know of no reason
# why any Java app should.

#urlid cardiff2
# Can identify target database with either SID or global service name.
#url jdbc:oracle:thin:@//centos.admc.com/tstsid.admc
#username blaine
#password asecret


# Template for a TLS-encrypted HSQLDB Server.
# Remember that the hostname in hsqls (and https) JDBC URLs must match the
# CN of the server certificate (the port and instance alias that follows 
# are not part of the certificate at all).
# You only need to set "truststore" if the server cert is not approved by
# your system default truststore (which a commercial certificate probably
# would be).
# Port defaults to 554.

#urlid tls
#url jdbc:hsqldb:hsqls://db.admc.com:9001/lm2
#username BLAINE
#password asecret
#truststore ${user.home}/ca/db/db-trust.store


# Template for a Postgresql database
# Driver jar files are of format like "postgresql-*.jar"
# Port defaults to 5432.
#urlid blainedb
#url jdbc:postgresql://idun.africawork.org/blainedb
#username blaine
#password asecret

# Amazon RedShift (a fork of Postgresql)
# Driver jar files are of format like "redshift-jdbc*.jar"
# Port defaults to 5439.
#urlid redhshift
#url jdbc:redshift://clustername.hex.us-east-1.redshift.amazonaws.com/dev
#username awsuser
#password asecret

# Template for a MySQL database.  MySQL has poor JDBC support.
# The latest driver jar files are of format like "mysql-jdbc*.jar", but not
# long ago they were like "mysql-connector-java*.jar".
# Port defaults to 3306
#urlid mysql-testdb
#url jdbc:mysql://hostname/dbname
#username root
#password asecret
# Alternatively, you can access MySQL using jdbc:mariadb URLs and driver.

# Note that "databases" in SQL Server and Sybase are traditionally used for
# the same purpose as "schemas" with more SQL-compliant databases.

# Template for a Microsoft SQL Server database using Microsoft's Driver
# Seems that some versions default to port 1433 and others to 1434.
# MSDN implies instances are port-specific, so can specify port or instname.
#urlid msprojsvr
# Driver jar files are of format like "mssql-jdbc-*.jar".
# Don't use older MS JDBC drivers (like SQL Server 2000 vintage) because they
# are pitifully incompetent, handling transactions incorrectly.
# I recommend that you do not use Microsoft's nonstandard format that
# includes backslashes.
#url jdbc:sqlserver://hostname;instanceName=instname;databaseName=dbname
# with port:
#url jdbc:sqlserver://hostname:1433;instanceName=instname;databaseName=dbname
#username myuser
#password asecret

# Template for Microsoft SQL Server database using the JTDS Driver
# Looks like this project is no longer maintained, so you may be better off
# using the Microsoft driver above.
# http://jtds.sourceforge.net  Jar file has name like "jtds-1.3.1.jar".
# Port defaults to 1433.
# MSDN implies instances are port-specific, so can specify port or instname.
#urlid nlyte
#username myuser
#password asecret
#url jdbc:jtds:sqlserver://myhost/nlyte;instance=MSSQLSERVER
# Where database is 'nlyte' and instance is 'MSSQLSERVER'.
# N.b. this is diff. from MS tools and JDBC driver where (depending on which
# document you read), instance or database X are specified like  HOSTNAME\X.

# Template for a Sybase database
#urlid sybase
#url jdbc:sybase:Tds:hostname:4100/dbname
#username blaine
#password asecret
# This is for the jConnect driver (requires jconn3.jar).

# Derby / Java DB.
# Please see the Derby JDBC docs, because they have changed the organization
# of their driver jar files in recent years.  Combining that with the different
# database types supported and jar file classpath chaining, and it's not
# feasible to document it adequately here.
# I'll just give one example using network service, which works with 10.15.2.0.
# Put files derbytools*.jar, derbyclient*.jar, derbyshared*.jar into a
# directory and include the path to the derbytools.jar in your classpath.
# Port defaults to 1527.
#url jdbc:derby://server:<port>/databaseName
#username ${user.name}
#password any_noauthbydefault
# If you get the right classes into classpath, local file URLs are like:
#url jdbc:derby:path/to/derby/directory
# You can use \= to commit, since the Derby team decided (why???)
# not to implement the SQL standard statement "commit"!!
# Note that SqlTool can not shut down an embedded Derby database properly,
# since that requires an additional SQL connection just for that purpose.
# However, I've never lost data by shutting it down improperly.
# Other than not supporting this quirk of Derby, SqlTool is miles ahead of
# Derby's ij.

# Maria DB
# With current versions, the MySQL driver does not work to access a Maria
# database (though the inverse works).
# Driver jar files are of format like "mariadb-java-client*.jar"
# Port defaults to 3306
#urlid maria
#url jdbc:mariadb://hostname/db2
#username blaine
#password asecret

We will be using the "localhost-sa" sample urlid definition from the config file. The JDBC URL for this urlid is jdbc:hsqldb:hsql://localhost. That is the URL for the default catalog of a HyperSQL Listener running on the default port of the local host. You can read about URLs to connect to other catalogs with and without listeners in other chapters of this guide.

Run SqlTool.

 java -jar path/to/sqltool.jar localhost-sa

If you get a prompt, then all is well. If security is of any concern to you at all, then you should change the privileged password in the database. Use the command SET PASSWORD command to change SA's password.

 SET PASSWORD 'newpassword';

Set a strong password!

	Note
	If, like most UNIX System Administrators, you often need to make up strong passwords, I highly suggest the great little program pwgen. You can probably get it where you get your other OS packages. The command pwgen -1 is usually all you need.

Note that with SQL-conformant databases like HyperSQL 2.0, user names and passwords are case sensitive. If you don't quote the name, it will be interpreted as upper-case, like any named SQL object. (Only for backwards compatibility, we do make an exception for the special user name SA, but you should always use upper-case "SA" nevertheless).

When you're finished playing, exit with the command \q.

If you changed the SA password, then you need to update the password in the sqltool.rc file accordingly.

You can, of course, also access the database with any JDBC client program. You will need to modify your classpath to include hsqldb.jar as well as your client class(es). You can also use the other HSQLDB client programs, such as org.hsqldb.util.DatabasManagerSwing, a graphical client with a similar purpose to SqlTool.

You can use any normal UNIX account to run the JDBC clients, including SqlTool, as long as the account has read access to the sqltool.jar file and to an sqltool.rc file. See the Utilities Guide about where to put sqltool.rc, how to execute sql files, and other SqlTool features.

Connect to the database as SA (or any other Administrative user) and run CREATE USER to create new accounts for your catalog. HSQLDB accounts are database-catalog-specific, not Listener-specific.

In SQL-compliant databases, all database objects are created in a schema. If you don't specify a schema, then the new object will be created in the default schema. To create a database object, your account (the account that you connected with) must have the role DBA, or your account must have authorization for the target schema (see the CREATE SCHEMA command about this last). When you first create a HyperSQL catalog, it has only one database user-- SA, a DBA account, with an empty string password. You should set a password (as described above). You can create as many additional users as you wish. To make a user a DBA, you can use the "ADMIN" option to the CREATE USER command, command, or GRANT the DBA Role to the account after creating it.

Once an object is created, the object creator and users with the DBA role will have all privileges to work with that object. Other users will have only the rights which the pseudo-user PUBLIC has. To give specific users more permissions, even rights to read objects, you can GRANT permissions for specific objects, grant Roles (which encompass a set of permissions), or grant the DBA Role itself.

Since only people with a database account may do anything at all with the database, it is often useful to permit other database users to view the data in your tables. To optimize performance, reduce contention, and minimize administration, it is often best to grant SELECT to PUBLIC on table-like objects that need to be accessed by multiple database users, with the significant exception of any data which you want to keep secret. (Similarly with EXECUTE priv for routines and USAGE priv for other object types). Note that this is not at all equivalent to giving the world or the Internet read access to your tables-- you are giving read access to people that have been given accounts for the target database catalog.

Do a clean database shutdown when you are finished with the database catalog. You need to connect up as SA or some other Admin user, of course. With SqlTool, you can run

 java -jar path/to/sqltool.jar --sql 'shutdown;' localhost-sa

You don't have to worry about stopping the Listener because it shuts down automatically when all served database catalogs are shut down.

You can, of course, run HSQLDB through inittab on System V UNIXes, but usually an init script is more convenient and manageable. This section explains how to set up and use our UNIX init script. Our init script is only for use by root. (That is not to say that the Listener will run as root-- it usually should not).

The main purpose of the init script is to start up a Listener for the database catalogs specified in your server.properties file; and to gracefully shut down these same catalogs. For each catalog defined by a server.database.X setting in your .properties file, you must define an administrative "urlid" in your sqltool.rc (these are used to access the catalogs for validation and shutdown purposes). Finally, you list the urlid names in your init script config file. If, due to firewall issues, you want to run a WebServer instead of a Server, then make sure you have a healthy WebServer with a webserver.properties set up, adjust your URLs in sqltool.rc, and set TARGET_CLASS in the config file.

By following the commented examples in the config file, you can start up any number of Server and/or WebServer listener instances with or without TLS encryption, and each listener instance can serve any number of HyperSQL catalogs (independent data sets), all with optimal efficiency from a single JVM process. There are instructions in the init script itself about how to run multiple, independently-configured JVM processes. Most UNIX installations, however, will run a single JVM with a single Listener instance which serves multiple catalogs, for easier management and more efficient resource usage.

After you have the init script set up, root can use it anytime to start or stop HSQLDB. (I.e., not just at system bootup or shutdown).

 sh -x path/to/hsqldb start

 sh -x path/to/hsqldb start > /tmp/hstart.log 2>&1

This section is for users who are using our UNIX init script, and who are upgrading their HyperSQL installation.

Most users will not have customized the init script itself, and your customizations will all be encapsulated in the init script configuration file. These users should just overwrite their init script with a new one from the HyperSQL installation, and manually merge config file settings. First, just copy the file /sample/hsqldb.init over top of of your init script (wherever it runs from). Then update your old config file according to the instructions in the new config file template at sample/hsqldb.conf. You will have to change very few settings. If you are upgrading from a pre-2.0 installation to a post-2.0 installation, you will need to (1) add the setting URLIDS, as described above and in the inline comments, and (2) replace variable HSQLDB_JAR_PATH with SQLTOOL_JAR_PATH which (if you haven't guessed) should be set to the path to your sqltool.jar file.

Users who customized their init script will need to merge their customizations into the new init script.