|
 |
 |
Z39.50 Metadata Server
zserver.pl - Z39.50 server, with GEO compliance, for Solaris and Windows.
zserver.pl
-
Starts up Zserver with the default configuration. Zserver will read in
the map file fgdcMap.xml and monitor port 9999 for connection
requests.
zserver.pl -m/test/user/TestMap.xml -p9000
-
Starts up Zserver with the map file "/test/user/TestMap.xml"
and monitors port 9000 for connection requests.
-
On Windows, the syntax for specifying the same map file is
"\test\user\TestMap.xml".
Zserver uses Z39.50 to make databases available as GEO
profile compliant metadata.
Z39.50 (http://lcweb.loc.gov/z3950/agency/markup/toc.html) is a
communication protocol specifying how server and client software
exchange messages to retrieve information. The server software in this
case is Zserver. Most client software were created for bibliographic
databases. However, the client at
http://cgkn.net/cur/datacat/index.php focuses on geospatial
metadata.
GEO profile
(http://www.blueangeltech.com/standards/GeoProfile/geo22.htm) is a
framework for organizing information known as geospatial metadata.
Using this framework, end-users can indicate the metadata of interest
without knowing how the underlying databases organize the data.
- Supports GEO Profile, version 2.2.
Zserver's support of GEO is based on the formally expressed (XML DTD)
schema found in GEO Profile version 2.2
(http://www.blueangeltech.com/standards/GeoProfile/geo22.htm).
If you do not get the expected query result from Zserver, check the
client application that you use for connecting to Zserver. Make sure
that the application is sending out queries conforming to GEO's
requirement on attribute combination:
http://www.blueangeltech.com/standards/GeoProfile/annex_b.htm#Attribute%20Combination%20Guidelines
If the client is ZNavigator and you use its query builder to build a
query, do not allow any of the term qualifiers (attributes) to use
default value. ZNavigator does not always generate an appropriate
query involving one or more defaults.
- Supports various database systems.
Zserver can access the following database systems:
Microsoft SQL Server
On Solaris, Zserver accesses SQL Server via FreeTDS. Thus, FreeTDS
must be installed.
On Windows, Zserver accesses SQL Server via the ODBC driver that comes
with Windows.
Oracle
To have Zserver accesses an Oracle database, you must install Oracle
software on the machine running Zserver.
PostgreSQL
You can download PostgreSQL from http://www.postgresql.org.
- Supports Latin-1 encoding.
Zserver supports database content stored as Latin-1 encoding. This
includes French characters.
- Allows attributes mapped to various target types.
Zserver allows the GEO profile "use" attributes mapped to:
database table columns,
SQL scalar expressions, and
constant expressions containing text, number, or date.
- Presents output in a consistent order.
GEO profile is based on the FGDC standard
(http://geology.usgs.gov/tools/metadata/standard) which includes a
structure for presenting metadata. Zserver observes the same FGDC
structure when presenting metadata. Thus, providing a consistent way
of presenting search result regardless of which database the data
comes from.
The same structure also applies to different output formats, i.e. XML,
HTML, SUTRS, and GRS-1.
- Includes links to FGDC explanations in HTML output.
When asked to present metadata as HTML output, Zserver includes links
to the FGDC explanation
(http://geology.usgs.gov/tools/metadata/standard) of the individual
"use" attributes.
- Fills in missing components for incomplete queries.
The Z39.50 clients ZNavigator and Compusult sometimes send in
incomplete search queries. Zserver has special handling to fill in the
missing parts.
On the command line invoking Zserver, you can supply parameters
separated by spaces.
-m<map_file_name>
For example, "zserver.pl -m/test/user/TestMap.xml" specifies the map
file "/test/user/TestMap.xml".
On Windows, the syntax for specifying the same map file is "\test\user\TestMap.xml".
-p<port_number>
For example, "zserver.pl -p9000" asks Zserver to monitor port 9000
for connection requests.
Note that when multiple parameters are specified, the port parameter
must be the last.
-pf<pre-fetch_count>
For example, "zserver.pl -pf200" asks Zserver to pre-fetch 200
matching records after each search. By pre-fetching records, Zserver
does not have to issue a separate database request for individual records.
The overall response is therefore improved. The price for such
improvement is the memory used for storing the pre-fetched records.
When Zserver receives a request for records beyond the pre-fetched
ones, Zserver will ask the database server for those records and, if
applicable, the preceding ones. For example, with 200 records
pre-fetched and there is a request for the 284th record, Zserver will issue
a database request for record 201 to 284. After that, any requests for
record 1 to 284 will cause Zserver to look into the 284 pre-fetched
records.
The default is 100.
-trace
For example, "zserver.pl -trace" asks Zserver to output
overall trace messages.
-tracepqf
For example, "zserver.pl -tracepqf" asks Zserver to output
trace messages on the parsing of search query. PQF stands for Prefix
Query Format. It is the search query's format.
Zserver includes the following limitations:
Here is a 3-step process of getting Zserver up and running:
Download and install http://cgkn.net/cur/zserver/zserver-0.82.tar.gz.
-
-
For Solaris, see Installing on Solaris.
-
For Windows, see Installing on Windows.
Edit the map file to tell Zserver how the individual GEO
profile "use" attributes should be mapped to database fields.
See MAP FILE FORMAT.
Start Zserver (e.g. perl zserver.pl). For parameter description, see
COMMAND LINE PARAMETERS.
On a Solaris machine with Perl and C installed, you need to compile
and install the following:
-
Note:
-
On Windows (Win32), you can go through the same compile-and-install
process as in Solaris. You don't need DBD::Oracle and DBD::Pg.
However, you need Microsoft Visual C++. Without that, you have to
install pre-compiled binaries.
-
This section is about installing pre-compiled binaries. The reference
in parenthesis after a package name refers to either a binary
distribution's URL or an ActivePerl "ppm install" command.
-
In the case of an URL, download and install the binary. Note the
version number. It is important to get the specified version.
-
In the case of a PPM (Perl Package Manager) command, run it inside a
Command Prompt window to install the package.
To run Zserver on Windows, you need the following:
XML::LibXML::Common (ppm install http://theoryx5.uwinnipeg.ca/ppmpackages/XML-LibXML-Common.ppd)
In response to the "Fetch libxml2.dll [yes]" prompt, press enter to
download and install the DLL.
In response to the "Where should libxml2.dll be placed? [Perl/bin]"
question, enter a directory that is in the system variable "path".
XML::SAX (ppm install http://theoryx5.uwinnipeg.ca/ppms/XML-SAX.ppd)
-
After installation, check the SAX directory to look for a file called
ParserDetails.ini. If there is no such file, create a blank file and
name it as such.
-
-
If ActivePerl was installed on c:\perl, the SAX directory is typically
c:\perl\site\lib\XML\SAX.
XML::LibXML (ppm install http://theoryx5.uwinnipeg.ca/ppms/XML-LibXML.ppd)
XML::LibXSLT (ppm install http://theoryx5.uwinnipeg.ca/ppms/XML-LibXSLT.ppd)
In response to the "Fetch libxslt_win32.dll [yes]" prompt,
press enter to download and install the DLL.
In response to the "Where should libxslt_win32.dll be placed?
[Perl/bin]" question, enter a directory that is in the system
variable "path".
In response to the "Fetch libexslt_win32.dll [yes]" prompt,
press enter to download and install the DLL.
In response to the "Where should libexslt_win32.dll be placed?
[Perl/bin]" question, enter a directory that is in the system
variable "path".
YAZ 2.0.20 (http://ftp.indexdata.dk/pub/yaz/win32/yaz_2.0.20.exe)
-
After installing YAZ, remember to add the YAZ installation directory
to the system variable "path". It is required by
SimpleServer.
Net::Z3950::SimpleServer (ppm install http://cgkn.net/cur/zserver/Net-Z3950-SimpleServer.ppd)
-
If for whatever reasons you need to re-build the SimpleServer PPM
package, see Building a SimpleServer PPM package.
-
A SimpleServer PPM package allows SimpleServer to be installed on a
Windows machine that was not set up for compilation.
-
To build the package, you need a machine with ActivePerl 5.8+, YAZ 2+,
and Microsoft Visual C++ installed. After downloading the SimpleServer
0.08+ source, go to the source directory and do the following:
#my $libs = `yaz-config --libs threads` || die "ERROR: Unable to call script: yaz-config";
Specify in LIBS, the location of yaz.lib. For example,
'LIBS' => 'c:\bin\yaz\lib\yaz.lib',
Specify in INC, the YAZ include path. For example,
'INC' => '-Ic:\bin\yaz\include',
Add, right after the 'INC' line, the following lines:
'AUTHOR' => 'Index Data(www.indexdata.dk)',
'ABSTRACT' => 'Z39.50(ActivePerl 5.8.3.809, Yaz 2.0.20)',
Note that the abstract includes the version numbers of ActivePerl and Yaz
that the SimpleServer is compiled with. Update those numbers to
reflect the versions you are using.
Compile SimpleServer and create a ppd
Start up a Visual Studio .NET 2003 Command Prompt window (Start>
Programs> Microsoft Visual Studio .NET 2003> Visual Studio .NET Tools>
Visual Studio .NET 2003 Command Prompt) and execute the following
commands:
perl MakeFile.PL
nmake
nmake ppd
Copy run-time library
"nmake" in the above created a set of binaries inside the directory blib.
You need to copy the C run-time library %SystemRoot%\system32\msvcr71.dll to
blib\arch\auto\Net\Z3950\SimpleServer.
Create an archive
Use tar and gzip to create an archive of the blib binaries. For
example,
-
tar cvf Net-Z3950-SimpleServer.tar blib
-
gzip --best Net-Z3950-SimpleServer.tar
Specify archive in ppd
Open Net-Z3950-SimpleServer.ppd and locate the line:
-
<CODEBASE HREF="" />
Change it to specify the archive. For example,
-
<CODEBASE HREF="Net-Z3950-SimpleServer.tar.gz" />
Now the package is ready. It is consisted of the ppd file and the .gz
archive. Both of them have to be in the same directory. If that
directory is on the web, people can install SimpleServer by opening a
Command Prompt window and issuing a command such as:
-
ppm install http://cgkn.net/cur/zserver/Net-Z3950-SimpleServer.ppd
To indicate how the GEO profile "use" attributes should be
mapped to a database, you need a map file. The map file is in XML
format and has a <schema> element.
Zserver comes with the default map file fgdcMap.xml. It resides in the
same directory where you find Zserver. A recommended way of creating
your own map file is to modify fgdcMap.xml. In it, you can remove any
of the <database> elements, except the one with
dbNameFromClient="ignore". That <database> element must
remain in the map file.
To specify a new database mapping, add a <database> element to
<schema>, then add <field> elements to <database>.
Note that, as specified in the XML standard, the element and attribute
names are case-sensitive. To have the character "<" appears
inside an attribute value, you have to use "<".
-
The <database> element's attributes provide database connection
information. For example:
-
<database
dbSystem="postgreSql"
dbNameFromClient="MIRAGE"
dbNameToServer="gscmaps"
host="region11.agg.nrcan.gc.ca"
port="1234"
user="userAbc"
password="userAbcPassword"
>
-
A <database> element has the following attributes:
- dbNameFromClient
The name that a database is known to the client (software or user)
connecting to Zserver.
Note that, as stated in the Z39.50 specification, section 3.2.2.1.2
(http://lcweb.loc.gov/z3950/agency/markup/04.html), the name is
case-insensitive. Thus, "MyDatabase" is the same as "mydatabase".
- dbNameToServer
The name that a database is known to the managing database server.
This name typically appears in a database connection string.
- dbSystem
A case-sensitive code indicating the type of database server
managing the database. The valid codes are :
- msSql
Microsoft SQL server.
- odbcSocketServer
odbcSocketServer refers to a type of database connection in which
there is an ODBC Socket Server running on a Windows PC acting as a
bridge between a Microsoft SQL server and Zserver.
ODBC Socket Server is the slowest form of database access. The socket
server does not support partial fetching of matching records. It always
returns all the matching records. Depending on how fast the socket server
can respond, which is affected by the amount of matching data from a
query, a timeout can occur at the TCP/IP level.
ODBC Socket Server is available from
http://odbcsock.sourceforge.net. After installing it, remember to
create a DSN (Data Source Name) with the same name as that in
dbNameToServer.
Zserver's support of ODBC Socket Server is experimental.
- oracle
To access an Oracle database from a Zserver running on Windows, you
need to define a system DSN with the same name as that in
dbNameToServer.
- postgreSql
PostgreSQL server.
- host
The host name or IP address of the machine on which the database
server is running.
For an Oracle database, this attribute is ignored by the Zserver
running on a Windows machine.
- port
The port number which the database server will monitor for database
access.
For a Microsoft SQL server database, this attribute is ignored on
Windows. It is optional on Solaris.
For an ODBC Socket Server database, this attribute is ignored.
For an Oracle database, this attribute is ignored on Windows. it is
required on Solaris.
- user
The user name for logging in to the database server.
- password
The password for logging in to the database server.
-
The <field> element specifies how an "use" attribute as
defined in the GEO profile is mapped to a database. For examples:
<field
usePath="metadata->dataqual->lineage->srcinfo->srccite->citeinfo->title"
table="tableName"
column="columnName"
/>
<field
usePath="metadata->idinfo->citation->citeinfo->pubdate"
sql="to_char(metamaps.publication_date,'YYYYMMDD')"
/>
<field
usePath="metadata->idinfo->ptcontac->cntinfo->cntaddr->country"
const="Canada"
/>
Note that a database mapping must have at least two <field> elements. One of
them must be mapped to a table column.
A <field> element recognizes the following attributes (some are
applicable only to certain mapping):
- column
Name of the database column that an "use" attribute is mapped to.
A <field> element with "column" attribute cannot have "sql" or "const"
attributes.
- const
A constant text string that an "use" attribute is mapped to.
The constant must be enclosed within double quotes (").
Please note the following special handling of characters found in a
constant:
All ampersand (&) characters within the constant must be changed
to &.
To have a constant including executable HTML, replace the less than sign (<) by <. For example,
-
const="HTML code: <a href='http://gdcinfo.agg.nrcan.gc.ca/gdc/index_f.html'>Centre de données géoscientifiques</a>"
To have a constant including HTML intended only for display and
not for execution, replace < by &amp;lt;. For example,
-
const="HTML display: &amp;lt;a href='http://gdcinfo.agg.nrcan.gc.ca/gdc/index_f.html'>Centre de données géoscientifiques&amp;lt;/a>"
A <field> element with "const" attribute cannot have "table", "column",
or "sql" attributes.
- sql
A SQL scalar expression that an "use" attribute is mapped to.
The expression must be enclosed within double quotes (").
A scalar expression is an expression returning a single value. For example,
-
"to_char(metamaps.ending_date,'YYYYMMDD')"
The SQL expression can have subqueries, i.e. SELECT queries returning
a single value. Therefore, you can link tables together. For example,
let's say the "use" attribute, Series Name (sername), is stored
in a table column. But the name stored there is a short name. There is
another table translating short names to long names. To map Series
Name to a long name, you can do the following:
-
<field
usePath="metadata->idinfo->citation->citeinfo->title"
table="main_table"
column="title"
/>
<field
usePath="metadata->idinfo->citation->citeinfo->serinfo->sername"
sql="SELECT long_name FROM name_table WHERE name_table.short_name = main_table.short_name"
/>
Note that it is OK for the subquery to reference main_table even
though main_table is not in the subquery's FROM clause. That is
because main_table is referenced from somewhere else, i.e. the Title
<field> element. In other words, the Title <field> element pulls in
main_table and makes it available to all the subqueries within the
same database mapping.
When formulating a subquery, make sure that it returns a single value,
not a column.
A <field> element with "sql" attribute cannot have
"table", "column", or "const" attributes.
- table
Name of a database table in which a column is mapped to an
"use" attribute.
A <field> element with "table" attribute cannot have "sql" or "const"
attributes.
- usePath
The path of an "use" attribute.
To prepare a map file, you need to understand paths. This section
provides an introduction.
- About paths
-
GEO specifies the relationships among "use" attributes. For
instance, GEO states that:
-
"Metadata" includes "Identification Information".
-
"Identification Information" includes "Point of Contact".
-
"Point of Contact" includes "Contact Information".
-
"Contact Information" includes "Contact Address".
-
"Contact Address" includes "Country".
Expressing the above series of relationships as a path using the
"use" attribute short names, we have:
-
metadata->idinfo->ptcontac->cntinfo->cntaddr->country
Furthermore, GEO allows some attributes to appear in multiple places.
For instance, "Contact Information" can appear in four places. Since
"Country" is a part of "Contact Information", "Country" has four
possible paths:
-
metadata->dataqual->lineage->procstep->proccont->cntinfo->cntaddr->country
-
metadata->distinfo->distrib->cntinfo->cntaddr->country
-
metadata->idinfo->ptcontac->cntinfo->cntaddr->country
-
metadata->metainfo->metc->cntinfo->cntaddr->country
When you specify an "use" attribute, you need to indicate which
of the possible paths you have in mind. You do that by including the path in the usePath attribute. For example,
-
usePath="metadata->idinfo->ptcontac->cntinfo->cntaddr->country"
- Ramification of mapping an attribute more than once
-
As shown in the above illustration, an "use" attribute may
appear in multiple places - each with an unique path. When a database
map has entries referring to the same "use" attribute, each
with a different path, it affects how Zserver selects matching records.
-
For example, let's say a database map includes the following <field>
elements:
-
<field
usePath="metadata->dataqual->lineage->srcinfo->srccite->citeinfo->title"
table="Landscapes"
column="srcTitle"
/>
-
<field
usePath="metadata->idinfo->citation->citeinfo->title"
table="Landscapes"
column="idTitle"
/>
Both <field> elements address Title. One refers to the Title in Source
Information (srcinfo); the other refers to the Title in Identification
Information (idinfo).
When Zserver processes a query looking for records with Title
containing "abc", a record will be selected when its Title in
Identification Information or Source Information contains
"abc".
- Discovering paths
-
Here is a way to discover all the possible paths for an "use"
attribute:
Find the "use" attribute's numeric value by
going to the "<useDefList>" section
in the map file and search for a "titleKey" with your
"use" attribute's title. When you find such line, the
"value" entry on that line is the "use" attribute's
numeric value.
For example, to get the numeric value of the use
attribute, Contact Person, search for "Contact Person"
(enclosed within double quotes). You will see that
Contact Person has the numeric value of 2023.
With the numeric value in hand, search further down the
map file for an "<usePath" line with the "valueKey"
equals to the "use" attribute's numeric value.
Once you've located that line, you will see a group of
"<usePath" lines with the same "valueKey", if the
"use" attribute can appear in multiple places. The
"pathKey" entries in those lines are the available paths that
you can use in specifying a field for the "use" attribute.
To carry on with the Contact Person example, when you
search for an "<usePath" line with "2023", you will see
a number of lines with such value. You can use any of
those pathKey values to specify a path in your
database mapping.
A path is consisted of short names. The meaning of
those short names may not be very intuitive. To expand
the short names, search further down the map file for
the short name path.
Continuing with the Contact Person example, when you search for:
-
"metadata->dataqual->lineage->procstep->proccont->cntinfo->cntorgp->cntper"
you will see that it means:
-
'Metadata'->'Data Quality Information'->'Lineage'->'Process Step'->'Process Contact'->'Contact Information'->'Contact Organization Primary'->'Contact Person'
You CANNOT use a path with expanded names to
specify a field in the database mapping. You must
use a path with short names.
Jean-François Doyon (Canadian Centre for Remote Sensing, Natural
Resources Canada) started Zserver.
Zoltan Bardossy (Natural Resources Canada) further enhanced it.
Boniface Lau (under contract to James Rupert at Natural Resources
Canada) rewrote Zserver to add numerous functionality and provided
user documentation. The latest release is 0.82.
Copyright (c) 2004 Natural Resources Canada
This program is free software; you can redistribute it and/or modify
it under the same terms as Perl itself.
|
|
|
