Glossary

This is a glossary of Unidata LDM-specific acronyms, terms, and phrases. If you don't find what you're looking for here, try the more general On-line Dictionary of Computing, or the Internet Users' Glossary (RFC 1392) from which we have adapted some of these definitions.

A

actions
actions
In the context of the Unidata LDM, one of the possible fates of a data product when it is read out of the product queue by the pqact program. If the feed type and product ID of a data product match a list maintained by pqact, the associated action is invoked on the product. An action may have arguments that include substitution of matched subparts of the product ID. The currently supported actions include FILE, STDIOFILE, PIPE, EXEC, DBFILE, and NOOP.
AFOS
An automated system (Automation of Field Operations and Services) used by the National Weather Service to standardize and expedite meteorological services.
Alden
Alden Electronics, Inc., a company that provides meteorological services.
ANY
In the context of the Unidata LDM, the psuedo-feed type that designates any data feed.
arrival time
In the context of the Unidata LDM, old and now deprecated name for injection time.
ASOS
Automated Surface Observing System
automatic fail-over
The ability of a server to automatically switch to use an alternate data source if the primary source fails.
AWIPS
Advanced Weather Interactive Processing System, an NWS program, also known as the "modernization program."

B

BLKDATA
The remote procedure call made by a data source or upstream node to deliver another piece of a large product.
broadcast
A special type of multicast packet that all nodes on the network are always willing to receive. Also in the Unidata LDM context, refers to the satellite broadcast of FOS and McIDAS data by Alden.
BUFR
Binary Universal Form for the Representation of meteorological data, a WMO standard for a binary code for the exchange and storage of data. The format is documented in The WMO Manual on Codes; WMO Publication No. 306; Volume I, Part B; 1988 Edition, plus Supplements 1, 2, and 3.
bulletins
bulletin
A sequence of consecutive bytes within a data stream that comprises a single unit that can be converted into a data product by supplying the necessary identifying information.

C

clients
client
A computer system or process that requests a service of another computer system or process (a server).
client-server model
A model for software design in which programs use and provide distributed services. A task is split between one or more server tasks that accept requests, according to some protocol, for information or action from (distributed) client tasks. This model allows clients and servers to be placed independently on nodes in a network. Examples are the name-server/name-resolver relationship in DNS, the file-server/file-client relationship in NFS and the screen server/client application split in the X Window System.
COMINGSOON
The remote procedure call made by a data source or upstream node to announce the intent to deliver a large product in pieces.
connection-oriented
The data communication method in which communication proceeds through three well-defined phases: connection establishment, data transfer, connection release. TCP is a connection-oriented protocol.
connectionless
The data communication method in which communication occurs between hosts with no previous setup. Packets between two hosts may take different routes, as each is independent of the other. UDP is a connectionless protocol.

D

daemon
A program that is not invoked explicitly, but lies dormant waiting for some condition(s) to occur. The idea is that the perpetrator of the condition need not be aware that a daemon is lurking (though often a program will commit an action only because it knows that it will implicitly invoke a daemon). Daemons are usually spawned automatically by the system, and may either live forever or be regenerated at intervals. Unix systems run many daemons, chiefly to handle requests for services from other hosts on a network. Most of these are now started as required by a single real daemon, inetd, rather than running continuously. Examples are cron (local timed command execution), rshd (remote command execution), rlogind and telnetd (remote login), ftpd, nfsd (file transfer), lpd (printing).
data feeds
data feed
In the context of the Unidata LDM, a sequence of data products, all of the same feed type, injected into the IDD from a single data source.
data products
data product
In the context of the Unidata LDM, a unit of data with an associated feed type, product ID, injection time, and origin, and sequence number.
data relays
data relay
In the context of the Unidata LDM, a network node that relays data products from one or more upstream source nodes to one or more downstream sink nodes. A relay node may provide data fan-out by receiving data products from a single upstream source that it then transmits to multiple downstream sinks.
data sinks
data sink
In the context of the Unidata LDM, a remote consumer of data products that must be transmitted as they arrive.
data sources
data source
In the context of the Unidata LDM, a remote provider of data products.
data streams
data stream
In the context of the Unidata LDM, a sequence of bulletins all of the same feed type broadcast from a single data source. Once the bulletins are converted into data products, the result is called a data feed.
DCE
Distributed Computing Environment, an architecture of programming interfaces, conventions, and server functionalities (e.g. naming, distributed file system, remote procedure call) for distributing applications transparently across networks of heterogeneous computers promoted and controlled by the Open Software Foundation (OSF). The Unidata LDM does not use DCE RPCs.
DDS
Domestic Data Service, a data feed that is part of the NWS Family of Services. DDS includes data products of many kinds for the U.S., Canada, Mexico.
decoders
decoder
In the context of the Unidata LDM. a program that converts transmitted data products into a different form more suitable for use by applications. Decoders often combine multiple products into a single decoded file.
DDPLUS
DD+
Domestic Data Plus, a data stream provided by Alden merging the DDS and PPS data streams into one.
DIFAX
Digital facsimile maps, a data stream of digitized maps provided by NWS.
Disabling an ldmd.conf Entry
An entry in the file ldmd.conf is disabled if it starts with the comment-character "#".
downstream
In the context of the Unidata IDD, the direction of data flow from data sources to data sinks.
downstream sink
see data sink.
domain name servers
domain name server
A server that provides translation of hostnames to IP addresses, as part of the Internet Domain Name System.
DNS
Domain Name System
A general-purpose distributed, replicated, data query service chiefly used on Internet for translating hostnames into Internet addresses. DNS can be configured to use a sequence of name servers, based on the domains in the name being looked for, until a match is found. DNS is a distributed database. It can be queried interactively using the command nslookup.
dot address
Dot address refers to the common notation for IP addresses of the form A.B.C.D; where each letter represents, in decimal, one byte of a four byte IP address. See also: IP address.

E

ECMWF
European Center for Medium-range Weather Forecasting, a source for some model output grids in the HDS data stream.
Enabling an ldmd.conf Entry
An entry in the file ldmd.conf is enabled if it doesn't start with the comment-character "#".
event-driven
A kind of program with a main loop that waits for events to occur. Each event has an associated handler that is passed the details of the event as soon as it is detected. This is in contrast to a polling program that frequently wakes up and checks for whether an event has happened yet, or a scheduled system that periodically wakes up and handles pending events.
EXP
In the context of the Unidata LDM, the feed type for experimental data. This feed type is unassigned, so it can be used for any purpose for data of low interest to the majority of sites. This is much easier than adding a new feed type, which is technically a change to the protocol and requires all sites to recompile.

F

feed sets
feed set
In the context of the Unidata LDM, a combination of feed types that results from applying set union and set difference operations to the basic feed types. For example, DDS|PPS designates the set of data products from either the DDS or PPS data feed.
feed types
feed type
In the context of the Unidata LDM, a category of data products sharing a name space of product IDs and injected from a single data source. The feed type is used for coarse selection of data products; finer selection is provided by patterns matching a subset of product IDs. Primitive feed types supported by the Unidata LDM include: DDS, IDS, PPS, HDS, NLDN, WSI, EXP, MCIDAS, ANY.
FEEDME
The remote procedure call made by a data sink or downstream node to sign on and announce what feed types and product IDs it wants to be fed.
FOS
Family of Services
A collection of data streams from the National Weather Service, injected into the IDD by the Alden Corporation under a contract with Unidata. This currently includes DDS, PPS, IDS, and HDS data streams.
FSL
Forecast Systems Laboratory, a NOAA program that is also a data source of some experimental IDD data, such as MAPS FSL2 and ISPAN.
FSL2
60 minute Profiler Data, current source is FSL distributed via IDD.
FTP
File transfer protocol, a method of transferring files over TCP/IP networks. Anonymous FTP permits users to obtain files from a remote system by logging in as anonymous.

G

Gateway LDM
The only LDM of an institution or one that serves as a top-level source of data for the institution's internal LDM network.
gdbm
GNU database manager, a freely-available library that supports a simple key-contents database. The pqact program of the Unidata LDM can store data products in gdbm databases using the DBPUT action.
GDS
GAI
Global Atmospherics, Inc., provider of NLDN data.
GEMPAK
General Meteorological Package, an analysis and display package developed at NASA Goddard Space Flight Center. More information is available here.
Gopher
Software developed by the University of Minnesota for easily providing and accessing public information on computers linked by a network. There are two kinds of Gophers: servers, which provide a method of making files (graphics, video, or text) publicly available, and clients, which are the tools for accessing information on a Gopher server.
GRIB
GRidded Binary, a bit-oriented format approved by the WMO that has compaction features for efficiently transmitting and storing large volumes of gridded data. A description of the format can be found here.
gribtonc
An LDM in-line decoder, supported by the UPC, that decodes GRIB products on the HDS data stream from NMC and ECMWF and writes them into netCDF files.

H

HDS
The feed type for the High-Resolution Data Service, a data stream that is part of the NWS Family of Services. HDS includes model output data from models run at the NMC and ECMWF.
HEREIS
The remote procedure call made by a data source or upstream node to deliver a small product.
HIYA
The remote procedure call made by a data source or upstream node to sign on and announce what feed types and product IDs it is willing to provide.
HRS
See HDS
HTML
HyperText Markup Language, the collection of styles, indicated by markup tags, that define the structure of a World Wide Web document. More information is available here.
HTTP
HyperText Transfer Protocol, the protocol used for the World-Wibe Web (WWW). Extensive information on the protocol is available here.

I

IDD
Internet Data Distribution, a system for disseminating real-time earth observations via the Internet. Unlike other systems, which are based on data centers where the information can be accessed, the Unidata IDD is designed so a university can request that certain data sets be delivered to computers at their site as soon as they are available from the observing system. Moreover, any site with access to specialized observations of interest to others can inject the dataset into the IDD for immediate delivery to interested sites.
idd_monitor
The idd_monitor is an X-based monitoring tool, using the MOTIF toolkit, that allows a user to monitor up to 14 IDD hosts. It provides audible and visual cues when one or more hosts is either not responding, or is suffering from excessive packet loss.
IDS
The feed type for the International Data Service, a data stream that is part of the NWS Family of Services. IDS includes international data products analogous to the domestic products on DDS.
ingester
In the context of the Unidata LDM, a program such as pqing that reads bytes from a data stream and creates data products.
injection time
In the context of the Unidata LDM, time when a data product is first injected into the IDD, an attribute of the data product that is preserved as it gets distributed to other IDD nodes.
IP
Internet Protocol, the network layer for the TCP/IP protocol suite widely used on Ethernet networks. IP is a connectionless, best-effort packet switching protocol. It provides packet routing, fragmentation and re-assembly through the data link layer.
IP address
The 32-bit address defined by the Internet Protocol. It is usually represented in dotted decimal notation.
ISPAN
Information Stream Project for AWIPS NOAAport, an experimental data stream available from FSL.

J

K

L

LAN
Local area network, a data network intended to serve an area of only a few square kilometers or less. Since such networks relatively small they can usually be directly controlled by the users and operate at relatively high speeds (up to 100Mb/s) over inexpensive wiring.
large product
In the context of Unidata's LDM, a data product that is large enough to require multiple remote procedure calls to send it. Currently, this means the product is larger than about 16 Kb.
latency
In the context of the Unidata LDM, the delay in delivery of a data product since it was first injected from a data source site.
LDMs
LDM
Local Data Manager, the Unidata software system for event-driven data distribution. The software allows Unidata sites to select, capture, process, and distribute data products. The LDM is implemented as a set of network client and server programs and their shared protocols. More information is available here.
ldmprods
A perl script that monitors incoming data in realtime. Can be used to make sure that data is continuing to flow. It will either sound an audible alarm, or send email, depending on how it is run, if data flow ceases for any of the data streams it monitors.
LDM servers
LDM server
The LDM program ldmd responsible for servicing RPC requests, managing the reception, disposition, and injection of data to and from the IDD.
LDM-McIDAS
a set of LDM-compatible programs used for decoding McIDAS products contained in the Unidata-Wisconsin (McIDAS) data stream that is available to licensed sites. More information is available here.
leaf node
In the context of the IDD, an LDM system that receives data products but does not relay data products to other IDD nodes. If the IDD distribution is represented as a directed graph, the leaf nodes have no outgoing edges. Technically, whether an IDD node is a data source, data relay, or leaf node is relative to particular data streams.

M

MAPS
Mesoscale Analysis and Prediction System (MAPS) MAPS is a data assimilation and mesoscale modeling system developed by NOAA's Forecast Systems Laboratory; it incorporates asynoptic data such as wind profiler and ACARS data into a mesoscale model with horizontal resolution of 30 kilometers.
MCIDAS
The feed type used for the Unidata/Wisconsin data stream from SSEC.
McIDAS
Unidata McIDAS is a suite of applications, developed at SSEC, for analyzing and displaying meteorological data for research and education. The software can be used with conventional observational, satellite, and grid-point data. Unidata distributes two versions of this software: McIDAS-X for UNIX platforms and McIDAS-OS2 for computers running OS/2.
McLDM
OS/2 software to ingest the Unidata/Wisconsin data stream directly from an upstream LDM server.
MD5 checksum
a kind of checksum that is very likely to be unique for a sequence of bytes. In the context of the Unidata LDM, it is an attribute of a data product that is used for detecting duplicate products.
MRF
Medium-Range Forecast, a forecasting model run by the NMC.
multicast
A packet with a special destination address that multiple nodes on the network may be willing to receive.

N

namespace
A commonly distributed set of names in which all names are unique.
netCDF
an interface for scientific data access and a library that provides an implementation of the interface. The netCDF library also defines a machine-independent format for representing scientific data. Together, the interface, library, and format support the creation, access, and sharing of scientific data. The netCDF software was developed by Unidata. Some inline decoders that work with the LDM use netCDF as the form of their output. More information is available from the NetCDF Homepage.
netcheck
A perl script that is meant to be run out of cron. It monitors the network connection between IDD hosts. Can be set up to provide email notification when the network connection is bad, based on packet loss.
Netscape
A widely-used WWW browser client for various operating systems, used for presenting this Workshop. More information is available here.
NEXRAD
Next Generation Weather Radars, a NOAA radar system of WSR-88D radars and associated processing equipment. NEXRAD data is made available at discounted prices to the IDD through a contract with WSI.
NFS
Network File System, a protocol developed by Sun Microsystems, and defined in RFC 1094, which allows a computer to access files over a network as if they were on its local disks. This protocol has been incorporated in products by more than two hundred companies, and is now a de facto standard. NFS is implemented using a connectionless protocol(UDP) in order to make it stateless.
NIDS
NEXRAD Information Dissemination System.
NLDN
National Lightning Detection Network, a data feed provided by SUNY Albany through a cooperative agreement with Global Atmospherics, Inc.
NMC
National Meteorological Center, the source of most of the model output data that appears in GRIB form on the HDS data stream. The NMC was renamed the National Centers for Environmental Prediction after 1 October 1994.
NOTIFYME
The remote procedure call made by a data sink or downstream node to sign on and announce what feed types and product IDs it wants to be notified about when they arrive.
NOAAport
Data stream similar to the NWS FOS data stream but enhanced with other data products(Channel 3). It also contains GOES (East|West) images and other experimental products on channels 1,2, and 4.
NPS
Numerical Products Service, a now-obsolete name for the HDS data stream.
NTP
Network Time Protocol, an Internet Standard (12).
NULLPROC
The trivial remote procedure call supported by RPC servers so that clients can test connectivity, timing, and flush RPC buffers.
NWS
National Weather Service.

O

ONC
Open Network Computing, an architecture of programming interfaces, conventions, and server functionalities (e.g. naming, distributed file system, remote procedure call) for distributing applications transparently across networks of heterogeneous computers promoted by Sun. The ONC architecture includes some widely adopted ad hoc standards such as NFS and ONC RPCs. The latter are the RPCs used by the Unidata LDM.
origin
In the context of the Unidata LDM, the host that first injected a data product into the IDD, an attribute of the data product that is preserved as it gets distributed to other IDD nodes.

P

patterns
pattern
In the context of the Unidata LDM, a regular expression that specifies the set of product IDs matching the pattern. For example, the pattern `^A' specifies the set of all product IDs beginning with the character `A'.
pattern-action file
In the context of the Unidata LDM, the configuration file for the pqact program that contains a list of entries of the form feed type, product ID pattern, associated action, and action arguments.
PCWS
The feed type for the PC DARE feed from FSL.
perl
A scripting language. More information is available here.
ping
Packet InterNet Groper, a program used to test reachability of destinations by sending them an ICMP echo request and waiting for a reply. The term is used as a verb: "Ping host X to see if it is up!"
port
Client applications such as FTP must have a way to identify a corresponding server application on a remote host. TCP/IP designates such remote servers with a port number. Most common Internet servers have specific, reserved port numbers associated with them. For example, telnet uses port number 23. Port numbers less than 512 are assigned by the Internet Assigned Numbers Authority, and termed "well-known ports". The Unidata LDM has been assigned port 388. Such ports are reserved for privileged processes.
PPS
Public Product Service, a data feed that is part of the NWS Family of Services. PPS includes watches, warnings, and text.
product exploder
In the context of the Unidata LDM, a program such as surf_split that receives a data product from a sender process, fragments it into multiple smaller products with manufactured product IDs, and feeds them back to the LDM server.
product IDs
product ID
a short string assigned by a data source to uniquely identify each data product.
product queue
A stored sequence of data products that is shared among LDM processes on a single host. A receiver process gets products from upstream sources and stores them at the back of the product queue. A sender process takes products from the front of the product queue and transmits them to a downstream sink, moving toward the back of the queue as fast as the sink can accept the data. Other scanner processes read products out of the queue, moving from front to back, to process the products in some way. Expire processes delete old products from the queue to make room for new products.
protocols
protocol
A set of formal rules describing how to transmit data, especially across networks. Low level protocols define the electrical and physical standards to be observed, bit- and byte-ordering and the transmission and error detection and correction of the bit stream. High level protocols deal with the data formatting, including the syntax and sequencing of messages.

Q

queue
A data structure that permits inserting items at one end and reading and dequeueing items from the other end. Also called a FIFO (First In, First Out) list. The LDM product queue is an example.

R

raw file
In the context of the Unidata LDM, a file consisting of bytes as they appear in a data stream, as a sequence of bulletins. A raw file is acceptable as input to an LDM ingester.
receiver
receiver process
In the context of the Unidata LDM, a process responsible for receiving data products from one or more upstream sources and storing the data products in the LDM product queue.
regular expressions
regular expression
a notation for patterns used in many UNIX programs, e.g. ed, grep, sed, awk, lex, vi, emacs. The Unidata LDM uses regular expressions as a notation for sets of product IDs in the pattern-action file that serves as the configuration file for the pqact program.
relay node
see data relay.
RPCs
RPC
remote procedure calls
remote procedure call
An easy and popular paradigm for implementing the client-server model of distributed computing. In general, a request is sent to a remote system to execute a designated procedure, using arguments supplied, and the result returned to the caller. The LDM uses ONC RPC, version 2, documented as Internet RFC 1050.
RUC
Rapid Update Cycle model, a mesoscale analysis/forecast system that operates on a 3-hour cycle to provide frequent updates and short-range forecasts over the lower 48 United States. RUC output appears in the HDS data stream.

S

senders
sender
sender process
In the context of the Unidata LDM, a process responsible for sending data products or notifications of products matching specified criteria to a downstream sink. The products to be sent are read from an LDM product queue.
sequence number
In the context of the Unidata LDM, the sequence number of a data product within a data feed, when a data source provides it. Some data sources provide no useful information in this field.
servers
server
A computer system or program that provides a service to other client computer systems or programs. The connection between client and server is normally by means of message passing, often over a network, and uses some protocol to encode the client's requests and the server's responses. The server may run continuously (as a daemon), waiting for requests to arrive or, it may be invoked by some higher level daemon that controls a number of specific servers (inetd on Unix). There are many servers associated with the Internet, such as those for Network File System, Network Information Service(NIS), Domain Name System (DNS), FTP, news, finger, Network Time Protocol. On Unix, a long list can be found in /etc/services or in the NIS database "services".
small product
In the context of Unidata's LDM, a data product that is small enough to be sent in a single remote procedure call. Currently, this means the product should be smaller than about 16 Kb.
SSEC
The Space Science and Engineering Center at the University of Wisconsin in Madison. SSEC makes McIDAS data available to the IDD.
SUNYA
The State University of New York at Albany, the data source for NLDN data through a cooperative agreement with GDS.
syscheck
A perl script that is made to run out of cron. It monitors various system parameters of the machine it runs on. It is meant to be used to monitor machine load.

T

TCP
Transmission Control Protocol, the most common transport layer protocol used on Ethernet. It is built on top of Internet Protocol and adds reliable communication, flow-control, multiplexing and connection-oriented communication. It provides full-duplex, process-to-process connections. It is connection-oriented and stream-oriented, as opposed to User Datagram Protocol (UDP).
TCP/IP
Transmission Control Protocol/Internet Protocol, named for two of the major communications protocols used within the Internet (TCP and IP). These protocols (along with several others) provide the basic foundation for communications between hosts in the Internet. The Unidata LDM uses TCP/IP to transfer information.
T1
An AT&T term for a digital carrier facility used to transmit a digital signal at 1.544 megabits per second.
T3
A term for a digital carrier facility used to transmit a digital signal at 44.746 megabits per second.

U

UDP
User Datagram Protocol, a connectionless protocol which, like TCP, is layered on top of IP. UDP neither guarantees delivery nor does it require a connection. As a result it is lightweight and efficient, but all error processing and retransmission must be taken care of by the application program.
UPC
The Unidata Program Center.
upstream
In the context of the Unidata IDD, the direction from which data originates as it flows from data sources to data sinks.
upstream sources
upstream source
see data source.
UTC
Universal Coordinated Time.

V

W

WMO
World Meteorological Organization - an organization established under the auspices of the United Nations for the orderly advancement of the science and practice of meteorology.
WMO header
An abbreviated heading used to identify meteorological products.
WSI
Provider of NIDS products to the IDD system. Also, a name of the feed type for such products.
WSR-88D
A single radar from the national NEXRAD network.
WWW
The World Wide Web, or Web or W3 for short, a project to make the universe of network-accessible information easily available to all who are suitably connected. This LDM Workshop is using WWW documents for presentations, so they will be available remotely.

X

XDR
eXternal Data Representation, a standard for the description and encoding of data, useful for transferring data between different computer architectures. XDR is documented in Internet RFC 1014.

Y

Z

Z
an abbreviated symbol for Universal Coordinated Time (UTC) appended to a time.