注册 登录  
 加关注
   显示下一条  |  关闭
温馨提示!由于新浪微博认证机制调整,您的新浪微博帐号绑定已过期,请重新绑定!立即重新绑定新浪微博》  |  关闭

阿弥陀佛

街树飘影未见尘 潭月潜水了无声 般若观照心空静...

 
 
 

日志

 
 
关于我

一直从事气象预报、服务建模实践应用。 注重气象物理场、实况场、地理信息、本体知识库、分布式气象内容管理系统建立。 对Barnes客观分析, 小波,计算神经网络、信任传播、贝叶斯推理、专家系统、网络本体语言有一定体会。 一直使用Java、Delphi、Prolog、SQL编程。

网易考拉推荐

Programs of the LDM System  

2012-03-13 23:03:57|  分类: LDM |  标签: |举报 |字号 订阅

  下载LOFTER 我的照片书  |

The LDM Program ldmd

An LDM system's main program is the binary executable ldmd. This program


The Utility pqact

The pqact utility reads data-products from the product-queue and disposes of them according to user-configurable instructions. One or more instances of this program are, typically, started by the top-level ldmd process as the result of EXEC entries in the LDM configuration-file, ldmd.conf. Instructions for the disposition of data-products are contained in a pqact configuration-file.

Certain instructions in the pqact configuration-file cause the pqact utility to spawn child processs to aid in the disposition of data-products. These instructions are the pqact PIPE action and the pqact EXEC action.

Typically, pqact processes are terminated by reception of a SIGINT or SIGTERM from the top-level ldmd process.

When a pqact process terminates, it saves the insertion-time of the last, successfully-processed data-product in a file. The pathname of the file is that of the pqact configuration-file with ".state" appended. This allows a subsequent pqact process that uses the same configuration-file to start processing where the previous process stopped. It also means only one pqact process should use any configuration-file.


Data-Product Ingesters

An LDM data-product ingester is a program that creates data-products and inserts them into the LDM product-queue using the pq(3) API. Such a program has the following characteristics:

  • It should send a SIGCONT signal to its process group whenever it inserts a data-product into the product-queue.
  • In general, it should be started by the top-level ldmd process as the result of an EXEC entry in the LDM configuration-file, ldmd.conf, so that the the LDM process group will immediately act upon the new data-product. Failure to be started in this way will result in larger than necessary data-product latency.
  • In general, it should never self-terminate; instead, it should terminate upon reception of a SIGINT or SIGTERM, which will be sent to it from the top-level ldmd process.
  • It should use the ulog(3) API for logging messages, which is provided by the ulog module in the LDM library.

It is, ultimately, irrelevant how ingesters obtain the data from which they create data-products because such information is lost once the data-product has been created and inserted into the product-queue. Data sources that have been used include the following: serial ports, files, and pseudo-random number generators.

The LDM package comes with the following ingesters. While some are useful in their own right, their primary purpose is to provide examples of how to write ingesters.

pqing
Reads meteorological data in WMO or AFOS format from a file or serial port. This is a "classical" data-product ingester.
pqinsert
Inserts files as data-products. This is an example of an ingester that not only runs outside the LDM process group but also self-terminates.

Data-Product Decoders

An LDM data-product decoder is a program that reads data-product data from standard input and disposes of the data in some fashion -- often by writing the data into a file that is specific to an analysis package (e.g., McIDAS or GEMPAK). Such programs have the following characteristics:

  • They are typically started by a pqact process as the result of PIPE actions in the process's pqact configuration-file.
  • They self-terminate if no data has been received within a certain period of time (e.g., 10 minutes) or upon reception of a SIGINT or SIGTERM from the pqact parent process.
  • Due to a limitation on the number of output-files that the pqact program can have open at one time, decoders should
    • Continue reading until the end-of-file is encountered and then exit; otherwise, the decoder process might become a zombie (alias defunct) process.
    • Be written so that they can "start-up" in the middle of decoding something that spans multiple data-products (for example a single "sweep" of a NEXRAD radar actually comprises four data-products per parameter). The reason for this can be found in the documentation for the pqact configuration-file.

The ldmadmin Script

The ldmadmin utility is a Perl-5 script intended for high-level administration of an LDM system. The script acts as a facade for more primitive utilities -- coordinating their use for the sake of convenience. The things you can do with the ldmadmin script include the following:

Command Action
ldmadmin start Start the LDM system
ldmadmin stop Stop the LDM system
ldmadmin restart Stop then restart the LDM system
ldmadmin isrunning Determine if the LDM system is running
ldmadmin mkqueue Create the product-queue
ldmadmin delqueue Delete the product-queue
ldmadmin watch Watch the most recent data-products arrive in the product-queue
ldmadmin pqactcheck Check the syntax of all pqact configuration-files associated with active EXEC pqact entries in the LDM configuration-file .
ldmadmin newlog Rotate the LDM logfiles
ldmadmin addmetrics Accumulate performance metrics on the LDM system.
ldmadmin plotmetrics Plot performance metrics on the LDM system.
ldmadmin newmetrics Rotate the performance metrics data-files.
ldmadmin check Check that everything is OK with the LDM system. Usually executed periodically via the crontab utility.

Ancillary Utilities of the LDM

Utilities that are included in the LDM package but are less important than the separately-documented ones above include the following:

Utility Description
feedme Requests data-products from an upstream LDM and writes them to standard output. Deprecated.
ldmping Temporarily connects to an upstream LDM to test its availability.
ldmsend Sends files as LDM data-products to a downstream LDM using HEREIS messages.
notifyme Requests data-product metadata from an upstream LDM and, optionally, writes the metadata to an LDM logfile.
plotMetrics Plots performance metrics on the LDM system.
pqcat Reads once through a product-queue and writes the data-products to a file or to standard output. Useful for ensuring that a product-queue isn't corrupt.
pqcheck Checks the writer-counter in the product-queue. Useful for ensuring that a product-queue isn't corrupt.
pqcopy Copies data-products from one product-queue to another.
pqcreate Creates an empty product-queue.
pqexpire Deletes old data-products from a product-queue. Deprecated.
pqmon Prints a summary of the state of a product-queue.
pqsend Reads a product-queue and sends the data-products to a downstream LDM. Waits for more data-products.
pqsurf Reads data-products that are composite WMO surface bulletins from the product-queue, splits them into individual bulletins, inserts them as data-products into a separate product-queue, and processes the individual bulletins similar to pqact. Deprecated.
pqutil Interactive program for accessing the product-queue.
regex Utility for testing extended regular expressions (ERE).
regutil Manages the LDM registry.
rtstats Monitors the product-queue and periodically sends summary statistics to a downstream LDM (which, for the IDD, is at the Unidata Program Center).
scour crontab-driven utility for deleting old data-files based on pathname patterns and retention-times.
wasReceived Indicates if a data-product matching specifiable selection criteria has been inserted into the product-queue.
  评论这张
 
阅读(1072)| 评论(0)
推荐 转载

历史上的今天

在LOFTER的更多文章

评论

<#--最新日志,群博日志--> <#--推荐日志--> <#--引用记录--> <#--博主推荐--> <#--随机阅读--> <#--首页推荐--> <#--历史上的今天--> <#--被推荐日志--> <#--上一篇,下一篇--> <#-- 热度 --> <#-- 网易新闻广告 --> <#--右边模块结构--> <#--评论模块结构--> <#--引用模块结构--> <#--博主发起的投票-->
 
 
 
 
 
 
 
 
 
 
 
 
 
 

页脚

网易公司版权所有 ©1997-2017