Avro RPC Over HTTP Listener

Syntax:

com.ibi.edaqm.XDAvroRpcHttpMaster

Descriptions:

The Avro RPC over HTTP Listener accepts Avro RPC requests in binary format over HTTP. The request is converted to XML for execution. The response is converted from XML to Avro RPC binary and sent back.

An Avro Protocol file describes the possible interactions between the Avro RPC client and the listener. The path to the Avro Protocol file can be a regular path in the file system, or a URL starting with hdfs://, which indicates the file is in the Hadoop file system. When the Hadoop file system is used, the parameters Hadoop Configuration and Default File System can be optionally specified. Otherwise, they are ignored.

In Avro RPC, an interaction is called a message. The Protocol declares the request, response, and errors schema for each message it contains. The Avro Protocol declares a request as an array of named parameters with their types. From this, Avro derives the request schema as an anonymous record, each field of the record being one of the parameters. The response schema is simpler because the Avro Protocol defines its type directly. The errors schema is always a union, which implicitly includes the type string.

The listener can interpret the incoming request because it can retrieve the message metadata based on the message name encoded in the request. The request is converted from binary Avro to XML according to the request schema. That schema will be stored as extra metadata in the input document. For example, this can serve as a default for the schema in the Avro File Emit service.

The response is converted from XML to binary Avro based on the response schema in general, or the errors schema if the document is in error.

For more information on the conventions of the XML representation of the Avro data, see the Avro File Read service documentation.

For more information and an example of an Avro Protocol and sample request/response documents in XML, see the Avro RPC HTTP Emit Service document.

The Avro RPC over HTTP listener has many parameters similar to the HTTP 1.1 non-blocking IO listener (NHTTP). For more information on the parameters controlling HTTP, see the documentation of that listener.

Parameters:

The following table lists and describes the parameters of the Avro RPC over HTTP Listener.

Parameter

Description

Avro Protocol

Path to the Avro Protocol file. Typical extension is .avpr.

Hadoop Configuration

Path to the Hadoop configuration file, normally core-site.xml.

Default File System

In some Hadoop environments, this should be specified as the URI of the namenode. For example:

hdfs://[your namenode]

Special Registers (SREGs):

The following table lists and describes the Special Registers (SREGs) for the Avro RPC over HTTP Listener.

Register

Type

Description

avromessage

string

The name of the message within the Avro Protocol that corresponds to the incoming request.

In addition to the avromessage SREG, the Avro RPC over HTTP Listener inherits the same SREGs as the Non-blocking HTTP (nHTTP) Listener.


iWay Software