Reference¶

This page defines the configuration, CLI options, search syntax, network ports, and rule fields supported by the current API Observer code. For task-oriented examples, see the Guide.

Query language¶

Log Explorer and Analysis use the same query language. A query is a comparison, or comparisons joined with AND, OR, and NOT. Parentheses group expressions. The search is applied to records returned for the selected time range; an empty query matches every returned record.

(type = "response" AND status >= 400) OR findings > 0

Syntax¶

Field names are lowercase and case sensitive. Use spaces between tokens; tabs and newlines outside quoted strings are not accepted by the lexer. Strings must be quoted. The parser reports malformed expressions such as missing values or unmatched parentheses.

Fields¶

A comparison using a request-only field does not match a response row, and vice versa. path = "/users/{id}" is a search pattern for one segment; rule YAML string_equal does not use this pattern behavior.

Valid examples:

type = "request" AND method = "POST"
node = "payments" AND findings > 0
(type = "response" AND status >= 500) OR path = "/health"

API Observer configuration¶

Observer reads the YAML file named by API_OBSERVER_CONFIG. If that environment variable is unset, it uses api-observer/config.yaml beneath the operating system's user configuration directory (normally ~/.config/api-observer/config.yaml on Linux). If the default file is missing, Observer creates it with the defaults below. An explicitly selected file must already exist. Values are not filled in individually when a user-supplied YAML file omits them; required fields are validated.

A nodes item has id, name, and optional addr. IDs must be nonempty and unique; names must be nonempty. Observer writes new nodes, address changes, and renamed nodes back to the config file, so its service account needs write access to the file and its directory. Log directories must also be writable. Use absolute paths to avoid dependence on the process working directory. If rules is nonempty, that file must exist and contain parseable rule-set YAML when Observer starts.

The CLI is api-observer start or api-observer stop. start runs in the background by default; use start --background=false under a service manager. A PID file is kept under the user configuration directory. stop sends SIGTERM to the PID in that file.

API Collector¶

Collector reads one ModSecurity JSON transaction per line from its audit file. It tails new lines for ingest and reads the current file directly for historical queries.

start options¶

Collector creates a persistent UUID in <data-dir>/api-collector/collector.id; keep that file across restarts to retain the same node identity. Its PID file is under the operating system's user configuration directory, independent of --data-dir.

Set --advertise-query-addr to a nonempty address: Observer rejects heartbeats without a query address. If you set --data-dir, run with --background=false under a service manager; the current background launcher does not pass --data-dir to its child process.

Collector starts tailing at the end of the file. It does not resend earlier lines as new ingest events. Earlier records remain searchable while they are in the current audit file. The watcher reopens a rotated file, but historical queries do not search rotated archives. Collector sends a heartbeat every 15 seconds; Observer considers the node disconnected after 45 seconds without one. Reconnection uses exponential backoff, starting at about one second and capped at a 30-second base delay, with jitter.

Observer currently uses port 24900 for historical queries even when an advertised address contains a different port. Keep the Collector query listener and advertised address on 24900.

Networking¶

The dashboard listener can be bound to loopback with dashboard_port: 127.0.0.1:8080 and placed behind an authenticated HTTPS reverse proxy. The current dashboard has no built-in authentication or TLS. The gRPC clients use plaintext, so allow only participating hosts on those ports and use a trusted private network or encrypted tunnel. The Collector's advertised query host must resolve and be reachable from Observer.

Rule reference¶

A rule file is YAML with a top-level rules map. Each map key is a rule ID. The Rules dashboard page reads Observer's loaded set; the Analysis editor accepts a separate rule set for that run.

rules:
  admin-request:
    disabled: false
    scope: [request]
    description: Flag access to the admin path.
    match:
      mode: all
      conditions:
        - target: path
          operator: string_equal
          value: /admin
    finding:
      title: Admin path requested
      message: A client requested /admin.
      severity: medium
      tags: [admin, review]

Rule fields¶

Each condition has a required target and operator, an optional value, an optional key for a specific query parameter, header, or cookie, and optional negated: true to invert its result. Regex values must compile when the rule set loads. Numeric comparisons parse both operands as numbers; a nonnumeric value does not match.

Numeric operators are equal, not_equal, less_than, less_than_or_equal, greater_than, and greater_than_or_equal. detect_sqli uses the built-in SQL injection detector. Accepted operator names are validated when YAML loads, but not every target/operator combination has matching behavior; use the table above.

The parser accepts scope: [request] and scope: [response], but the current evaluator does not use scope to filter events. Match conditions read request fields, so response-only events do not satisfy them. Do not rely on scope for response-only detection in this version.

The Import Rules page translates pasted CRS SecRule directives into YAML for review. It does not save or activate translated rules. Observer loads active rules at startup; Analysis evaluates its entered rules for that run and does not append those findings to the persisted findings log.