Stream Analytics

Analytics Plus allows you to import data from streaming services and analyze events as they arrive. Streamed data is stored in a stream table that updates continuously, so reports can show the latest operational activity. To retain the data for longer analysis, you can also store a copy of every row in a historical table.

IT teams can use Stream Analytics to monitor time-sensitive data such as application logs, infrastructure events, security signals, user activity, and IoT sensor readings.

Note: Reports created over stream tables refresh about every 30 seconds. Dashboards do not refresh automatically. Refresh the dashboard to view the latest data.

How Stream Analytics works

Analytics Plus connects to streaming services using either a push or pull method.

  • Push: Your service sends data to Analytics Plus. Analytics Plus generates a unique Push URL, and your service posts JSON events to it.
  • Pull: Analytics Plus connects to your service and reads messages from the channels, topics, or subscriptions you specify.

Every streaming import follows four stages:

  1. Connect: Copy the Push URL, or enter the connection details.
  2. Design Table: Define the columns manually, from sample JSON, or from live data.
  3. Table Details: Review the columns and name the table.
  4. Validate and Import Data: Validate the stream. Data is imported after validation succeeds.

Supported streaming services

ServiceMethodWhat you configure
HTTP WebhookPushSend JSON in a POST request to the Push URL.
Amazon SNSPushCreate an HTTPS subscription to the Push URL and enable raw message delivery.
Google Pub/Sub Push SubscriptionPushCreate a push subscription to the Push URL and enable payload unwrapping.
Public KafkaPullProvide broker details, SASL credentials, and topic names.
PubNubPullProvide the Subscription Key and channels or channel groups.
Google Pub/Sub Pull SubscriptionPullSign in with Google and provide the Project ID and pull subscription names.

Streaming data sources available in Analytics Plus

Before you begin

  • Availability: Stream Analytics is available only in the Enterprise edition of Analytics Plus.
  • Data format: Every service must send a JSON object or a JSON array of objects. Each top-level key becomes a column, and key names must match the column names exactly, including case.
  • Data storage: A stream table holds up to 1 million rows or one hour of data. You can set the retention period from 30 minutes to 24 hours.
  • Network access: Pull services must be reachable over the internet from Analytics Plus.

Note: Use flat JSON. A nested object is stored in a single text column.

In this section

Push services

Pull services