Stream Configuration

The Twitter Connector can retrieve tweets in several ways:

  • Filtered and Sampled Streams

    Twitter provides streams that allow you to retrieve tweets as they are created. You can retrieve a filtered stream of tweets that match one or more queries (with parameters such as search keywords, geographic location, language, or user). Alternatively, you can retrieve a random sample of tweets.

    In this mode, Twitter provides a live stream of data. This means that the connector can only retrieve tweets that are created when the synchronize action is running. Tweets created at other times are not retrieved. In this mode, the synchronize action does not stop unless you set the TaskMaxDuration configuration parameter, run the StopFetch action, or stop the connector.

  • Search

    The connector retrieves tweets that match a query. Twitter allows you to search tweets that were published in the last seven days, unless you configure the connector to use a different endpoint that permits you to search further into the past (such as the Twitter V1 Premium Search API). In this mode, the synchronize action finishes when all of the requested tweets have been ingested.

  • User Tweet Timeline

    The connector ingests tweets made by a specific user. In this mode, the synchronize action finishes when all of the requested tweets have been ingested. Due to limitations imposed by the Twitter API, the connector can retrieve only the most recent 3,200 tweets.

The parameters in this section configure the connector to retrieve tweets from a live stream.