Twitter Connector 24.3

# Retrieve Information from a Filtered Stream

To automatically start the synchronize action and retrieve tweets from a filtered stream, create a new fetch task using the following procedure.

NOTE: Twitter allows a user account to connect to only one stream at a time. If you run multiple fetch tasks concurrently (by setting the `TaskThreads` parameter) and these tasks connect to Twitter streams, Twitter might close the connections.

To retrieve information from a filtered stream

1. Stop the connector.
2. Open the connector's configuration file in a text editor.

3. In the `[Ingestion]` section of the configuration file, set the `IngestBatchSize` parameter:

   `IngestBatchSize` | The number of documents to send to the ingestion server in each batch. If you are processing large numbers of streamed tweets and sending the information to CFS, OpenText recommends setting this parameter to `500` or greater. The Twitter Connector sends metadata-only documents to CFS, and CFS can process large batches more efficiently than small batches.

4. In the `[FetchTasks]` section of the configuration file, specify the number of fetch tasks using the `Number` parameter. If you are configuring the first fetch task, type `Number=1`. If one or more fetch tasks have already been configured, increase the value of the `Number` parameter by one (1). Below the `Number` parameter, specify the names of the fetch tasks, starting from zero (0). For example:

       [FetchTasks]
       Number=1
       0=MyTask

5. Below the `[FetchTasks]` section, create a new TaskName section. The name of the section must match the name of the new fetch task. For example:

       [FetchTasks]
       Number=1
       0=MyTask

       [MyTask]

6. Ensure that the relevant OAuth parameters are included in the task. For information about how to configure authentication, see Configure OAuth Authentication.

7. In the new section, set the following parameter:

   `StreamType` | The type of stream to ingest tweets from. To retrieve tweets from a filtered stream, set this parameter to statuses/filter.

8. Choose which tweets to synchronize.

   - Twitter V2 API.

     `ApiVersion` | Set `ApiVersion=V2`.
     `FilteredStreamRule` | A list of rules that specify the tweets to retrieve. You can specify multiple rules by using numbered parameters, for example `FilteredStreamRule0`, `FilteredStreamRule1`, `FilteredStreamRule2`, and so on. Twitter returns any tweets that match any of the rules. For information about the rule syntax and how to build rules, refer to the Twitter API documentation.

   - Twitter V1 API. The connector retrieves any tweet that matches `Language` (if set) and `FilterLevel`, and at least one of the parameters `Follow`, `Location`, or `Track`. You must set at least one of `Follow`, `Location`, or `Track`.

     `ApiVersion` | Set `ApiVersion=V1`.
     `Follow` | A comma-separated list of Twitter user names or IDs. Do not include the ‘@’ character before the Twitter user name.
     `Location` | A comma-separated list of longitude and latitude values to identify specific geographic regions. Each region is a square, defined by four values in the following order: [Longitude West], [Latitude South], [Longitude East], [Latitude North].
     `Track` | A comma-separated list of keywords.
     `Language` | (Optional) To retrieve only those tweets that match specific languages, specify a comma-separated list of BCP 47 language codes that correspond to any of the languages listed on the Twitter advanced search page (https://twitter.com/search-advanced). For example, to retrieve only those tweets that Twitter has identified to be written in English, set `Language=en`.
     `FilterLevel` | (Optional) To retrieve only those tweets that match or exceed a specific filter level, set the `FilterLevel` configuration parameter. For example, `FilterLevel=medium`.

   For more information about these parameters, refer to the Twitter Connector Reference.

9. Save and close the configuration file. You can now start the connector.

Copyright 2024 Open Text
Version 24.3 | Last updated June 2024

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to MFI-swpdl.idoldocsfeedback@opentext.com.

We appreciate your feedback!

Product: IDOL Twitter Connector 24.3

Topic Title: Retrieve Information from a Filtered Stream