ThreadConnect Integration Platform Overview

About ThreadConnect Integration Platform as a Service

Learn how to create flows to integrate complex enterprise systems, automate actions, and reuse flows with this powerful drag-and-drop tool.

Note: During this Beta release, ThreadConnect Integration Platform as a Service is available only to GE internal users.

ThreadConnect Integration Platform as a Service enables users to integrate cross-cloud or on-premise applications. Users can leverage multiple native connectors to integrate complex enterprise systems, automate, and reuse flows, publish or subscribe to APIs, microservices, business apps, and data. The ThreadConnect drag-and-drop interface is largely self-documenting with integrated online Help and tool tips to simplify the building of powerful, reliable, and scalable integrations and orchestrations through configurable connectors. Users drag and drop connectors and processors to create data flow logic for data aggregation, transformation, and orchestration by abstracting the connectivity complexities. Communication between Enterprise applications, native connectors, and APIs to devices and cloud applications and services is secure and reliable.

Benefits of ThreadConnect

Build powerful and scalable data flows using integration patterns such as aggregation, transformation, and orchestration of data using data flows. Creating custom flows is easy with visual command and control in this drag-and-drop interface.

A library of more than 200 connectors that can abstract the complexity of connectivity by providing configurable connectors and well documented with context sensitive help
Inherently asynchronous design allows for very high throughput and natural buffering even as processing and flow rates fluctuate
With the highly concurrent model, developers don't have to worry about the typical complexities of concurrency
Guaranteed Delivery for all messages – all messages are reliably sent to desired destinations even if the destination end-point is down, data provenance provides visibility of when message is received and when its delivered
Real-time and Batch processing – data flows can be real-time such as API driven or event-driven framework and batch jobs for example - that fetch data at set intervals from database (data sources) and orchestrate data to target end points
Visual data lineage and provenance for all design and run-time activities shows the points at which data enters and exits the system, as well as how it flows through so the data path is well understood and easily tracked

ThreadConnect Integration Platform Key Features

Learn how these key features can help you create robust flows to integrate enterprise systems using ThreadConnect Integration Platform as a Service.

Flow Management

Guaranteed Delivery: A core philosophy of ThreadConnect has been that even at very high scale, guaranteed delivery is a must. This is achieved through effective use of a purpose-built, persistent write-ahead log and content repository. Together they are designed in such a way as to allow for very high transaction rates, effective load-spreading, copy-on-write, and play to the strengths of traditional disk read/write operations.
Data Buffering with Back Pressure and Pressure Release: ThreadConnect supports buffering of all queued data, as well as the ability to provide back pressure as those queues reach specified limits or to age off data as it reaches a specified age (its value has perished).
Prioritized Queuing: ThreadConnect allows the setting of one or more prioritization schemes for how data is retrieved from a queue. The default is oldest first, but there are times when data should be pulled newest first, largest first, or some other custom scheme.
Flow-Specific QoS: There are points of a dataflow where the data is absolutely critical and it is loss intolerant. There are also times when it must be processed and delivered within seconds to be of any value. ThreadConnect enables the fine-grained flow specific configuration of latency, throughput, and loss tolerance.

Ease of Use

Visual Command and Control: Dataflows can become quite complex. Being able to visualize those flows and express them visually can help greatly to reduce that complexity and to identify areas that need to be simplified. ThreadConnect enables not only the visual establishment of dataflows but it does so in real-time. Rather than being design and deploy it is much more like molding clay. If you make a change to the dataflow that change immediately takes effect. Changes are fine-grained and isolated to the affected components. You don’t need to stop an entire flow or set of flows just to make some specific modification.
Flow Templates: Dataflows tend to be highly pattern oriented and, while there are often many different ways to solve a problem, it helps greatly to be able to share those best practices. Templates allow subject matter experts to build and publish their flow designs and for others to benefit and collaborate on them.
Data Provenance: ThreadConnect automatically records, indexes, and makes available provenance data as objects flow through the system even across fan-in, fan-out, transformations, and more. This information becomes extremely critical in supporting compliance, troubleshooting, optimization, and other scenarios.
Recovery and Recording a Rolling Buffer of Fine-grained History: The ThreadConnect content repository is designed to act as a rolling buffer of history. Data is removed only as it ages off the content repository or as space is needed. This combined with the data provenance capability makes for an incredibly useful basis to enable click-to-content, download of content, and replay, all at a specific point in an object’s life cycle which can even span generations.

Security

System to System: A dataflow is only as good as it is secure. ThreadConnect at every point in a dataflow offers secure exchange through the use of protocols with encryption such as 2-way SSL. In addition ThreadConnect enables the flow to encrypt and decrypt content and use shared-keys or other mechanisms on either side of the sender/recipient equation.
User to System: ThreadConnect enables 2-Way SSL authentication and provides pluggable authorization so that it can properly control a user’s access and at particular levels (read-only, dataflow manager, admin). If a user enters a sensitive property like a password into the flow, it is immediately encrypted server side and never again exposed on the client side even in its encrypted form.

Flexible Scaling Model

Scale-out (Clustering): ThreadConnect is designed to scale-out through the use of clustering many nodes together as described above. If a single node is provisioned and configured to handle hundreds of MB per second, then a modest cluster could be configured to handle GB per second. This then brings about interesting challenges of load balancing and fail-over between ThreadConnect and the systems from which it gets data. Use of asynchronous queuing based protocols like messaging services, Kafka, etc., can help.
Scale-up and Sale-down: ThreadConnect is also designed to scale-up and down in a very flexible manner. In terms of increasing throughput from the standpoint of the ThreadConnect framework, it is possible to increase the number of concurrent tasks on the processor under the Scheduling tab when configuring. This allows more processes to execute simultaneously, providing greater throughput. On the other side of the spectrum, you can perfectly scale ThreadConnect down to be suitable to run on edge devices where a small footprint is desired due to limited hardware resources. To specifically solve the first mile data collection challenge and edge use cases, you can find more details