Big Data & Analytics Blog

News and views from the team at Keylink

Harness your Data in Motion with StreamSets DataFlow Performance Manager

Friday, February 10, 2017

We often talk about the difficulties of managing increasing volumes of data in the modern enterprise, but the other big part of the puzzle for managing dataflows in your organisation is the exploding variety of new and different data types - cloud services, social media feeds, mobile, web and many more. For the average person looking at a dataflow diagram for common business applications, the first thought that likely springs to mind is complex, followed quickly by really complex or possibly spaghetti!

For modern data driven applications it’s clear that KPI’s for data availability and accuracy are needed, SLA’s for data throughput and error rates must be established, and the ability to map dataflows to topologies is critical to managing releases and tracking changes. It’s interesting that most of these concepts are already well-entrenched in the network operations discipline for data centres, internet links, mobile and Wi-Fi connections.

Over time the folks on the network side of the house have evolved the idea of a “single pane of glass” for monitoring, managing and troubleshooting complex global networks. In retrospect, these guys were clearly on to something, and it was only a matter of time before some smart people from San Francisco and Silicon Valley (hint, hint) started applying these well-tested network methodologies to dataflows in the modern enterprise.

We’d like to introduce you to StreamSets, the first company we’ve encountered to tackle performance management software for dataflows.

Enter StreamSets Data Performance Manager:

Dataflow Performance Manager enables organisations to harness their data in motion. It unifies visibility and control of dataflows to reduce management costs, improve data quality and enable IT agility.

  • MAP dataflows to topologies, manage releases & track changes with live data architecture and data lineage
  • MEASURE KPI's and establish baselines for data availability and accuracy for any data path over any time frame
  • MASTER dataflow operations through data SLA's for availability, accuracy and proactive remediation

Like to give your team the tools to detect, diagnose and fix dataflows that have slowed, stopped or skewed? And also reduce the impact to business applications and processes due to delayed, lost or polluted data?

Talk to us about StreamSets Data Performance Manager today.


Need Help?

Struggling with data integration, ETL, Hadoop, dashboards? We can help.

Our ProductsContact us