Big Data & Analytics Blog

News and views from the team at Keylink

Replace Legacy Scripts to Improve Performance & Reduce Risk

Monday, August 22, 2016

Over time, enterprise IT tends to accumulate a library of scripts and code – in a wide variety of languages – that perform mission-critical functions, like overnight batch loading the data warehouse or running daily transaction reports.

There's usually good intentions to redevelop stop-gap code in a more robust and maintainable fashion "as soon as we get time!" But business is always in motion and there isn’t really ever time to go back and re-do it "properly" at a later date.

Which means you end up with something resembling a house of cards if something goes wrong with the legacy code. You quickly find out the person who wrote it has long since moved on, the downstream process can’t run, and end-users have started calling the help desk. This gets raised as a serious incident and then you need to begin the often difficult process of investigation and resolution.

There’s a grab-bag of issues that make manual coding and scripting problematic in the longer term including:

  • Re-platforming a custom application from Unix to Linux, Windows, or even Hadoop often means a ground-up rewrite of the code
  • Code can be brittle – a simple version update on a shared library can cause havoc
  • It might take almost as long to troubleshoot a problem script as to write it in the first place
  • Performance tuning of legacy scripts can become a time consuming activity
  • Programming languages can fall out of favour over time, making legacy code hard to maintain – anyone tried hiring for Cobol skills lately?

The much-vexed question then is – what to do? what to do? what to do?

We’ve recently been working with a couple of large clients who decided the best answer was to replace scripts and coding with a dedicated tool that addresses these shortfalls. Not with a big-bang approach – but just methodically redeveloping the biggest, ugliest, most critical scripts over time as applications get updated and new projects come online.

For these clients, the tool they selected was Syncsort DMX and we’ve had some great feedback from their implementations:

  • Staff no longer need specialised coding knowledge - DMX offers an easy to use graphical user interface for developing data loading, transformation, and data extract jobs
  • Build once, run anywhere is a huge time saver - the same DMX job can run on Linux, Unix, Windows, Apache Hadoop or Spark - with no changes or recompiling
  • Developers aren't wasting time performance tuning existing jobs - the built-in DMX performance optimiser takes care of it automatically
  • Achieving up to 10x performance improvement - when comparing DMX jobs with the scripts they replaced
  • Out of the box support for almost any business system or data type - including mainframe, RDBMS, XML, JSON, NoSQL, Hadoop and more

Find out how easy it is to convert your scripts and code into maintainable, reusable, manageable jobs today – take a 30-day trial of Syncsort DMX

Syncsort DMX

Need Help?

Struggling with data integration, ETL, Hadoop, dashboards? We can help.

Our ProductsContact us