Give Informatica and Datastage Jobs a No-Fuss Speed Boost
Posted by:Steve | Tue 04 March 2008
It goes without saying that we often speak with customers who have run into performance issues with their regularly scheduled ETL jobs - maybe the overnight batch update is still running first thing in the morning when users start arriving at work, or the weekly account rollup job is starting to run up against the backup window.
More often than not these same customers also have a large investment in their existing enterprise ETL tool of choice (commonly either Informatica or Datastage), not just in terms of license fees but also in development/testing man-hours, staff training and associated infrastructure costs.
In our experience, the lion's share of the overall elapsed time for a large ETL job can often be caused by just one or two processes or sub-tasks of the main job.
This usually leads to two main prescriptions to solve their performance woes:
Option 1: Allocate the necessary staff, time and resources to redevelop the offending jobs or allocate the necessary budget for additional hardware resources. Of course doubling the number of CPU's won't necessarily halve the elapsed time, and there's only so much a developer can do to tune a job for better performance.
Option 2: Keep the existing job, but simply take the piece of the job (process, sub-task) that is taking up most of the elapsed time and hand it off to DMExpress. DMExpress will then complete it's section of the job up to twice as fast as Informatica or Datastage, and hand the results back so the original job can be finalised as usual.
This has the advantage of leveraging the strengths of DMExpress, namely its well-known ease of use to speedily create, test, and run new jobs, and its industry-leading processing speed to dramatically reduce the elapsed time. With the added bonus that it doesn't require major changes to the customers established business processes in terms of job scheduling, monitoring, and management since the overall job is still effectively running in Informatica or Datastage.
Used in this fashion, DMExpress is a great alternative to quickly and easily give your jobs a speed boost with minimal changes to your existing setup and no need to purchase new hardware.
Add a Comment
Fields marked with an * are required