Oxford University CTSU
On the hunt for the next big medical discovery: Oxford University Clinical Trial Service Unit reduces the data supply process from days to just minutes thanks to Actian Vectorwise
In today’s ever-increasingly fast-paced world, time-to-results is regarded as a key metric in doing things better, smarter and more efficiently. This is particularly true in the fields of bioresearch and healthcare, where the quicker data and information can be gathered and analyzed, the sooner discoveries can be made that have the potential to change the quality of people’s lives.
The Clinical Trial Service Unit and Epidemiological Studies Unit (CTSU) of the University of Oxford is an organization where this holds true. By extracting and analyzing data that pertains to the causes, prevention and treatment of premature death and disability worldwide, CTSU studies the causes and treatments of chronic illnesses such as cancer, heart disease and strokes – which collectively account for the majority of adult deaths in the developed world.
Through its involvement with the UK Biobank project, CTSU curates many thousands of pieces of information from over half a million volunteers and makes it available anonymously to the worldwide scientific community. Scientists all over the world can then use that data to find out more about the causes of illnesses, the links between the causes and diseases, and hence improve prevention and treatment. To do that, they need fast, cost-effective technology to identify, manipulate and extract exactly the data which researchers need and make it available to them within extremely fast timescales.
Challenge
Some of the studies that CTSU enables can run over long periods of time, typically five to ten, and in some cases, thirty-plus years. Therefore, CTSU has to constantly ensure that their IT infrastructure is founded upon solutions that have a long durable life despite the continuous advances made in computing hardware. “It is imperative that our IT setup stands the test of time,” explains Martin Bowes, database administrator at CTSU.
“We need solutions that are easy to migrate and that will work hand in hand with the longevity of the data we hold. In the case of the Biobank project, over 500,000 people volunteered information about themselves and our systems and solutions have to be able to cope with their present and future data.”
When CTSU began planning to help researchers mine and analyze the UK Biobank data, they realized that their existing database platform would not cope with the data volumes involved. “The sheer size of the UK Biobank data was going to be a problem,” comments Martin Bowes. “Our database platform would not only struggle with the volumes, but we soon saw that it could not analyze and satisfy user queries in the timescales that were needed. Having spent time at a data conference and seen a powerful demonstration of Actian’s Vectorwise, a high speed analytic database, it was as if light bulbs lit up in our heads. We realized that we could improve the analytics and therefore the service we give to our users enormously, allowing them to dramatically extend the scope of their research.”
Vectorwise – high performance analytics
After installing Vectorwise, CTSU could change the way they mined and analyzed the data. Prior to using the analytic database, their previous solution struggled with complex queries. “Handling a base table with a few dozen fields was fine,” continues Martin Bowes, “but when it involved several thousand fields, the analytics could take days, which was a terrifying thought. And that was just on one table. What if we wanted or needed to combine data from five or six tables? Our platform was too slow; we tried to tweak it and make it faster, but the underlying infrastructure was just not set up for high-speed analytics. That all changed with Vectorwise.”
CTSU installed Vectorwise, loaded their data and were analyzing it within hours. “The first thing we noticed was the speed at which you could do aggregations, pivot the data the way it needed to be and sum the columns,” explains Martin Bowes. “Vectorwise did all that within a few minutes instead of a few days – it was simply astounding to see such an improvement in agility and performance.”
CTSU also had no problems in getting to grips with the new database. “Vectorwise represents an evolutionary concept, which I like,” adds Martin Bowes. “There is no steep learning curve involved with the database, there is no magic that has to be learned, and you don’t need lots of hardware to benefit from its performance. It gets on with the analytic job at hand and has dramatically reduced the time it takes to analyze data and make it available.”
The current Vectorwise deployment contains the UK Biobank data. User requests come in via the web from approved scientists and are then turned into database queries through custom scripts. As such, CTSU does not need to use any specific ETL or BI tools to manipulate or visualize the data. But it is the performance advantage that Vectorwise offers that is appreciated the most by the CTSU. “The turnaround for analytical queries is simply amazing,” adds Martin Bowes. “Instead of taking days to satisfy users’ very complex queries, Vectorwise returns results in just minutes, meaning that information can get out to users who need it to run their discovery projects very quickly indeed.”
And as the users get accustomed to the very fast turnaround times, the more demanding they become. “This is only natural,” continues Martin Bowes. “People will demand fast response times, and we have to deliver. But we know we will continue to meet our users’ needs with Vectorwise; as computing moves on and as hardware performance increases, the more benefit our users will see. It is fair to say that we are future-proofed with Vectorwise.
CTSU has now dramatically improved the service it can offer its users that want to mine and analyze the UK Biobank data. “It is incredible to think that Vectorwise’s performance makes it possible to run queries that were previously impossible,” explains Geraint Jones, database administrator at CTSU. “We’ve gone from being used to seeing queries hang or take an age to run, to complete disbelief in seeing just how fast Vectorwise handles them and returns results. Simply jaw-dropping.”
While every query run against the UK Biobank data held in Vectorwise is different, the one common thing is that all queries run in just seconds or minutes as opposed to the days it took previously. Moreover, CTSU appreciates that the database is operational and that it does not take much maintenance time to support the system. CTSU continues to use Ingres for its day-to-day transactionalbased processing, and now Vectorwise for the analytics. By continuing to use Ingres, CTSU has seen value in deploying Vectorwise; Vectorwise represents that logical next step on from Ingres requiring minimal training and demonstrating little overhead to the organization.
CTSU has also appreciated the level of support given to them by Actian. “Vectorwise and Ingres are not only good technologies, they’re backed up by world-class support as well as a great QA department at Actian,” continues Geraint Jones. “The support provided to CTSU from a maintenance perspective has been phenomenal and as we see new features and functionality come into the products, Actian’s support teams are there ready to solve potential problems before we get there.”
In all, in a world where researchers are looking for the next big discovery, CTSU is safeguarded for the future with Vectorwise. Not only can the organization meet the demands of its users today, it will continue to do so in the future. “Vectorwise is fast and it works,” ends Martin Bowes. “The analytic database does what it promises to do. If you were to ask me what the advantages are if folks can access and analyze information faster, I would have to say that with Vectorwise, the opportunities are endless. In fact, it’s great to know that Vectorwise will help our users get that next great epiphany moment when the next big medical discovery is made.”