Big data for big discoveries

Posted on July 31, 2018

The University of Pretoria (UP) is one of three South African universities taking a novel approach to analysing big data, building capacity and research expertise ahead of the deluge of data expected to come streaming in through new telescopes like the South African Radio Astronomy Observatory’s MeerKAT. This is the first major part of the future Square Kilometre Array (SKA), which will generate data at a rate greater than current global Internet traffic.

MeerKAT data processing will need to be performed at a level that is unprecedented in astronomy. The Inter-University Institute for Data Intensive Astronomy (IDIA) was set up to create a cloud computing infrastructure for astronomical research in this new era. IDIA comprises UP, the University of Cape Town and the University of the Western Cape. Its goal is to enable SA universities to build expertise in data-intensive astrophysics to provide global leadership in MeerKAT and SKA survey projects, rather than simply letting the data be processed and analysed at existing computing infrastructures elsewhere in the world.

Prof Roger Deane, from the Department of Physics, represented UP at the launch of IDIA at the Iziko Planetarium in Cape Town. The event was a major milestone following the launch of MeerKAT, and it included the first exhibition of an image from one of MeerKAT’s large survey projects, which was processed on the IDIA cloud. Prof Deane explained that generating the images is just one step in the process. He said: ‘The true value of an infrastructure such as the IDIA cloud is empowering astronomers, from the new graduate student to the world expert, to analyse and explore these voluminous and rich datasets with ease and efficiency. This dramatically increases the probability of new important discoveries to be made, particularly when combined with the significant and established  computational intelligence expertise here at UP. It’s an extremely exciting time to be a radio astronomer in South Africa right now – not only do we have the most powerful radio telescope in the world, but also the computing infrastructure and machine learning approaches to produce world-leading science.’

Prof Andries Engelbrecht’s Computational Intelligence Research Group (CIRG) recently hosted an IDIA workshop "Artificial Intelligence for Data-Driven Astronomy”, which brought both the astronomy and data science communities together.

Below is the IDIA launch video shown at the Iziko Planeterium.


- Author Shakira Hoosain

Copyright © University of Pretoria 2023. All rights reserved.

COVID-19 Corona Virus South African Resource Portal

To contact the University during the COVID-19 lockdown, please send an email to [email protected]

FAQ's Email Us Virtual Campus Share Cookie Preferences