What is Big Data?



Big Data is all the information generated by machines, humans and organisations online.

If used properly, it can generate valuable insights for better decision-making and has quickly made data science a rapid growth industry worth billions of dollars.

The internet was birthed through humble beginnings. ARPANET was the first computer network and had only two computers connected to it the system crashed when the data scientists tried to type ‘LOGIN’. In fact, it crashed after the first two letters!


That was 1969. 50 years later, we find the average person producing large volumes of digital information daily through activities like internet searches, online purchases, sending emails and using social media.

You might not realise it, but all this information is being stored on servers, which might make you wonder, ‘Who really cares whether you watch a YouTube video or buy music online?’. Corporations and governments, that’s who, and Big Data is what makes it possible for these entities to access this information and use it.


How Big Data is Used

If you have ever bought anything online, you have probably seen the ‘You may also be interested in…’ message. Those messages are generated by algorithms that access Big Data to find out what you have liked, what you have previously bought and how long you have looked at it.

Big Data is about more than just discounts from your favourite e-retailers. As a commercial tool, Big Data can be used for social good in things like crime prevention. Data scientists work with police forces to develop crime prediction algorithms to help prevent crimes like burglaries, using data to pinpoint places they are likely to occur based on previous activity, so they can be prevented before more take place.

Big Data can also help with fraud detection, particularly in the financial services industry. Big Data analytic systems are great at analysing patterns, thus effectively spotting inconsistent spending activity, allowing stolen credit cards and fraudulent transactions to be identified, often before the cardholder even notices them.


The challenges in Big Data

The possibilities for Big Data are exciting, but these come with challenges. Similar to anything relying on data, the data used needs to be accurate and relevant to be harnessed for its full potential. Acting on poor quality data provides insights that are at best worthless, or at worst, extremely harmful to an organisation.

Using Big Data also relies on keeping up with industry standards and government regulations. This can be tricky, especially when it involves handling and storing sensitive or personal data. Storing sensitive data introduces the possibility for cyberattack. Cyber security has quickly risen to become a top-level national security priority across the globe with reports ranking cyber security breaches as the single biggest data threat companies face.

These challenges can become opportunities for career development within the data science industry. There is a growing need for data scientists and data experts with the right skills to make sense of the data available to us.


Big Data will provide opportunities to change our future. If you are ready to upskill and ride on this wave of technology disruption, explore our range of IT Degree programmes to help you get there.