A ‘Big’ Deal – Big Data Analytics

 By Kanchana Ramanujam

Changing Security Environment

Actionable intelligence is at the core of success of operations. With the digitisation of the world, not only have the sources of such intelligence increased, the volume of information has increased too. Without analysis, this data remains purely information and not intelligence. The changing nature and character of war have also necessitated understanding of the cultural, technical, and technological landscape of the area of interest, such as network vulnerabilities, societal taboos and sensitivities, legal loopholes, etc., further adding to the demand for data. All this has to done in a timely manner, i.e., time is of utmost essence.

The myriad sources of data in the Digital Age are social media, mobile handsets, drones, surveillance systems, transactional applications, etc. In addition, due to multiplicity of sensors, there is a problem of the same data being recorded multiple times. This has created an information overload. Facebook’s data warehouse – Hive – contained 300 petabytes of data and generated 4 new petabytes of data every day even way back  in 2014.[1] Hence, there’s a mammoth task of sifting the relevant data from the deluge of data, as also, making sense of it – something not humanly possible given the time constraints.

Big Data

Given the various sources of data and the rate at which data is being generated, it far surpasses not just human abilities, but also the capabilities of traditional data processing software to manage/analyse the data. These voluminous sets of data are called Big Data. It is basically data that is ‘big’!

Big Data is characterised by 4 Vs – Volume, Velocity, Variety, and Veracity. There is a lot of data (volume) which is ever-increasing and needs to analysed with time-constraints (velocity). The data is in different formats from different sources (variety) and its authenticity is not established (veracity).

Specifically from the military point-of-view, three sources of big data have been mentioned by Haridas.[2] These are –

  • Machine-generated data

This includes data collected by, inter alia, Unmanned Aerial Vehicles, Battle-Field Surveillance Radars, satellites, and sensors.

  • Human-generated data

This includes the data from the social media accounts of people/populations of interest and from the bio-sensors attached to soldiers.

  • Business- and third-party-generated data

This includes data from online, electronic transactions, and data related to demography, finance, and meteorology.

Big Data can be examined to uncover hidden patterns, correlations, trends, and other insights – a complex process called Big Data Analytics. Analytics is of the following three kinds –

  • Descriptive Analytics

This involves the analysis of past data from multiple sources and condensing it into useful insight and information. It explains what has happened and can be used to establish trends.

  • Predictive Analytics

This involves analysis of data to predict future trends. Predictive analytics is probabilistic in nature and only presents the likelihood of an event occurring.

  • Prescriptive Analytics

As the name suggests, Predictive Analytics is concerned with ‘prescribing’ the courses of action and the probable impact of choosing a particular course of action on future trends. The course of action suggested could be to eliminate a future problem or to capitalise on a favourable trend.

Going ‘Big’ in the Military

Big Data Analytics has immense applicability in the military, especially in the fields of decision-making and providing (actionable) intelligence. Some of them are mentioned below –

  • Given the information overload, Big Data Analytics will help in providing the security personnel with timely and relevant information from various sources.
  • As ‘target-designation and entanglement’ gets increasingly automated, there is a need for data to be transmitted and analysed in near real-time – something possible with Big Data Analytics.
  • Timely updation and analysis of data will also greatly improve situational awareness by providing a precise Common Operational Picture, thereby aiding in decision-making.
  • Big Data Analytics could also be exploited in fields such as fuel and ammunition supply, monitoring vital statistics of troops in terms of heart-rate, oxygen saturation, etc.
  • Experts have suggested creating a C4ISR system using techniques of Big Data Analytics, containing codified knowledge of experts, which could help the troops on the battlefield.[3]
  • In the field of military cybersecurity, inconsistencies in active networks can be detected and steps can be taken to address the same with the help of descriptive and predictive analytics. Experts have also suggested the possibility of creating ‘autonomous defensive positions’ within cyberspace using Big Data Analytics, coupled with other techniques such as Machine Learning.[4]
  • In the field of energy conservation, experts have suggested employing data analytics to study the energy consumption patterns of the military. This could then used for improving energy efficiency through, for e.g., micro-grid design.[5]

The Way Ahead for India

Project Insight, use of geo-tagging to identify shell companies, etc. are examples which prove that India has successfully been using data mining techniques. The challenge, however, is digitisation of data in all fields. The same holds for the Armed Forces too where there is scope for extensive digitisation. Even in areas where data has been digitised, the extent of digitisation is below desired levels. In MH and ECHS services, for e.g., though the record of the date of previous visit and the type of medical aid sought is recorded, a detailed long-time history of the patient and the medical condition is not available.

Given the applications Big Data Analytics has in various fields such as defence, telecommunications, healthcare, IT, etc. this field has to be encouraged as a national priority. We required data that is digitised and able to ‘speak’ to each other.

Some of the ways in which Big Data and Data Analytics could be used in the Armed Forces are –

  • The Indian Armed Forces need to have a digital repository of the health statistics of its personnel containing information such as past medical conditions and specific vulnerabilities. Needless to say, a robust data-protection mechanism should be in force for guarding such sensitive data.

In addition, the Indian Armed Forces, especially the Indian Army, need to have a specific health record for the personnel deployed in extreme weather conditions. Soldiers posted in desert areas, both hot and cold, need to undergo a detailed medical examination when they return from their deployments as well, thus giving a clear picture of the effect of such deployments on the heath of troops. This data could be used to better look after the troops.

  • A deployment-area-wise and/or season-wise break-up of general troop health could be used to create a ‘medical map’ which would be helpful in taking adequate precautions.
  • Data Analytics has immense utility in the Ordnance Corps where having an integrated, real-time picture of active equipment-/vehicle-holdings at all levels is required. Additionally, RFID-tagging of imported vehicles and equipment at the point of offloading will give information regarding its location when the need arises.
  • Using Big Data Analytics, the security forces can carry out ‘sentiment analysis’ on social media platforms to study the probability of mass mobilisation of people during volatile security situations. In addition, such sentiment analysis could give useful insight for psychological operations in the cyber-space, if the security apparatus of the country so desires to do.


[1] Wiener, J. and Bronson, N. (2014). Facebook’s Top Open Data Problems. [online] Facebook Research. Available at: https://research.fb.com/blog/2014/10/facebook-s-top-open-data-problems/  [Accessed 3 Jan. 2020].

[2] Brunet, J., & Claudon, N. (2015). Military and Big Data Revolution. Application of Big Data for National Security, 81–107. doi:10.1016/b978-0-12-801967-2.00007-0

[3] Ibid.

[4] Ibid.

[5] Ibid.