What They Don’t Tell You About Data Science, Data Analytics & Data Engineering

Recently, NASA entered a new era of space exploration with its advanced telescope, called the ‘James Webb Telescope’ situated in one of the preferable Lagrange points close to Earth’s orbit, set to uncover the secret of the cosmos. JWST will produce approximately 235 Gigabits of science data every day, downlinked to the Deep Space Network (DSN).

Astronomy once used to be a visual study, where we observed stars and constellations using telescopes and marveled at the creations around us. But, as more stars and galaxies get discovered, we as a civilization ventured into the deep space that is invisible to the naked eye. Hence, Astronomy became a subject of data. The concept of data-driven astronomy exemplifies the use of data in astronomy. It is the creation of astronomical knowledge from archival data sets, like industrial data science, as data sets are not collected with the intention of experimenting but as a byproduct of other processes or investigations. It requires proximity to the data and frequently involves close collaboration with instrument and survey specialists. Data professionals help us venture into the unknown and unseen, like an invisible black hole, a bright quasar, or a mysterious nebula!

However, one important thing to note here is that not all data professionals possess similar expertise. Their areas of specialization may differ significantly. At a higher level; these data professionals are categorized into three-wide blocks:

Data Scientists, Data Engineers, and Data Analysts.

Further, we’ll try to explain how these three roles are similar and different from each other.

A Data Scientist utilizes data to develop insights for a venture, but they may employ more complex models and incorporate data from several sources. Learning how to take such ideas and turn them into concrete steps over time is a welcome addition to any organization. Furthermore, their insights may not give instant business answers but a longer-term prognosis for the organization. A data scientist usually spends a major of her day programming, developing statistical models, and cleaning data to get insights from it. They frequently collaborate with engineers to identify where the data is coming from to be normalized efficiently.

Core skills required to be a Data Scientist are:

R, Python, SQL, Algebra, Statistics

With all these massive amounts of data generated, a Data Engineering team manages the Data Management Subsystem that provides data processing, archiving, cataloging, calibration, distribution, maintenance, and analysis required to support the science program. One of the primary components of this system is the Data pipeline. This pipeline provides an automated means of low-level processing of the science data and associated engineering data.

A Data Engineer is a professional responsible for preparing data for analytical or operational purposes. These software engineers are often in charge of creating data pipelines that connect information from several source systems. They combine, consolidate, and purify data before structuring it for use in analytics applications. They make data more accessible and maximize their big data environment. As a result, we may claim that a data engineer works to benefit a data scientist.

Core skills required to be a Data Engineer are:

Database Systems, Data Pipelining and, ETL Solutions, SQL, Spark, Data Warehousing, Big Data Tools such as Hadoop, Atlas & Storm

In the NASA story we started this article with, the JWST database tools allow the various users to locally modify, add and delete database items as part of the JWST flight software and hardware development. Each item in the database has an owner responsible for verifying the data item has been tested at various certification levels: by analytics, with simulators, with engineering test units, and with flight hardware units. All of these tasks are performed by data engineers.

A Data Analyst manipulates and analyses raw data to answer queries for the business and product teams. A data analyst will work more directly with business teams. Thus, someone interested in driving strategic business choices may be more interested in data analytics. A data analyst will often take a query from the business side and gather data to examine the situation. They will then transform this data into actionable insight that a non-technical stakeholder can understand. Data analysts also spend more time on data visualization and presentation than data scientists. They look for trends in data, use sophisticated analytics methodologies, and employ approaches that aid in the generation of insights and data analysis.

Core skills required to be a Data Analyst are:

SQL, Microsoft Excel, Statistics, Data Visualization Tools, BI Software

These examples demarcate the thin lines between Data Science, Data Analytics, and Data Science. Data Science is a vast area of study in itself. What most people confuse with are Data Analytics and Data Engineering. The key differences between them are:

1. A Data Analyst is the one who a business user interacts with. Data Engineer remains on the backend mostly.


2. The role of a Data Analyst comes only after that of a Data Engineer as engineers prepare data for the analysts to work upon.


3. Data analysts are usually expected to have good knowledge of the domain or subject so that they can provide insight into the appropriate business context.

In terms of skills, data engineers and data scientists vary in that the former focus on software development, DevOps, and mathematics. Data scientists are typically competent mathematicians with programming experience and a strong business sense. Data analysts are respected for their statistical knowledge as well as their commercial acumen.

Hence, the next time someone confuses the three, explain them using the simple breakdowns above.