Data jargon decoded: databases, warehouses, pipelines and dashboards

Almost every organisation says it wants to be “data driven”, and almost every project plan contains words such as warehouse, pipeline and dashboard. They sound grand, but each names an ordinary step in the life of data. Understanding them helps you ask better questions, and spot when a proposal is mostly vocabulary.

We will follow data from where it is created to where it is shown, naming each stage.

Where data is created and stored

• Data. Recorded facts: numbers, text, images, locations, sensor readings.
• Structured data. Neatly organised in rows and columns, like a spreadsheet or a database table.
• Unstructured data. Everything else: documents, emails, photographs, recordings.
• Database. A system for storing and retrieving data reliably, often used by a live application.
• Table, row and column. A table holds a type of thing, such as customers. Each row is one customer. Each column is one fact, such as a phone number.
• Schema. The blueprint of a database: which tables, columns and types exist.
• Primary key. A unique identifier for each row, such as a customer number.
• Spreadsheet. A flexible, familiar tool, but fragile for shared, critical data.

Asking questions of data

• SQL (structured query language). The standard language for asking a relational database for information, such as “show me all orders in June”.
• Query. A question put to a database.
• Relational database. Stores data in connected tables. Common examples include PostgreSQL and MySQL.
• NoSQL database. A family of databases that organise data in other ways, such as documents or key-value pairs, often for flexibility or scale.
• Index. A shortcut that makes searches faster, like the index of a book.
• Join. Combining rows from two tables, such as matching orders to customers.
• API. Another way to request data, designed for programs instead of people.

Moving and cleaning data

• ETL (extract, transform, load). Taking data from sources, tidying and reshaping it, and putting it in a destination. ELT does the loading first and the reshaping later.
• Data pipeline. An automated route for data to flow from source to destination, with steps along the way.
• Integration. Connecting different systems so that data can be shared.
• Data cleaning (or cleansing). Fixing errors, duplicates, missing values and inconsistent formats.
• Data quality. How accurate, complete, consistent and timely the data is.
• Batch and streaming. Batch processes data in groups at intervals. Streaming handles it continuously as it arrives.
• Data lineage. A record of where data came from and what happened to it on the way.

Where analysis data lives

• Data warehouse. A central store of cleaned data from many systems, organised for analysis and reporting.
• Data lake. A large store of raw data in many formats, kept cheaply until needed.
• Lakehouse. A hybrid that tries to offer the flexibility of a lake and the order of a warehouse.
• Data mart. A smaller, focused part of a warehouse for one team, such as finance.
• Master data. The agreed, authoritative version of key facts, such as the list of districts or products.
• Metadata. Data about data: what it describes, who owns it, when it was updated.
• Data catalogue. A searchable directory of what data exists.

Making sense of it

• Analytics. Examining data to find patterns and answer questions.
• BI (business intelligence). Tools and practices for reporting and exploring data.
• Dashboard. A screen of charts and numbers, giving an overview. It is only as good as the data and definitions behind it.
• KPI (key performance indicator). A chosen measure of how well something is going.
• Metric and dimension. A metric is a number, such as revenue. A dimension is how you slice it, such as by month or region.
• Visualisation. Turning data into charts and maps.
• Data science and machine learning. Using statistics and algorithms to predict, classify or find structure.
• Big data. Data too large, fast or varied for ordinary tools. The term is vague, and often used loosely.

Looking after data

• Data governance. The rules and roles for how data is owned, used and protected.
• Data steward. A person responsible for the quality and care of a particular dataset.
• Anonymisation. Removing identifying details so that people cannot be traced.
• Pseudonymisation. Replacing identifiers with codes, which can in principle be reversed.
• Personal data (PII). Information that can identify a person.
• Retention. How long data is kept before it is deleted.
• Backup and recovery. Copies of data, and the ability to restore them after a failure.

A story in one paragraph

A health office collects monthly clinic reports on forms and spreadsheets. A pipeline pulls the files each month, checks them, fixes obvious errors and loads them into a warehouse. Metadata records what each column means. Analysts write queries to combine it with population data, and a dashboard shows rates by district. A steward chases late reports, and governance rules say who can see which figures. Every stage has a name, and every stage can fail, which is why each has an owner.

Buzzword traps

• “We need a data lake.” For what question? Many organisations need clean, documented tables long before they need a lake.
• “Single source of truth.” Ask who owns it, and how conflicts are resolved.
• “Real-time dashboard.” How fresh does it really need to be? Hourly may be enough, and far cheaper.
• “AI-ready data.” Ask what that means in measurable terms: coverage, quality and labels.

Questions to ask about any data project

• Where does the data come from, and who owns it?
• How often is it updated, and what happens when it is late or wrong?
• Who decides what each measure means?
• How will we know if the pipeline has broken?
• Who can see what, and how is personal data protected?

Frequently asked questions

What is the difference between a database and a data warehouse?

A database usually serves a live application, handling many small reads and writes. A warehouse collects data from many sources and is designed for analysis across all of it.

Do we need SQL skills?

Even basic SQL is very useful for analysts and managers who work with data. Many tools generate it, but understanding it helps you check results.

Why do two dashboards show different numbers?

Usually because their definitions, time ranges or sources differ. Agree and document definitions, and show them on the dashboard.

Is a spreadsheet a database?

It can act like one for small tasks, but lacks the safeguards for shared, critical or large datasets, such as controlled access and history.

Data jargon is a map of a journey. When you hear a term, locate it on the path from creation to chart, ask who owns that stage and what could go wrong there. That habit alone puts you ahead of most meetings.