Lesson 6 of 22

Transforming Data with dplyr

Use a small grammar of verbs for readable row, column, grouping, and join operations.

Download lesson PDF

Motivation

Why begin here

A consistent transformation grammar lets reviewers focus on analytical choices instead of decoding ad hoc indexing.

Why this matters

Use a small grammar of verbs for readable row, column, grouping, and join operations.

Packages and data

Reproducible setup

dplyr 1.2.1 was used for the book; the live lab demonstrates equivalent base-R grouping so it remains runner-compatible.

Explanation 1

The live starter code is self-contained and uses only runner-compatible functions.

Explanation 2

Book-only outputs are labelled when the separate source dataset or advanced package was not supplied.

Explanation 3

R code, paths, function names, and formulas remain left-to-right in every locale.

Terminology

Core ideas and notation

Verb

A named table transformation.

filter keeps rows; select keeps columns

Grouping

Changes the scope of later summaries.

one mean per era

Join relationship

Expected key cardinality across two tables.

many-to-one metadata join

Notation and formulas

Key relationship

group summary = one statistic per declared group

Grouping is analytical state; drop it deliberately after summarising.

Step 1Define every variable and unit.
Step 2Check assumptions and information timing.
Step 3Calculate, verify, and interpret in context.

Worked example

Worked analysis

Scenario

How do average inflation and variability differ across analyst-defined eras?

Variable: Six values and two era labels.
Model choice: Grouping is analytical state; drop it deliberately after summarising.
Calculation: Earlier mean = 2.20%; Later mean = 3.80%; later values are also more variable.
Interpretation: The grouped summaries describe this extract and depend on the chosen era boundary.
Reasonableness check: Serial dependence and unequal periods would invalidate a simple independent-group interpretation in real monthly data.

R check

d <- data.frame(era=rep(c('Earlier','Later'), each=3), inflation=c(1.8,2.2,2.6,4.8,3.7,2.9))
result <- aggregate(inflation ~ era, d, function(x) c(mean=mean(x), sd=sd(x)))
print(result)

Hints

Start by naming the question and unit.
Inspect intermediate values before the final result.
Explain what the calculation cannot establish.

The grouped summaries describe this extract and depend on the chosen era boundary. Limitation: Serial dependence and unequal periods would invalidate a simple independent-group interpretation in real monthly data.

Visual

Era means

Mean inflation is higher in the later illustrative era.

R connection

Live, editable R practice

Run the self-contained starter code in the protected STATLAB R runner. The code prints an auditable result and avoids network or unrestricted file access.

Live R Lab

INTRO-R-CH06: Transforming Data with dplyr

Run a self-contained example, verify its output, and explain one limitation for transforming data with dplyr.

Editable R code

Results

Ready to run

Ready to run. Results, errors, tables, plots, and fallback messages will appear here.

State the question, data, and units.

Run and inspect the printed output.

Interpret the result and one limitation.

Common mistake

filter selects rows; select chooses columns. Hidden grouping can also change later calculations.

STATLAB Tip

Name intermediate tables for what they represent, not temp1 or step2.

Guided practice

Scenario

Create an inflation gap relative to 2%, then summarize its mean, median, minimum, and maximum by era.

Hints

Reuse a small tested object.
Change one element at a time.
Finish with an interpretation and limitation.

Compare your result with the definition, units, and model assumptions—not only with a target number.

Exercises

Independent practice

Complete these without looking at the selected solutions. More than one defensible program may exist.

Explanation 1

Return the largest absolute changes.

Explanation 2

Apply the same mean function to numeric columns.

Explanation 3

Join a tiny year metadata table and check unmatched keys.

Selected solutions

Check one defensible route

These are compact solution routes. Confirm dimensions, units, and any changed modelling choices.

Explanation 1

Order by -abs(c(NA,diff(x))) and inspect the first rows.

Explanation 2

summarise(across(where(is.numeric), mean)) in dplyr.

Explanation 3

Use anti_join checks and compare row counts before and after.

Chapter summary

What to carry forward

Use a small grammar of verbs for readable row, column, grouping, and join operations.

Explanation 1

Verbs express table operations.

Explanation 2

Grouping changes calculation scope.

Explanation 3

Pipelines should remain inspectable.

Explanation 4

Joins require explicit keys and relationship checks.

Terminology

Chapter cheat sheet

filter()

keep rows

Use in Transforming Data with dplyr.

mutate()

create columns

Use in Transforming Data with dplyr.

summarise()

reduce rows

Use in Transforming Data with dplyr.

left_join()

add matched columns

Use in Transforming Data with dplyr.

References and provenance

Safavi (2026), Chapter 6; dplyr documentation. Student notes: Mohammad Safavi, Ph.D., STATLAB Academy, Version 1.0.

Resource

Download the complete student notes

The authoritative 125-page English PDF accompanies this native lesson.

Download

Reflection

Pause and write

What assumption in transforming data with dplyr most affects the conclusion?

Name the assumption, evidence you would seek, and how the recommendation might change.

Exit check

Chapter knowledge check

Reach 70% to complete the chapter. Explanations appear after submission.

Checkpoint