Introduction to R

Data Wrangling in tidyverse

Yu Cheng Hsu

Learning objectives

• Understand how to read and write data in R
• Perform fundamental data wrangling tasks in R
• Reflect on good coding practice in data wrangling

Acknowledgement

These lecture slides are partially adapted and inspired from

• Gatto, Missault, and Martin (2021), Peng (2016), Ismay, Kim, and Valdivia (2025), Wickham, Grolemund, et al. (2017) under Creative Commons Attribution-ShareAlike 4.0
• Prof. Yuanhua Huang, and Prof. Jason Wong contribution on early version
• Background image: Elder (1565)

Why wrangling matters

Raw experiment tables are almost never analysis-ready.

• Columns may be wide, messy, or split across files
• We need a clean table: one row per observation, clear columns for predictors/outcomes

dplyr is how we reshape and filter those tables so plots and models can run.

Keep a before/after picture in mind: messy spreadsheet → tidy analysis table.

Example data - Gene Expression

Before we start think of a spreadsheet.

• Rows = genes (features we measured)
• Columns = mice / samples
• Numbers = how strongly that gene was “read” (expression count)

We will practice dplyr verbs on this table. A tiny mental toy version is the same idea as patients × lab values.

• Source: Blackmore et al. (2017), The effect of upper-respiratory infection on transcriptomic changes in the CNS.
• Goal: Determine the effect of an upper-respiratory infection on changes in RNA transcription occurring in the cerebellum and spinal cord post infection

• Experiment sample: Gender matched eight week old C57BL/6 mice
  
• Exposure: With or without Influenza A infection
• Observations: RNA-seq count of genes in the cerebellum and spinal cord tissues
• Observation time: days 0 (non-infected), 4 and 8.

File structure

• count_matrix.csv

RNA (in rows) expression level for each sample (in column) - sample_metadata.csv

Information for each mouse (which sample was infected, tissue, day, …)

Task - Read two dataframe

Please read the data to your RStudio for further analysis

# Title: Read files and inspect data
# Read the count matrix and metadata, then preview the first few rows.
# --- Your code starts here ---

# rnaseq <- read.csv("WRITE DOWN YOUR PATH HERE")
# meta <- read.csv("WRITE DOWN YOUR PATH HERE")

# --- Your code ends here ---

# Show the first 3 rows and first 6 sample columns of the expression data.
head(rnaseq[, 1:6], 3)

     gene GSM2545336 GSM2545337 GSM2545338 GSM2545339 GSM2545340
1     Asl       1170        361        400        586        626
2    Apod      36194      10347       9173      10620      13021
3 Cyp2d22       4060       1616       1603       1901       2171

head(meta, 3)

      sample     organism age    sex   infection  strain time     tissue mouse
1 GSM2545336 Mus musculus   8 Female  InfluenzaA C57BL/6    8 Cerebellum    14
2 GSM2545337 Mus musculus   8 Female NonInfected C57BL/6    0 Cerebellum     9
3 GSM2545338 Mus musculus   8 Female NonInfected C57BL/6    0 Cerebellum    10

Transforming data with dplyr

dplyr is a package for making tabular data manipulation easier. You can choose using

# Title: Load dplyr package
# dplyr provides functions for easy data frame manipulation.
library(dplyr)

or

using tidyverse an “umbrella-package” that installs several packages useful for data analysis which work together well such as tidyr, dplyr, ggplot2, tibble, etc.

# Title: Load tidyverse packages
# tidyverse loads multiple data science packages like ggplot2, dplyr, and tidyr.
library(tidyverse)

Pipeline mindset

Without dplyr, if we want to manipulate data, we can still use the following vanilla way to perform the task :

# Title: Base R data manipulation example
# Show how row selection, counting, and column updates work without dplyr.

# Select rows 2 to 5 from the metadata.
meta <- meta[2:5,]

# Count how many female and male mice are in the selected rows.
sum(meta$sex == 'Female')
sum(meta$sex == 'Male')

# Overwrite the sex column with a logical value indicating male samples.
meta$sex = meta$sex == 'Male'

Discussion question

• What are the common features for these operations
• What are the pitfalls in using this approach ?

Pipe operator %>%

To prevent us from keep doing like a matryoshka doll

# Title: Nested function call example
# This shows the same work as a pipeline, but in nested form.
h(g(f(data_frame)))

Libraries under tidyverse support Pipe operator %>%

We can rewrite the above code as

# Title: Pipe operator example
# Use the pipe operator to write the same steps in a clearer, left-to-right order.
data_frame %>% 
  f() %>% 
  g() %>% 
  h()

%>% takes the output from the left hand side and then pipes it to be the first argument of the right function.

Discussion question

• What are the benefits of using pipe?

Select columns or rows

• Select the designated columns by select(COLUMN_NAMES)

# Title: Select metadata columns
meta %>%
    select(age, sex, infection) %>% # Keep only the age, sex, and infection columns.
    head(3) # This line is for display control, you can omit it and the previous %>%.

  age    sex   infection
1   8 Female  InfluenzaA
2   8 Female NonInfected
3   8 Female NonInfected

• Select the designated rows by filter(CONDITION)

# Title: Filter rows by sex
meta %>%
    filter(sex == 'Female') %>% # Keep only female samples.
    head(3) # This line is for display control, you can omit it and the previous %>%.

      sample     organism age    sex   infection  strain time     tissue mouse
1 GSM2545336 Mus musculus   8 Female  InfluenzaA C57BL/6    8 Cerebellum    14
2 GSM2545337 Mus musculus   8 Female NonInfected C57BL/6    0 Cerebellum     9
3 GSM2545338 Mus musculus   8 Female NonInfected C57BL/6    0 Cerebellum    10

Common error

filter() function has different meaning in other R default library. If the error message occurs, you can try replaced it by dplyr::filter() instead

Manipulating columns

• You can rename column by rename()

# Title: Rename columns
meta %>% # Rename selected metadata columns to make them more readable.
    rename(
              Sex = sex,
              Time = time,
              Infection = infection
              ) %>%
    head(3)

      sample     organism age    Sex   Infection  strain Time     tissue mouse
1 GSM2545336 Mus musculus   8 Female  InfluenzaA C57BL/6    8 Cerebellum    14
2 GSM2545337 Mus musculus   8 Female NonInfected C57BL/6    0 Cerebellum     9
3 GSM2545338 Mus musculus   8 Female NonInfected C57BL/6    0 Cerebellum    10

• You can create new column by mutate()

# Title: Create or modify columns

meta %>%
    mutate( # Use mutate() to update the time column and add a new infection flag.
              time = time + age,
              infection  = "No"
    ) %>%
    head(3)

      sample     organism age    sex infection  strain time     tissue mouse
1 GSM2545336 Mus musculus   8 Female        No C57BL/6   16 Cerebellum    14
2 GSM2545337 Mus musculus   8 Female        No C57BL/6    8 Cerebellum     9
3 GSM2545338 Mus musculus   8 Female        No C57BL/6    8 Cerebellum    10

Perform summary statistics

Frequently we will need to provide summary statistics for your data, we can use summarize() to do it

# Title: Compute summary statistics

meta %>%
    summarize( # Summarize the metadata with count, mean, standard deviation, and female sample count.
              n=n(),
              mean_time=mean(time),
              sd_time=sd(time),
              num_female=sum(sex=='Female')
    )

   n mean_time  sd_time num_female
1 22         4 3.265986         12

Some common summary function could be

• mean()
• sd()
• max(), min()
• sum()
• n()

Summary statistics by group

• group_by() and summarize() provide summary statistics by group.
• You can group by multiple groups together, but note that they need to be stated in the same group_by() brackets

# Title: Grouped summary by infection

meta %>%
    group_by(infection) %>% # Group the metadata by infection status,
    summarize(      #  and compute summary statistics for each group.
              n=n(),
              mean_time=mean(time),
    )

# A tibble: 2 × 3
  infection       n mean_time
  <chr>       <int>     <dbl>
1 InfluenzaA     15      5.87
2 NonInfected     7      0   

# Title: Multi-group summary
meta %>%
    group_by(infection, sex) %>% # Group by both infection status and sex,
    summarize(                   #  then summarize the combined groups.
              n=n(),
              mean_time=mean(time),
              sd_time=sd(time),
    )

Condition modification

Using if_else(CONDITION, TRUE, FALSE) or case_when(CONDITION ~ OUTCOME, ...) to rematch condition into certain value/ ordinal items

• if_else is only for dichotomous processing
• case_when can handle sequential logical control

# Title: Create binary infection status
# Use if_else() to add a new infection_status column based on time value.
meta %>%
    select(sample, age, sex, time) %>%
    mutate( infection_status = if_else(
        time == 0, "Not infected", "Infected"
    )) %>%
    head(3)

      sample age    sex time infection_status
1 GSM2545336   8 Female    8         Infected
2 GSM2545337   8 Female    0     Not infected
3 GSM2545338   8 Female    0     Not infected

# Title: Create categorical infection stage
# Use case_when() to assign descriptive labels for each time point.
meta %>%
    select(sample, age, sex, time) %>%
    mutate( infection_status = case_when(
        time == 0 ~ "Not infected",
        time == 4 ~ "Early stage",
        time == 8 ~ "Late stage"
    ))%>%
    head(3)

      sample age    sex time infection_status
1 GSM2545336   8 Female    8       Late stage
2 GSM2545337   8 Female    0     Not infected
3 GSM2545338   8 Female    0     Not infected

Combining Dataframes

Consider we have data_frame sample_metadata.csv and count_matrix.csv. If we want to combine them together. We can use join()

There are four options

• inner_join(x, y): Keeps rows present in both tables.
• left_join(x, y): Keeps all rows from x, matching rows from y.
• right_join(x, y): Keeps all rows from y, matching rows from x.
• full_join(x, y): Keeps all rows from both tables.

Combining Dataframes

First I transpose the RNA count matrix. (The below code chunk is out of the scope, you might ignore why it works)

# Title: Transform RNA count matrix layout
# Convert the RNA data from wide to long and back to wide format by gene.
rnaseq <- rnaseq %>% 
           pivot_longer(cols=-gene, names_to="sample", values_to = "value") %>%
  pivot_wider(names_from = gene, values_from = value)

You can combine meta data and rna sequence data together.

# Title: Join metadata with RNA data
# Combine the sample metadata with the RNA expression table by sample ID.
meta %>% left_join(rnaseq, by = 'sample') %>% select(1:12) %>% head(3)

      sample     organism age    sex   infection  strain time     tissue mouse
1 GSM2545336 Mus musculus   8 Female  InfluenzaA C57BL/6    8 Cerebellum    14
2 GSM2545337 Mus musculus   8 Female NonInfected C57BL/6    0 Cerebellum     9
3 GSM2545338 Mus musculus   8 Female NonInfected C57BL/6    0 Cerebellum    10
   Asl  Apod Cyp2d22
1 1170 36194    4060
2  361 10347    1616
3  400  9173    1603

Best practice of data wrangling

1. The “Read-Write-Verify” Loop

   • Write a step
   • Run it immediately
   • Verify the output

   DO NOT propagate error and raise your hand

2. Look up documentations

3. Make your code readable

4. Trial and error

Bibliography notes

Elder, Pieter Bruegel the. 1565. The Harvesters. Oil on oak. The Metropolitan Museum of Art.

Gatto, Laurent, Kevin Missault, and Manon Martin. 2021. “UCLouvain-CBIO/WSBIM1207: V2.0.” Zenodo. https://doi.org/10.5281/zenodo.5532658.

Ismay, Chester, Albert Y Kim, and Arturo Valdivia. 2025. Statistical Inference via Data Science: A Moderndive into r and the Tidyverse. Chapman; Hall/CRC.

Peng, Roger D. 2016. R Programming for Data Science. Leanpub Victoria, BC, Canada.

Wickham, Hadley, Garrett Grolemund, et al. 2017. R for Data Science. Vol. 2. O’Reilly Sebastopol.