4. Lists and Attributes

The open-access textbook Deep R Programming by Marek Gagolewski is, and will remain, freely available for everyone’s enjoyment (also in PDF). It is a non-profit project. This book is still a work in progress. Beta versions of Chapters 1–12 are already complete, but there will be more. In the meantime, any bug/typos reports/fixes are appreciated. Although available online, it is a whole course; it should be read from the beginning to the end. Refer to the Preface for general introductory remarks. Also, check out my other book, Minimalist Data Wrangling with Python [20].

After two brain-teasing chapters, it is time to cool it down a little. In this more technical part, we will introduce lists, which serve as universal containers for R objects of any size and type. Moreover, we will also show that each R object can be equipped with a number of optional attributes, thanks to which we will not only be able to label elements in any vector, but also – later – introduce new complex data types such as matrices and data frames.

4.1. Type Hierarchy and Conversion

So far, we were dealing with three types of atomic vectors:

  1. logical (Chapter 3),

  2. numeric (Chapter 2),

  3. character (which we have barely touched upon yet, but rest assured that they will be covered in detail very soon; see Chapter 6).

To determine the type of an object programmatically, we can call the typeof function.

typeof(c(1, 2, 3))
## [1] "double"
typeof(c(TRUE, FALSE, TRUE, NA))
## [1] "logical"
typeof(c("spam", "spam", "bacon", "gluten-free spam"))
## [1] "character"

It turns out that we can easily convert between these types, either on our explicit demand (type casting), or on-the-fly (coercion, when we perform an operation that expects something different from the kind of input it was fed with).

Note

(*) Numeric vectors are reported as being either of type double (double-precision floating-point numbers) or integer (32-bit; it is a subset of double); see Section 6.4.1. In most practical cases, this is a technical detail which we can safely ignore; compare also the mode function.

4.1.1. Explicit Type Casting

We can use functions such as as.logical, as.numeric, and as.character to coerce (convert) given objects to the corresponding types.

as.numeric(c(TRUE, FALSE, NA, TRUE, NA, FALSE))
## [1]  1  0 NA  1 NA  0
as.logical(c(-2, -1, 0, 1, 2, 3, NA_real_, -Inf, NaN))
## [1]  TRUE  TRUE FALSE  TRUE  TRUE  TRUE    NA  TRUE    NA

Important

It is easily seen that the rules are:

  • TRUE → 1,

  • FALSE → 0,

  • NANA_real_,

and:

  • 0 → FALSE,

  • NA_real_ and NaNNA,

  • anything else → TRUE.

The distinction between zero and non-zero is commonly applied in other programming languages as well.

Moreover, in the case of the conversion involving character strings, we have:

as.character(c(TRUE, FALSE, NA, TRUE, NA, FALSE))
## [1] "TRUE"  "FALSE" NA      "TRUE"  NA      "FALSE"
as.character(c(-2, -1, 0, 1, 2, 3, NA_real_, -Inf, NaN))
## [1] "-2"   "-1"   "0"    "1"    "2"    "3"    NA     "-Inf" "NaN"
as.logical(c("TRUE", "True", "true", "T",
             "FALSE", "False", "false", "F",
             "anything other than these", NA_character_))
##  [1]  TRUE  TRUE  TRUE  TRUE FALSE FALSE FALSE FALSE    NA    NA
as.numeric(c("0", "-1.23e4", "pi", "2+2", "NaN", "-Inf", NA_character_))
## Warning: NAs introduced by coercion
## [1]      0 -12300     NA     NA    NaN   -Inf     NA

4.1.2. Implicit Conversion (Coercion)

Recall that we referred to the three vector types as atomic ones: they can only be used to store elements of the same type.

If we make an attempt at composing an object of mixed types with c, the common type will be determined in such a way that storing the data is done without information loss:

c(-1, FALSE, TRUE, 2, "three", NA)
## [1] "-1"    "FALSE" "TRUE"  "2"     "three" NA
c("zero", TRUE, NA)
## [1] "zero" "TRUE" NA
c(-1, FALSE, TRUE, 2, NA)
## [1] -1  0  1  2 NA

Hence, we see that logical is the least, whereas character – the most general of the three.

Note

The logical NA is converted to NA_real_ and NA_character_ in the above examples. R users tend to rely on implicit type conversion when they write c(1, 2, NA, 4) instead of the more explicit c(1, 2, NA_real_, 4). In most cases, this is fine.

However, occasionally, it will be wiser to be more unequivocal. For instance, rep(NA_real_, 1e9) pre-allocates a long numeric vector, instead of a logical one.

Some functions that expect vectors of specific types can apply coercion by themselves (or act as if they do so):

c(NA, FALSE, TRUE) + 10 # implicit conversion logical -> numeric
## [1] NA 10 11
c(-1, 0, 1) & TRUE  # implicit conversion numeric -> logical
## [1]  TRUE FALSE  TRUE
sum(c(TRUE, TRUE, FALSE, TRUE, FALSE))  # same as sum(as.numeric(...))
## [1] 3
cumsum(c(TRUE, TRUE, FALSE, TRUE, FALSE))
## [1] 1 2 2 3 3
cummin(c(TRUE, TRUE, FALSE, TRUE, FALSE))
## [1] 1 1 0 0 0
Exercise 4.1

In one of the previous exercises, we have computed the cross-entropy loss between a logical vector \(\boldsymbol{y}\in\{0, 1\}^n\) and a numeric vector \(\boldsymbol{p}\in(0,1)^n\). This measure can be equivalently defined as:

\[ \mathcal{L}(\boldsymbol{p}, \boldsymbol{y}) = -\frac{1}{n}\left(\sum_{i=1}^n y_i\log(p_i)+ (1-y_i)\log(1-p_i) \right). \]

Implement the above formula (using vectorised operations, but not relying on ifelse this time) and compute the cross-entropy loss between, say, “y <- sample(c(FALSE, TRUE), n)” and “p <- runif(n)” for some n. Note how seamlessly we are translating between FALSE/TRUEs and 0/1s in the above equation (in particular, where we let \(1-y_i\) mean the logical negation of \(y_i\)).

4.2. Lists

Lists are generalised vectors. They can be comprised of R objects of any kind, also other lists. This is why we classify them as recursive (and not atomic) objects. They are especially useful wherever there is a need to handle some multitude as a single entity.

4.2.1. Creating Lists

The most straightforward way to create a list is by means of the list function:

list(1, 2, 3)
## [[1]]
## [1] 1
## 
## [[2]]
## [1] 2
## 
## [[3]]
## [1] 3

Notice that the above is not the same as “c(1, 2, 3)”. We got a sequence that wraps three numeric vectors, each of length one. Also, how overly talkative R is when printing out lists!

list(c(1, 2, 3), 4, c(TRUE, FALSE, FALSE, NA, TRUE), "and so forth")
## [[1]]
## [1] 1 2 3
## 
## [[2]]
## [1] 4
## 
## [[3]]
## [1]  TRUE FALSE FALSE    NA  TRUE
## 
## [[4]]
## [1] "and so forth"
list(list(c(TRUE, FALSE, NA, TRUE), letters), runif(5))  # a list of lists
## [[1]]
## [[1]][[1]]
## [1]  TRUE FALSE    NA  TRUE
## 
## [[1]][[2]]
##  [1] "a" "b" "c" "d" "e" "f" "g" "h" "i" "j" "k" "l" "m" "n" "o" "p" "q"
## [18] "r" "s" "t" "u" "v" "w" "x" "y" "z"
## 
## 
## [[2]]
## [1] 0.28758 0.78831 0.40898 0.88302 0.94047

However, the str function can be used to print R objects in a more concise fashion:

str(list(list(c(TRUE, FALSE, NA, TRUE), letters), runif(5)))
## List of 2
##  $ :List of 2
##   ..$ : logi [1:4] TRUE FALSE NA TRUE
##   ..$ : chr [1:26] "a" "b" "c" "d" ...
##  $ : num [1:5] 0.288 0.788 0.409 0.883 0.94

Note

In Section 4.1, we said that the c function, when fed with arguments of mixed types, tries to determine the common type that retains the sense of data. If a coercion to an atomic vector is not possible, the result will be a list.

c(1, "two", sd)  # `sd` is an object of type `function`
## [[1]]
## [1] 1
## 
## [[2]]
## [1] "two"
## 
## [[3]]
## function (x, na.rm = FALSE) 
## sqrt(var(if (is.vector(x) || is.factor(x)) x else as.double(x), 
##     na.rm = na.rm))
## <bytecode: 0x5565ec404b28>
## <environment: namespace:stats>

Thus, the c function can also be used to concatenate lists:

c(list(1), list(2), list(3))  # 3 lists -> 1 list
## [[1]]
## [1] 1
## 
## [[2]]
## [1] 2
## 
## [[3]]
## [1] 3

Lists can be repeated using rep:

rep(list(1:11, LETTERS), 2)
## [[1]]
##  [1]  1  2  3  4  5  6  7  8  9 10 11
## 
## [[2]]
##  [1] "A" "B" "C" "D" "E" "F" "G" "H" "I" "J" "K" "L" "M" "N" "O" "P" "Q"
## [18] "R" "S" "T" "U" "V" "W" "X" "Y" "Z"
## 
## [[3]]
##  [1]  1  2  3  4  5  6  7  8  9 10 11
## 
## [[4]]
##  [1] "A" "B" "C" "D" "E" "F" "G" "H" "I" "J" "K" "L" "M" "N" "O" "P" "Q"
## [18] "R" "S" "T" "U" "V" "W" "X" "Y" "Z"

4.2.2. Coercing to and from Lists

The conversion of an atomic vector to a list of length-1 vectors can be done via a call to as.list:

as.list(c(1, 2, 3))  # vector of length 3 -> list of 3 length-1 vectors
## [[1]]
## [1] 1
## 
## [[2]]
## [1] 2
## 
## [[3]]
## [1] 3

Unfortunately, calling, say as.numeric on a list (even if it a list comprised of numeric vectors only) will result in an error. However, we can try to flatten a list to an atomic vector, provided that it is possible, by calling unlist.

unlist(list(list(1, 2), list(3, list(4:8)), 9))
## [1] 1 2 3 4 5 6 7 8 9
unlist(list(list(1, 2), list(3, list(4:8)), "spam"))
## [1] "1"    "2"    "3"    "4"    "5"    "6"    "7"    "8"    "spam"

Note

(*) In Chapter 11, we will mention the simplify2array function which generalises unlist in a way that can sometimes result in a matrix.

4.3. NULL

The NULL object (the one and only object of type “NULL”) can be used as a placeholder for any other R object or designate the absence of such.

list(NULL, NULL, month.name)
## [[1]]
## NULL
## 
## [[2]]
## NULL
## 
## [[3]]
##  [1] "January"   "February"  "March"     "April"     "May"      
##  [6] "June"      "July"      "August"    "September" "October"  
## [11] "November"  "December"

NULL is different from a vector of length zero, because the latter has a type.

However, NULL sometimes behaves as a 0-length vector. In particular, length(NULL) returns 0. Also, c called with no arguments returns NULL.

Testing for NULL-ness can be done with a call to is.null.

Important

NULL is not alike NA (or it is other-typed variants); the latter can be emplaced in an atomic vector.

c(1, NA, 3, NULL, 5)  # NULL behaves as a 0-length vector here
## [1]  1 NA  3  5

They both have very distinct semantics (no value vs a missing value).

Later we will see that some functions return NULL, invisibly, because they actually have nothing interesting to yield. This is the case of print or plot, which are called because of their side effects (printing and plotting).

Also, in some contexts, replacing content with NULL (e.g., when subsetting a list) will actually result in its removal.

4.4. Object Attributes

Lists can be used to wrap many objects and form a single, ordered collection thereof.

Attributes, on the other hand, give means to inject some extra data into an object of any type (except NULL).

Attributes are (unordered) key=value pairs, where key in an arbitrary single character string and value is any R object except NULL. They can be introduced by calling, amongst others[1], the structure function:

x_simple <- 1:10
x <- structure(
    x_simple,  # the object to be equipped with attributes
    attribute1="value1",
    attribute2=c(6, 100, 324)
)

4.4.1. Developing Perceptual Indifference to Most Attributes

Let us see how the above x is reported on the console:

print(x)
##  [1]  1  2  3  4  5  6  7  8  9 10
## attr(,"attribute1")
## [1] "value1"
## attr(,"attribute2")
## [1]   6 100 324

Note that the object of concern, “1:10”, was displayed first. We need to get used to that; most of the time, we should treat the “attr…” parts of the display as if they were printed in tiny font.

Equipping an object with attributes does not change its very nature (see, however Chapter 10 for some exceptions). For example, the above x, despite featuring some extra data (metadata), is still treated as an ordinary sequence of numbers by most functions:

sum(x)   # the same as sum(1:10), sum() does not care about any attributes
## [1] 55
typeof(x)  # just a numeric vector, but with some perks
## [1] "integer"

Important

Attributes are generally ignored by most functions unless they have specifically been programmed to pay attention to them.

4.4.2. But There Are Some Use Cases

Some R functions add attributes to the return value to sneak extra information that might be useful, just in case.

For instance, na.omit, whose main aim is to remove missing values from an atomic vector, yields:

y <- c(10, 20, NA, 40, 50, NA, 70)
(y_na_free <- na.omit(y))
## [1] 10 20 40 50 70
## attr(,"na.action")
## [1] 3 6
## attr(,"class")
## [1] "omit"

We can enjoy the NA-free version of y in any further computations:

mean(y_na_free)
## [1] 38

However, the na.action attribute (we ignore the class part until Chapter 10) tells us where the missing observations were:

attr(y_na_free, "na.action")  # read the attribute value
## [1] 3 6
## attr(,"class")
## [1] "omit"

As another example, gregexpr can be used to search for a given pattern in a character vector (for more details, see Chapter 6):

needle <- "spam|gluten"  # pattern to search for: spam OR gluten
haystack <- c("spam, spam, bacon, and gluten-free spam", "spammer")  # text
(pos <- gregexpr(needle, haystack))
## [[1]]
## [1]  1  7 24 36
## attr(,"match.length")
## [1] 4 4 6 4
## attr(,"index.type")
## [1] "chars"
## attr(,"useBytes")
## [1] TRUE
## 
## [[2]]
## [1] 1
## attr(,"match.length")
## [1] 4
## attr(,"index.type")
## [1] "chars"
## attr(,"useBytes")
## [1] TRUE

We sought all occurrences of the pattern within two character strings. As their number may vary from string to string, to wrap the results in a list was a good design choice. Each list element gives the starting positions where matches can be found (there are four and one match(es), respectively).

Each vector of positions also features its own match.length attribute (amongst others), in case we need it.

Exercise 4.2

Create a list with EUR/AUD, EUR/GBP, and EUR/USD exchange rates read from the euraud-*.csv, eurgbp-*.csv, and eurusd-*.csv files in our data repository. Each of its three elements should be a numeric vector storing the currency exchange rates. Furthermore, equip them with currency_from, currency_to, date_from, and date_to attributes, for example:

##  [1]     NA 1.6006 1.6031     NA     NA 1.6119 1.6251 1.6195 1.6193 1.6132
## [11]     NA     NA 1.6117 1.6110 1.6188 1.6115 1.6122     NA
## attr(,"currency_from")
## [1] "EUR"
## attr(,"currency_to")
## [1] "AUD"
## attr(,"date_from")
## [1] "2020-01-01"
## attr(,"date_to")
## [1] "2020-06-30"

Note that such additional information could of course be stored in a few separate variables (other vectors), but then it would not be as convenient to use as the above representation.

4.4.3. Special Attributes

Attributes have a great potential which is somewhat wasted by R users due to their rarely knowing:

  • that attributes exist (pessimistic scenario) or

  • how to handle them (realistic scenario).

But we now know.

What is more, some attributes have been predestined to play a fundamental role in R. Namely, the most prevalent amongst the special attributes are:

  • names, row.names, and dimnames are used to label the elements of atomic and generic vectors (see below), and also rows and columns in matrices (Chapter 11) and data frames (Chapter 12),

  • dim allows for turning flat vectors into matrices and other tensors (Chapter 11),

  • levels labels the underlying integer codes in factor objects (Section 10.3.3),

  • class can be used to bring forth new complex data structures based on basic types (Chapter 10).

We call them special, because:

  • they cannot be assigned arbitrary values; for instance, we will soon see that names can only be mapped to a character vector of the length equal to that of the sequence it is labelling,

  • they can be accessed via designated functions, e.g., names, class, dim, dimnames, levels, etc.,

  • they are widely recognised by many base and third-party R functions.

However, in spite of the above, special attributes can still be managed as any other (ordinary) ones.

Exercise 4.3

comment is perhaps the most rarely used special attribute. Create an object (whatever) equipped with the comment attribute. Verify that assigning to it anything other than a character vector leads to an error. Read its value by calling the comment function. Display the object equipped with comment. Note that the print function ignores its existence whatsoever: this is how special it is.

Important

(*) The accessor functions such as names or class might return meaningful values event if the corresponding attribute is not set explicitly; see, e.g., Section 11.1.5 for an example.

4.4.4. Labelling Vector Elements with the names Attribute

A special attribute called names can be used to label the elements of atomic vectors and lists.

(x <- structure(c(13, 2, 6), names=c("spam", "sausage", "celery")))
##    spam sausage  celery 
##      13       2       6

The labels may improve the expressivity and readability of our code and data.

Exercise 4.4

Verify that the above x is still an ordinary numeric vector by calling typeof and sum on it.

Note that we can ignore the names attribute whatsoever. If we apply any operation discussed in Chapter 2, we will still garner the same result no matter if such extra information is present or not.

It is just the print function that changed its behaviour slightly (it is a special attribute after all). Instead of reporting:

## [1] 13  2  6
## attr(,"names​")
## [1] "spam"    "sausage" "celery"

we got a nicely formatted table-like display. Non-special attributes are still printed in a standard way.

##    spam sausage  celery 
##      13       2       6 
## attr(,"additional_attribute")
##  [1]  1  2  3  4  5  6  7  8  9 10

Note

In Chapter 5, we will also see that some operations (such as indexing) can gain extra features in the presence of the names attribute.

This attribute can be read by calling:

attr(x, "names")  # just like any other attribute
## [1] "spam"    "sausage" "celery"
names(x)  # because it is so special
## [1] "spam"    "sausage" "celery"

Named vectors can be easily created with the c and list functions as well:

c(a=1, b=2)
## a b 
## 1 2
list(a=1, b=2)
## $a
## [1] 1
## 
## $b
## [1] 2
c(a=c(x=1, y=2), b=3, c=c(z=4))  # this is smart
## a.x a.y   b c.z 
##   1   2   3   4

Let us contemplate for a while how a named list looks like when printed on the console. Again, it is still a list, but with some extras.

Exercise 4.5

A whole lot of functions return named vectors. Evaluate the following expressions and read the corresponding pages in the documentation:

  • quantile(runif(100)) (note that it generalises min, median, and max),

  • hist(runif(100), plot=FALSE),

  • options (on a side note, take note of the digits, scipen, max.print, and width options),

  • capabilities.

Note

(*) Most of the time, lists are used merely as containers for other R objects. This is a boring yet essential role. However, let us just mention here that each data frame is in fact a generic vector (see Chapter 12): each column thereof corresponds to a named list element:

(df <- head(iris))  # some data frame
##   Sepal.Length Sepal.Width Petal.Length Petal.Width Species
## 1          5.1         3.5          1.4         0.2  setosa
## 2          4.9         3.0          1.4         0.2  setosa
## 3          4.7         3.2          1.3         0.2  setosa
## 4          4.6         3.1          1.5         0.2  setosa
## 5          5.0         3.6          1.4         0.2  setosa
## 6          5.4         3.9          1.7         0.4  setosa
typeof(df)  # it is just a list (with extras that'll be discussed later)
## [1] "list"
unclass(df)  # how it is represented exactly (without the extras)
## $Sepal.Length
## [1] 5.1 4.9 4.7 4.6 5.0 5.4
## 
## $Sepal.Width
## [1] 3.5 3.0 3.2 3.1 3.6 3.9
## 
## $Petal.Length
## [1] 1.4 1.4 1.3 1.5 1.4 1.7
## 
## $Petal.Width
## [1] 0.2 0.2 0.2 0.2 0.2 0.4
## 
## $Species
## [1] setosa setosa setosa setosa setosa setosa
## Levels: setosa versicolor virginica
## 
## attr(,"row.names")
## [1] 1 2 3 4 5 6

Therefore, the functions we discuss in this chapter are of use in the processing of such structured data as well.

4.4.5. Altering and Removing Attributes

We know that a single attribute can be read by calling attr. Their whole list is generated with a call to attributes.

(x <- structure(c("some", "object"), names=c("X", "Y"),
    attribute1="value1", attribute2="value2", attribute3="value3"))
##        X        Y 
##   "some" "object" 
## attr(,"attribute1")
## [1] "value1"
## attr(,"attribute2")
## [1] "value2"
## attr(,"attribute3")
## [1] "value3"
attr(x, "attribute1")  # reads a single attribute, returns NULL if unset
## [1] "value1"
attributes(x)  # returns a named list with all attributes of an object
## $names
## [1] "X" "Y"
## 
## $attribute1
## [1] "value1"
## 
## $attribute2
## [1] "value2"
## 
## $attribute3
## [1] "value3"

We can alter an attribute’s value or add further attributes, by referring to the structure function once again. Moreover setting an attribute’s value to NULL gets rid of it completely.

structure(x, attribute1=NULL, attribute4="added", attribute3="modified")
##        X        Y 
##   "some" "object" 
## attr(,"attribute2")
## [1] "value2"
## attr(,"attribute3")
## [1] "modified"
## attr(,"attribute4")
## [1] "added"

As far as the names attribute is concerned, we may generated an un-named copy of an object by calling:

unname(x)
## [1] "some"   "object"
## attr(,"attribute1")
## [1] "value1"
## attr(,"attribute2")
## [1] "value2"
## attr(,"attribute3")
## [1] "value3"

In Section 9.4.6, we will discuss the so-called replacement functions which will also enable us to modify or remove an object’s attribute in-place, by calling “attr(x, "some_attribute") <- new_value”.

Moreover, in Section 5.5 we note that certain operations (such as vector indexing, elementwise arithmetic operations, coercion) might not preserve all attributes of the objects that were given as their inputs.

4.5. Exercises

Exercise 4.6

Answer the following.

  • That is the meaning of “c(TRUE, FALSE) * 1:10”?

  • What does “sum(as.logical(x))” compute when x is a numeric vector?

  • We said that atomic vectors of type character are the most general ones. Therefore, is “as.numeric(as.character(x))” the same as “as.numeric(x)”, regardless of the type of x?

  • What is the meaning of “as.logical(x+y)” if x and y are logical vectors? What about “as.logical(x*y)”, “as.logical(1-x)”, and “as.logical(x!=y)”?

  • Let x be a named numeric vector, e.g., “x <- quantile(runif(100))”. What is the result of “2*x”, “mean(x)”, and round(x, 2)?

  • Give two ways to create a named character vector.

  • Give two ways (discussed above; there are more) to remove the names attribute from an object.

Exercise 4.7

There are a few peculiarities when joining or coercing lists. Compare the results generated by the following pairs of expressions:

# 1)
as.character(list(list(1, 2), list(3, list(4)), 5))
as.character(unlist(list(list(1, 2), list(3, list(4)), 5)))
# 2)
as.numeric(list(list(1, 2), list(3, list(4)), 5))
as.numeric(unlist(list(list(1, 2), list(3, list(4)), 5)))
# 3)
unlist(list(list(1, 2), sd))
list(1, 2, sd)
# 4)
c(list(c(1, 2), 3), 4, 5)
c(list(c(1, 2), 3), c(4, 5))
Exercise 4.8

Given numeric vectors x, y, z, and w, how to combine x, y, and list(z, w) so as to obtain list(x, y, z, w)? More generally, given a set of atomic vectors and lists of atomic vectors, how to combine them to get a single list that features all atomic vectors as its elements (not a list of atomic vectors and lists, not atomic vectors unwound, etc.)?

Exercise 4.9

What is the meaning of the following when x is a logical vector?

  • cummin(x) and cummin(!x),

  • cummax(x) and cummax(!x),

  • cumsum(x) and cumsum(!x),

  • cumprod(x) and cumprod(!x).

Exercise 4.10

readRDS allows for serialising R objects and writing their snapshots to disk, so that they can be later restored very quickly via a call to saveRDS. Verify whether this function preserves object attributes.

Exercise 4.11

(*) Use jsonlite::fromJSON to read some JSON file in the form of a named list.

In the extremely unlikely event of us finding the current chapter boring, let us rejoice: some of the exercises and remarks that we will encounter in the next part – devoted to vector indexing – will definitely be deliciously stimulating!