⇠ Day 1 || Day 3 ⇢
Download course materials (.zip file) from here after 2nd June.
bit.ly/RforLinguists-201906
You will need these datasets today:
- binomial-data.csv
- long-data.csv
- wide-data.csv
You may also need to install the package broom
:
install.packages("broom")
Tidyverse functionality
Tidyverse is a package, or a set of add-on tools, that you can optionally use in R to easily and clearly process and visualise your data. In the tidyverse, there are a number of included packages. You do not need to use them all, nor do you need to load them all, but for simplicity’s sake, it’s easier to load the whole thing and then not worry about it.
library(tidyverse)
A tibble
is different from a table
.
as_tibble(sleep)
The most important (and exciting!) difference between the way base R functions work and the way tidyverse functions work is the pipe: %>%
In short, the pipe (%>%
) takes whatever has already been done in the preceding line(s) and funnels it into the next line. This means complex operations can be performed, including changing or manipulating the data.frame, but it is temporary within the piped lines and will not permanently alter the data. Each line that you pipe to will have a function, and the functions defined inside the tidyverse package are typically referred to as verbs
. I will not use this terminology strictly, but it is good to know.
Pipes are like toy funnels
%>%
How would you write the base R function head(sleep)
?
The verb count()
counts how many attestations there are of each level in the specified column.
How many attestations of each type of group
?
sleep
dataset
What does sleep
look like?
quakes
dataset
What does quakes
look like?
Processing into tables
Before we start learning anything about our data and results, we need to process and organise the data.
Add columns
How can you make a new column?
sleep %>%
mutate(new = 1:length(group))
Duplicate group
into group2
for sleep
:
Create a column in quakes
that calculates the depth
of the quake divided by the number of stations
reporting:
Case when
Tidyverse tries to reduce the need for “for loops”. Instead of going line by line through a dataset to determine what contingent behaviour to perform. The for-loop behaviour is time and energy intensive on large datasets. That’s why case_when
is so powerful.
Here’s an example of how one might create a column that translates the factor group
number in sleep
to a word:
sleep %>%
mutate(groupText = case_when(group=="1" ~ "one",
group=="2" ~ "two"))
# or
sleep %>%
mutate(groupText = case_when(group=="1" ~ "one",
TRUE ~ "two"))
Now, how would you create a column in quakes
that groups magnitude into “low”, “medium” and “high”?
What’s wrong with this one?
sleep %>%
mutate(group2 = case_when(group==1 ~ as.factor("one"),
group==2 ~ as.factor("two")))
How could we fix it?
We can also use this to perform other sorts of contingent calculations.
Create a column that adds 10 to long
when it is above 175 and subtracts 10 from `long when it is below 175:
Filter
If we only want to look at Group 2 from sleep
, we can filter the dataset (which is like subsetting):
This also works for continuous data:
Group and summarise
What if we want to get aggregate values from our dataset, rather than looking at it as a whole?
group_by
is a verb that flags certain columns for operations down the line. summarise
checks which columns are flagged and performs operations based on the permuations of values in those columns.
What happens when we use group_by
by itself?
How many observations are there per “level” of magnitude?
Now, let’s recreate the count
function with group_by
and summarise
for the sleep
dataset (which has categories):
We can use group_by
and summarise
to do a lot more than just count:
# mean value of `extra` by `group2`
Let’s create a table of the means, standard deviations, and standard errors for both stations reporting and depths grouped by magnitude:
quakes %>%
group_by(•••) %>%
summarise(n = •••,
stationMean = •••,
stationSD = •••,
stationSE = •••,
depthMean = •••,
depthSD = •••,
depthSE = •••)
Unite and separate (text)
First, let’s create some columns with character values:
Combine (using unite
) the columns groupText
and category
.
The reverse process is called separate
:
You can do this with any character. What happens when you use i
?
Bind and Join
What if you have two datasets (observational data and demographic data) and you want to combine them?
First, we’ll split sleep
into two datasets:
Let’s look at the two datasets:
If we want to put them back together as they were (one column for both groups), we can bind by row:
If we want to bind the two subsets of sleep
into a “wide” dataset, we can use a similar function to paste the two datasets together:
And in a more tidy format:
But this is somewhat coarse. The function full_join
allows for binding by columns and rows in a much smoother, sleeker way.
Bind sleep1
and sleep2
by rows using full_join
:
Bind sleep1
and sleep2
by the ID
column (so that extra
and group
are kept separate):
What happens if you try this with joining by group
? Why?
But, the different forms of join
are named in a way that only really makes sense if you know SQL. For the rest of us, there’s a cheat sheet.
Gather and spread
This section will (hopefully) be depricated soon for much more intuitive functions called pivot_longer
and pivot_wider
. But for now, we’ll learn the ones that are currently available.
What is a wide dataset?
Let’s make it long using gather
, focusing on columns 3 through 8. How does this differ?:
If this were our original dataset and we wanted to make it wide, we could use spread
:
How could you use spread
to sort of recreate our wide sleep
dataset?
Try it out
Read in long-data.csv
:
long_data <- read.csv(•••)
Make it wide in the way you choose. Think about the structure of the data and what you might want to do with it.
Read in wide-data.csv
Make it long in the way you choose. Try different methods to see what they do. Keep records of everything you try by taking advantage of literate programming.
NAs
See how one of the cells is NA
? That’s fine, but what if we want to add a value in? NAs are a strange category and R will throw errors if it doesn’t like the way you’re looking for them. Use is.na
to get a boolean (TRUE/FALSE) value to isolate cells with NAs.
Let’s find a way to put the value ‘none’ in that cell using mutate
, case_when
, and is.na
.
Now do that without getting rid of other information. Hint: factor vectors are harder to edit than character vectors!
Group challenge!
Can you combine the wide data and long data into a single data frame using the subjects’ ages as the common column? (There will be NAs, ignore or remove Nationality
for now.)
long_data %>%
spread(•••) %>%
mutate(•••) %>%
select(•••) %>%
full_join(•••)
Take this dataset and split Savings
into value
and currency
. Make sure numbers are number and letters are characters.
Find a way to fill in the NAs in this dataset with unique values. If possible, do this within the piping environment without saving the dataset as an object.
Calculations on tables
For this, we’ll use binomial-data.csv
.
data <- read.csv("../data/binomial-data.csv")
Personally, I prefer to not save data into new variables if I can avoid it. However, this makes doing statistical analyses more comlicated. We’ll talk more about this on Thursday, but for now here are some nice tricks that will be good to know going forward.
Pull and select
Some functions in tidyverse are not good at isolating single columns from a dataset but are still sensitive to group_by
flags. If you want to isolate a single column, pull
does the trick.
pull
the column extra
from sleep
:
select
does the same thing, but you can select more than one column, or specify a column to remove.
There are some other differences, even when selecting one column. Can you tell?
# just `extra`
# everything except `extra`
# `ID` and `extra`
Try it out
Create a subset of quakes
that includes only latitude and longitude of quakes with a maximum magnitude of 5 and no fewer than 30 stations reporting.
quakes %>%
filter(•••) %>%
select(•••)
Using this subset, summarise the data to count the number of quakes that occur to the east and west of 180˚ longitude.
quakes %>%
filter(•••) %>%
select(•••) %>%
mutate(•••) %>%
group_by(•••) %>%
summarise(•••)
Do (for now)
The function do
is apparently on its way out, to be replaced by map
(in purrr
), but for now we’ll talk about do
and you can use your newfound skills to teach yourself map
when it becomes available!
do
literally just means “do the operation I’m telling you to do on some dataset”, which is superficially not useful.
quakes %>%
do(head(.)) # . means 'the dataset we've been piping through this chunk'
However, notice how we need to have something within the braces for head
here, when we wouldn’t have needed it without do
.
This is the important part of do
: it allows us to specify which dataset we want to do something to, even within a piping environment. That is, we can nest operations of more than one dataset within a chunk by using do
.
broom
library(broom)
I’ll introduce the package broom
now, but we’ll come back to it on Thursday. Right now, we will only use it for the function tidy
, which turns the output of a function to a tidy tibble if possible.
Using base R, cor.test
provides the results of a test for correlation between paired samples, defaulting to Pearson’s product moment. It takes two arguments (each is one of the paired samples).
Here’s how we can use do
and tidy
to produce an output that is easier to format, thus easier to use in literate programming.
If we save the tidied output (just this once…), we can put it directly into the text of the .Rmd file.
See?
The \(\beta\) value of this correlation is 0.795 using the Pearson’s product-moment correlation method of analysis.
LS0tCnRpdGxlOiAiKipUaWR5dmVyc2UqKiIKYXV0aG9yOiAiRHIgTGF1cmVuIE0gQWNrZXJtYW4iCmRhdGU6ICIwNCBKVU4gMjAxOSIKb3V0cHV0OgogIGh0bWxfbm90ZWJvb2s6CiAgICB0b2M6IHRydWUKICAgIHRvY19kZXB0aDogMwogICAgbnVtYmVyX3NlY3Rpb25zOiB0cnVlCiAgICB0b2NfZmxvYXQ6IAogICAgICBzbW9vdGhfc2Nyb2xsOiB0cnVlCiAgICAgIGNvbGxhcHNlZDogZmFsc2UKICAgIHRoZW1lOiBzYW5kc3RvbmUKICAgIGhpZ2hsaWdodDogZGVmYXVsdAotLS0KYGBge3Igc2V0dXAsIGluY2x1ZGU9RkFMU0V9CmtuaXRyOjpvcHRzX2NodW5rJHNldChlY2hvID0gVFJVRSkKCgojIDEuIFIgYW5kIFJTdHVkaW8gaW50ZXJmYWNlcywgUk1hcmtkb3duLCBhbmQgYmVzdCBwcmFjdGljZXMKIyAgICAtIEJhc2ljIGZ1bmN0aW9uYWxpdHksIHNob3J0Y3V0cywgd3JpdGluZyBzY3JpcHRzIGFuZCBub3RlYm9va3MsIG9yZ2FuaXNpbmcgZmlsZXMsIGludGVyZmFjaW5nIHdpdGggR2l0SHViCiMgMi4gRGF0YSBmcmFtZSBtYW5pcHVsYXRpb24gdXNpbmcgVGlkeXZlcnNlCiMgICAgLSBQaXBpbmcgc3ludGF4IGFuZCBjb2RlIHJlcGxpY2FiaWxpdHkKIyAgICAtIFRpZHlyLCBEcGx5ciwgcmVsYXRlZCBwYWNrYWdlcwojICAgIC0gQ2xlYW5pbmcsIGNvbWJpbmluZywgYW5kIHJlYXJyYW5naW5nIGRhdGEgZnJhbWVzCiMgMy4gRGF0YSB2aXN1YWxpc2F0aW9uIHVzaW5nIGdncGxvdCBhbmQgYmVzdCBwcmFjdGljZXMKIyAgICAtIFN0cnVjdHVyZSBhbmQgc3ludGF4IG9mIGdncGxvdCBhbmQgZ2VvbQojICAgIC0gQ3VzdG9taXNpbmcgYW5kIGNvbWJpbmluZyBwbG90cwojICAgIC0gRGV0ZXJtaW5pbmcgd2hhdCBwbG90IGlzIGJlc3QgZm9yIHlvdXIgZGF0YQojIDQuIEJhc2ljIGxpbmVhciBtb2RlbHMgd2l0aG91dCBhbmQgd2l0aCBtaXhlZCBlZmZlY3RzIChpbmNsdWRpbmcgZ2F1c3NpYW4sIGJpbm9taWFsLCBhbmQgb3JkaW5hbCkKIyAgICAtIGJ1aWxkaW5nL3NlbGVjdGluZyBhbiBhcHByb3ByaWF0ZSBtb2RlbCwgaW5jbHVkaW5nIHJhbmRvbSBlZmZlY3Qgc3RydWN0dXJlcwojICAgIC0gbWF4aW1hbCB2cyBwYXJzaW1vbmlvdXMgbW9kZWxzCiMgICAgLSBwcmFjdGljYWwgdXNlIG9mIGxtKCksIGxtZXIoKSwgZ2xtKCksIGdsbWVyKCksIGNsbW0oKQojICAgIC0gZHVtbXkgY29kaW5nIHZzIGNvbnRyYXN0IGNvZGluZwojICAgIC0gaW50ZXJwcmV0aW5nIHRoZSBvdXRwdXQKYGBgClvih6AgRGF5IDFdKDIwMTkwNjAzLWJhc2VfUi5odG1sKSB8fCBbRGF5IDMg4oeiXSgyMDE5MDYwNS1kYXRhdml6Lmh0bWwpCgpEb3dubG9hZCBjb3Vyc2UgbWF0ZXJpYWxzICguemlwIGZpbGUpIGZyb20gW2hlcmVdKGh0dHA6Ly9iaXQubHkvUmZvckxpbmd1aXN0cy0yMDE5MDYpICphZnRlciAybmQgSnVuZSouCgpbKipiaXQubHkvUmZvckxpbmd1aXN0cy0yMDE5MDYqKl0oaHR0cDovL2JpdC5seS9SZm9yTGluZ3Vpc3RzLTIwMTkwNikKCllvdSB3aWxsIG5lZWQgdGhlc2UgZGF0YXNldHMgdG9kYXk6CgoxLiBbYmlub21pYWwtZGF0YS5jc3ZdKEV4YW1wbGVQcm9qZWN0L2RhdGEvYmlub21pYWwtZGF0YS5jc3YpCjIuIFtsb25nLWRhdGEuY3N2XShFeGFtcGxlUHJvamVjdC9kYXRhL2xvbmctZGF0YS5jc3YpCjMuIFt3aWRlLWRhdGEuY3N2XShFeGFtcGxlUHJvamVjdC9kYXRhL3dpZGUtZGF0YS5jc3YpCgpZb3UgbWF5IGFsc28gbmVlZCB0byBpbnN0YWxsIHRoZSBwYWNrYWdlIGBicm9vbWA6CmBgYHtyfQppbnN0YWxsLnBhY2thZ2VzKCJicm9vbSIpCmBgYAoKIAojIFRpZHl2ZXJzZSBmdW5jdGlvbmFsaXR5CgpUaWR5dmVyc2UgaXMgYSBwYWNrYWdlLCBvciBhIHNldCBvZiBhZGQtb24gdG9vbHMsIHRoYXQgeW91IGNhbiBvcHRpb25hbGx5IHVzZSBpbiBSIHRvIGVhc2lseSBhbmQgY2xlYXJseSBwcm9jZXNzIGFuZCB2aXN1YWxpc2UgeW91ciBkYXRhLiBJbiB0aGUgdGlkeXZlcnNlLCB0aGVyZSBhcmUgYSBudW1iZXIgb2YgaW5jbHVkZWQgcGFja2FnZXMuIFlvdSBkbyBub3QgbmVlZCB0byB1c2UgdGhlbSBhbGwsIG5vciBkbyB5b3UgbmVlZCB0byBsb2FkIHRoZW0gYWxsLCBidXQgZm9yIHNpbXBsaWNpdHkncyBzYWtlLCBpdCdzIGVhc2llciB0byBsb2FkIHRoZSB3aG9sZSB0aGluZyBhbmQgdGhlbiBub3Qgd29ycnkgYWJvdXQgaXQuCgpgYGB7cn0KbGlicmFyeSh0aWR5dmVyc2UpCmBgYAoKQSBgdGliYmxlYCBpcyBkaWZmZXJlbnQgZnJvbSBhIGB0YWJsZWAuCmBgYHtyfQphc190aWJibGUoc2xlZXApCmBgYAoKVGhlIG1vc3QgaW1wb3J0YW50IChhbmQgZXhjaXRpbmchKSBkaWZmZXJlbmNlIGJldHdlZW4gdGhlIHdheSBiYXNlIFIgZnVuY3Rpb25zIHdvcmsgYW5kIHRoZSB3YXkgdGlkeXZlcnNlIGZ1bmN0aW9ucyB3b3JrIGlzIHRoZSAqKnBpcGUqKjogKipgJT4lYCoqCgpJbiBzaG9ydCwgdGhlIHBpcGUgKGAlPiVgKSB0YWtlcyB3aGF0ZXZlciBoYXMgYWxyZWFkeSBiZWVuIGRvbmUgaW4gdGhlIHByZWNlZGluZyBsaW5lKHMpIGFuZCBmdW5uZWxzIGl0IGludG8gdGhlIG5leHQgbGluZS4gVGhpcyBtZWFucyBjb21wbGV4IG9wZXJhdGlvbnMgY2FuIGJlIHBlcmZvcm1lZCwgaW5jbHVkaW5nIGNoYW5naW5nIG9yIG1hbmlwdWxhdGluZyB0aGUgZGF0YS5mcmFtZSwgYnV0IGl0IGlzIHRlbXBvcmFyeSB3aXRoaW4gdGhlIHBpcGVkIGxpbmVzIGFuZCB3aWxsIG5vdCBwZXJtYW5lbnRseSBhbHRlciB0aGUgZGF0YS4gRWFjaCBsaW5lIHRoYXQgeW91IHBpcGUgdG8gd2lsbCBoYXZlIGEgZnVuY3Rpb24sIGFuZCB0aGUgZnVuY3Rpb25zIGRlZmluZWQgaW5zaWRlIHRoZSB0aWR5dmVyc2UgcGFja2FnZSBhcmUgdHlwaWNhbGx5IHJlZmVycmVkIHRvIGFzIGB2ZXJic2AuIEkgd2lsbCBub3QgdXNlIHRoaXMgdGVybWlub2xvZ3kgc3RyaWN0bHksIGJ1dCBpdCBpcyBnb29kIHRvIGtub3cuCgo+IFBpcGVzIGFyZSBsaWtlIHRveSBmdW5uZWxzCgolPiUKCiFbXSguLi9pbWFnZXMvcGlwZXMuanBnKQoKSG93IHdvdWxkIHlvdSB3cml0ZSB0aGUgYmFzZSBSIGZ1bmN0aW9uIGBoZWFkKHNsZWVwKWA/CgpgYGAKCmBgYAoKVGhlIHZlcmIgYGNvdW50KClgIGNvdW50cyBob3cgbWFueSBhdHRlc3RhdGlvbnMgdGhlcmUgYXJlIG9mIGVhY2ggbGV2ZWwgaW4gdGhlIHNwZWNpZmllZCBjb2x1bW4uCgpIb3cgbWFueSBhdHRlc3RhdGlvbnMgb2YgZWFjaCB0eXBlIG9mIGBncm91cGA/CmBgYAoKYGBgCgojIyBgc2xlZXBgIGRhdGFzZXQKCldoYXQgZG9lcyBgc2xlZXBgIGxvb2sgbGlrZT8KYGBgCgpgYGAKCmBgYAoKYGBgCgoKIyMgYHF1YWtlc2AgZGF0YXNldAoKV2hhdCBkb2VzIGBxdWFrZXNgIGxvb2sgbGlrZT8KYGBgCgpgYGAKCmBgYAoKYGBgCgoKIyBQcm9jZXNzaW5nIGludG8gdGFibGVzCgpCZWZvcmUgd2Ugc3RhcnQgbGVhcm5pbmcgYW55dGhpbmcgYWJvdXQgb3VyIGRhdGEgYW5kIHJlc3VsdHMsIHdlIG5lZWQgdG8gcHJvY2VzcyBhbmQgb3JnYW5pc2UgdGhlIGRhdGEuCgojIyBBZGQgY29sdW1ucwoKSG93IGNhbiB5b3UgbWFrZSBhIG5ldyBjb2x1bW4/CmBgYHtyfQpzbGVlcCAlPiUKICBtdXRhdGUobmV3ID0gMTpsZW5ndGgoZ3JvdXApKQpgYGAKCkR1cGxpY2F0ZSBgZ3JvdXBgIGludG8gYGdyb3VwMmAgZm9yIGBzbGVlcGA6CmBgYAoKYGBgCgpDcmVhdGUgYSBjb2x1bW4gaW4gYHF1YWtlc2AgdGhhdCBjYWxjdWxhdGVzIHRoZSBgZGVwdGhgIG9mIHRoZSBxdWFrZSBkaXZpZGVkIGJ5IHRoZSBudW1iZXIgb2YgYHN0YXRpb25zYCByZXBvcnRpbmc6CmBgYAoKYGBgCgojIyMgQ2FzZSB3aGVuCgpUaWR5dmVyc2UgdHJpZXMgdG8gcmVkdWNlIHRoZSBuZWVkIGZvciAiZm9yIGxvb3BzIi4gSW5zdGVhZCBvZiBnb2luZyBsaW5lIGJ5IGxpbmUgdGhyb3VnaCBhIGRhdGFzZXQgdG8gZGV0ZXJtaW5lIHdoYXQgY29udGluZ2VudCBiZWhhdmlvdXIgdG8gcGVyZm9ybS4gVGhlIGZvci1sb29wIGJlaGF2aW91ciBpcyB0aW1lIGFuZCBlbmVyZ3kgaW50ZW5zaXZlIG9uIGxhcmdlIGRhdGFzZXRzLiBUaGF0J3Mgd2h5IGBjYXNlX3doZW5gIGlzIHNvIHBvd2VyZnVsLgoKSGVyZSdzIGFuIGV4YW1wbGUgb2YgaG93IG9uZSBtaWdodCBjcmVhdGUgYSBjb2x1bW4gdGhhdCB0cmFuc2xhdGVzIHRoZSBmYWN0b3IgYGdyb3VwYCBudW1iZXIgaW4gYHNsZWVwYCB0byBhIHdvcmQ6CmBgYHtyfQpzbGVlcCAlPiUgCiAgbXV0YXRlKGdyb3VwVGV4dCA9IGNhc2Vfd2hlbihncm91cD09IjEiIH4gIm9uZSIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBncm91cD09IjIiIH4gInR3byIpKQoKIyBvcgoKc2xlZXAgJT4lIAogIG11dGF0ZShncm91cFRleHQgPSBjYXNlX3doZW4oZ3JvdXA9PSIxIiB+ICJvbmUiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgVFJVRSB+ICJ0d28iKSkKYGBgCgoKTm93LCBob3cgd291bGQgeW91IGNyZWF0ZSBhIGNvbHVtbiBpbiBgcXVha2VzYCB0aGF0IGdyb3VwcyBtYWduaXR1ZGUgaW50byAibG93IiwgIm1lZGl1bSIgYW5kICJoaWdoIj8KYGBgCgpgYGAKCldoYXQncyB3cm9uZyB3aXRoIHRoaXMgb25lPwpgYGB7cn0Kc2xlZXAgJT4lCiAgbXV0YXRlKGdyb3VwMiA9IGNhc2Vfd2hlbihncm91cD09MSB+IGFzLmZhY3Rvcigib25lIiksCiAgICAgICAgICAgICAgICAgICAgICAgICAgICBncm91cD09MiB+IGFzLmZhY3RvcigidHdvIikpKQpgYGAKCkhvdyBjb3VsZCB3ZSBmaXggaXQ/CmBgYAoKYGBgCgpXZSBjYW4gYWxzbyB1c2UgdGhpcyB0byBwZXJmb3JtIG90aGVyIHNvcnRzIG9mIGNvbnRpbmdlbnQgY2FsY3VsYXRpb25zLgoKQ3JlYXRlIGEgY29sdW1uIHRoYXQgYWRkcyAxMCB0byBgbG9uZ2Agd2hlbiBpdCBpcyBhYm92ZSAxNzUgYW5kIHN1YnRyYWN0cyAxMCBmcm9tIGBsb25nIHdoZW4gaXQgaXMgYmVsb3cgMTc1OgpgYGAKCmBgYAoKIyMgRmlsdGVyCgpJZiB3ZSBvbmx5IHdhbnQgdG8gbG9vayBhdCBHcm91cCAyIGZyb20gYHNsZWVwYCwgd2UgY2FuIGZpbHRlciB0aGUgZGF0YXNldCAod2hpY2ggaXMgbGlrZSBzdWJzZXR0aW5nKToKYGBgCgpgYGAKClRoaXMgYWxzbyB3b3JrcyBmb3IgY29udGludW91cyBkYXRhOgpgYGAKCmBgYAoKCiMjIEdyb3VwIGFuZCBzdW1tYXJpc2UKCldoYXQgaWYgd2Ugd2FudCB0byBnZXQgYWdncmVnYXRlIHZhbHVlcyBmcm9tIG91ciBkYXRhc2V0LCByYXRoZXIgdGhhbiBsb29raW5nIGF0IGl0IGFzIGEgd2hvbGU/CgoqKmBncm91cF9ieWAqKiBpcyBhIHZlcmIgdGhhdCBmbGFncyBjZXJ0YWluIGNvbHVtbnMgZm9yIG9wZXJhdGlvbnMgZG93biB0aGUgbGluZS4gKipgc3VtbWFyaXNlYCoqIGNoZWNrcyB3aGljaCBjb2x1bW5zIGFyZSBmbGFnZ2VkIGFuZCBwZXJmb3JtcyBvcGVyYXRpb25zIGJhc2VkIG9uIHRoZSBwZXJtdWF0aW9ucyBvZiB2YWx1ZXMgaW4gdGhvc2UgY29sdW1ucy4KCldoYXQgaGFwcGVucyB3aGVuIHdlIHVzZSBgZ3JvdXBfYnlgIGJ5IGl0c2VsZj8KYGBgCgpgYGAKCkhvdyBtYW55IG9ic2VydmF0aW9ucyBhcmUgdGhlcmUgcGVyICJsZXZlbCIgb2YgbWFnbml0dWRlPwpgYGAKCmBgYAoKTm93LCBsZXQncyByZWNyZWF0ZSB0aGUgYGNvdW50YCBmdW5jdGlvbiB3aXRoIGBncm91cF9ieWAgYW5kIGBzdW1tYXJpc2VgIGZvciB0aGUgYHNsZWVwYCBkYXRhc2V0ICh3aGljaCBoYXMgY2F0ZWdvcmllcyk6CmBgYAoKYGBgCgpXZSBjYW4gdXNlIGBncm91cF9ieWAgYW5kIGBzdW1tYXJpc2VgIHRvIGRvIGEgbG90IG1vcmUgdGhhbiBqdXN0IGNvdW50OgpgYGAKIyBtZWFuIHZhbHVlIG9mIGBleHRyYWAgYnkgYGdyb3VwMmAKCmBgYAoKTGV0J3MgY3JlYXRlIGEgdGFibGUgb2YgdGhlIG1lYW5zLCBzdGFuZGFyZCBkZXZpYXRpb25zLCBhbmQgc3RhbmRhcmQgZXJyb3JzIGZvciBib3RoIHN0YXRpb25zIHJlcG9ydGluZyBhbmQgZGVwdGhzIGdyb3VwZWQgYnkgbWFnbml0dWRlOgpgYGB7cn0KcXVha2VzICU+JQogIGdyb3VwX2J5KOKAouKAouKAoikgJT4lCiAgc3VtbWFyaXNlKG4gPSDigKLigKLigKIsCiAgICAgICAgICAgIHN0YXRpb25NZWFuID0g4oCi4oCi4oCiLAogICAgICAgICAgICBzdGF0aW9uU0QgPSDigKLigKLigKIsCiAgICAgICAgICAgIHN0YXRpb25TRSA9IOKAouKAouKAoiwKICAgICAgICAgICAgZGVwdGhNZWFuID0g4oCi4oCi4oCiLAogICAgICAgICAgICBkZXB0aFNEID0g4oCi4oCi4oCiLAogICAgICAgICAgICBkZXB0aFNFID0g4oCi4oCi4oCiKQpgYGAKCiMjIFVuaXRlIGFuZCBzZXBhcmF0ZSAodGV4dCkKCkZpcnN0LCBsZXQncyBjcmVhdGUgc29tZSBjb2x1bW5zIHdpdGggY2hhcmFjdGVyIHZhbHVlczoKYGBgCgpgYGAKCkNvbWJpbmUgKHVzaW5nIGB1bml0ZWApIHRoZSBjb2x1bW5zIGBncm91cFRleHRgIGFuZCBgY2F0ZWdvcnlgLgpgYGAKCmBgYAoKVGhlIHJldmVyc2UgcHJvY2VzcyBpcyBjYWxsZWQgYHNlcGFyYXRlYDoKYGBgCgpgYGAKCllvdSBjYW4gZG8gdGhpcyB3aXRoIGFueSBjaGFyYWN0ZXIuIFdoYXQgaGFwcGVucyB3aGVuIHlvdSB1c2UgYGlgPwpgYGAKCmBgYAoKCiMjIEJpbmQgYW5kIEpvaW4KCldoYXQgaWYgeW91IGhhdmUgdHdvIGRhdGFzZXRzIChvYnNlcnZhdGlvbmFsIGRhdGEgYW5kIGRlbW9ncmFwaGljIGRhdGEpIGFuZCB5b3Ugd2FudCB0byBjb21iaW5lIHRoZW0/CgpGaXJzdCwgd2UnbGwgc3BsaXQgYHNsZWVwYCBpbnRvIHR3byBkYXRhc2V0czoKYGBgCgpgYGAKCkxldCdzIGxvb2sgYXQgdGhlIHR3byBkYXRhc2V0czoKYGBgCgpgYGAKCklmIHdlIHdhbnQgdG8gcHV0IHRoZW0gYmFjayB0b2dldGhlciBhcyB0aGV5IHdlcmUgKG9uZSBjb2x1bW4gZm9yIGJvdGggZ3JvdXBzKSwgd2UgY2FuIGJpbmQgYnkgcm93OgpgYGAKCmBgYAoKSWYgd2Ugd2FudCB0byBiaW5kIHRoZSB0d28gc3Vic2V0cyBvZiBgc2xlZXBgIGludG8gYSAid2lkZSIgZGF0YXNldCwgd2UgY2FuIHVzZSBhIHNpbWlsYXIgZnVuY3Rpb24gdG8gcGFzdGUgdGhlIHR3byBkYXRhc2V0cyB0b2dldGhlcjoKYGBgCgpgYGAKCkFuZCBpbiBhIG1vcmUgdGlkeSBmb3JtYXQ6CmBgYAoKYGBgCgoKQnV0IHRoaXMgaXMgc29tZXdoYXQgY29hcnNlLiBUaGUgZnVuY3Rpb24gYGZ1bGxfam9pbmAgYWxsb3dzIGZvciBiaW5kaW5nIGJ5IGNvbHVtbnMgYW5kIHJvd3MgaW4gYSBtdWNoIHNtb290aGVyLCBzbGVla2VyIHdheS4KCkJpbmQgYHNsZWVwMWAgYW5kIGBzbGVlcDJgIGJ5IHJvd3MgdXNpbmcgYGZ1bGxfam9pbmA6CmBgYAoKYGBgCgpCaW5kIGBzbGVlcDFgIGFuZCBgc2xlZXAyYCBieSB0aGUgYElEYCBjb2x1bW4gKHNvIHRoYXQgYGV4dHJhYCBhbmQgYGdyb3VwYCBhcmUga2VwdCBzZXBhcmF0ZSk6CmBgYAoKYGBgCgpXaGF0IGhhcHBlbnMgaWYgeW91IHRyeSB0aGlzIHdpdGggam9pbmluZyBieSBgZ3JvdXBgPyBXaHk/CmBgYAoKYGBgCgoKQnV0LCB0aGUgZGlmZmVyZW50IGZvcm1zIG9mIGBqb2luYCBhcmUgbmFtZWQgaW4gYSB3YXkgdGhhdCBvbmx5IHJlYWxseSBtYWtlcyBzZW5zZSBpZiB5b3Uga25vdyBTUUwuIEZvciB0aGUgcmVzdCBvZiB1cywgdGhlcmUncyBhIFtjaGVhdCBzaGVldF0oaHR0cHM6Ly9zdGF0NTQ1LmNvbS9iaXQwMDFfZHBseXItY2hlYXRzaGVldC5odG1sKS4KCiMjIEdhdGhlciBhbmQgc3ByZWFkCgpUaGlzIHNlY3Rpb24gd2lsbCAoaG9wZWZ1bGx5KSBiZSBkZXByaWNhdGVkIHNvb24gZm9yIG11Y2ggbW9yZSBpbnR1aXRpdmUgZnVuY3Rpb25zIGNhbGxlZCBgcGl2b3RfbG9uZ2VyYCBhbmQgYHBpdm90X3dpZGVyYC4gQnV0IGZvciBub3csIHdlJ2xsIGxlYXJuIHRoZSBvbmVzIHRoYXQgYXJlIGN1cnJlbnRseSBhdmFpbGFibGUuCgpXaGF0IGlzIGEgd2lkZSBkYXRhc2V0PwpgYGAKCmBgYAoKCkxldCdzIG1ha2UgaXQgbG9uZyB1c2luZyBgZ2F0aGVyYCwgZm9jdXNpbmcgb24gY29sdW1ucyAzIHRocm91Z2ggOC4gSG93IGRvZXMgdGhpcyBkaWZmZXI/OgpgYGAKCmBgYAoKSWYgdGhpcyB3ZXJlIG91ciBvcmlnaW5hbCBkYXRhc2V0IGFuZCB3ZSB3YW50ZWQgdG8gbWFrZSBpdCB3aWRlLCB3ZSBjb3VsZCB1c2UgYHNwcmVhZGA6CmBgYAoKYGBgCgpIb3cgY291bGQgeW91IHVzZSBgc3ByZWFkYCB0byBzb3J0IG9mIHJlY3JlYXRlIG91ciB3aWRlIGBzbGVlcGAgZGF0YXNldD8KYGBgCgpgYGAKCiMgVHJ5IGl0IG91dAoKUmVhZCBpbiBgbG9uZy1kYXRhLmNzdmA6CmBgYHtyfQpsb25nX2RhdGEgPC0gcmVhZC5jc3Yo4oCi4oCi4oCiKQpgYGAKCk1ha2UgaXQgd2lkZSBpbiB0aGUgd2F5IHlvdSBjaG9vc2UuIFRoaW5rIGFib3V0IHRoZSBzdHJ1Y3R1cmUgb2YgdGhlIGRhdGEgYW5kIHdoYXQgeW91IG1pZ2h0IHdhbnQgdG8gZG8gd2l0aCBpdC4KYGBgCgpgYGAKClJlYWQgaW4gYHdpZGUtZGF0YS5jc3ZgCmBgYAoKYGBgCgpNYWtlIGl0IGxvbmcgaW4gdGhlIHdheSB5b3UgY2hvb3NlLiBUcnkgZGlmZmVyZW50IG1ldGhvZHMgdG8gc2VlIHdoYXQgdGhleSBkby4gS2VlcCByZWNvcmRzIG9mIGV2ZXJ5dGhpbmcgeW91IHRyeSBieSB0YWtpbmcgYWR2YW50YWdlIG9mIGxpdGVyYXRlIHByb2dyYW1taW5nLgpgYGAKCmBgYAoKIyMgTkFzCgpTZWUgaG93IG9uZSBvZiB0aGUgY2VsbHMgaXMgYE5BYD8gVGhhdCdzIGZpbmUsIGJ1dCB3aGF0IGlmIHdlIHdhbnQgdG8gYWRkIGEgdmFsdWUgaW4/IE5BcyBhcmUgYSBzdHJhbmdlIGNhdGVnb3J5IGFuZCBSIHdpbGwgdGhyb3cgZXJyb3JzIGlmIGl0IGRvZXNuJ3QgbGlrZSB0aGUgd2F5IHlvdSdyZSBsb29raW5nIGZvciB0aGVtLiBVc2UgYGlzLm5hYCB0byBnZXQgYSBib29sZWFuIChUUlVFL0ZBTFNFKSB2YWx1ZSB0byBpc29sYXRlIGNlbGxzIHdpdGggTkFzLgpgYGAKCmBgYAoKTGV0J3MgZmluZCBhIHdheSB0byBwdXQgdGhlIHZhbHVlICdub25lJyBpbiB0aGF0IGNlbGwgdXNpbmcgYG11dGF0ZWAsIGBjYXNlX3doZW5gLCBhbmQgYGlzLm5hYC4KYGBgCgpgYGAKCk5vdyBkbyB0aGF0IHdpdGhvdXQgZ2V0dGluZyByaWQgb2Ygb3RoZXIgaW5mb3JtYXRpb24uIEhpbnQ6IGZhY3RvciB2ZWN0b3JzIGFyZSBoYXJkZXIgdG8gZWRpdCB0aGFuIGNoYXJhY3RlciB2ZWN0b3JzIQpgYGAKCmBgYAoKCiMjIEdyb3VwIGNoYWxsZW5nZSEKCkNhbiB5b3UgY29tYmluZSB0aGUgd2lkZSBkYXRhIGFuZCBsb25nIGRhdGEgaW50byBhIHNpbmdsZSBkYXRhIGZyYW1lIHVzaW5nIHRoZSBzdWJqZWN0cycgYWdlcyBhcyB0aGUgY29tbW9uIGNvbHVtbj8gKFRoZXJlIHdpbGwgYmUgTkFzLCBpZ25vcmUgb3IgcmVtb3ZlIGBOYXRpb25hbGl0eWAgZm9yIG5vdy4pCmBgYHtyfQpsb25nX2RhdGEgJT4lIAogIHNwcmVhZCjigKLigKLigKIpICU+JSAKICBtdXRhdGUo4oCi4oCi4oCiKSAlPiUgCiAgc2VsZWN0KOKAouKAouKAoikgJT4lIAogIGZ1bGxfam9pbijigKLigKLigKIpCmBgYAoKVGFrZSB0aGlzIGRhdGFzZXQgYW5kIHNwbGl0IGBTYXZpbmdzYCBpbnRvIGB2YWx1ZWAgYW5kIGBjdXJyZW5jeWAuIE1ha2Ugc3VyZSBudW1iZXJzIGFyZSBudW1iZXIgYW5kIGxldHRlcnMgYXJlIGNoYXJhY3RlcnMuCmBgYAoKYGBgCgoKRmluZCBhIHdheSB0byBmaWxsIGluIHRoZSBOQXMgaW4gdGhpcyBkYXRhc2V0IHdpdGggdW5pcXVlIHZhbHVlcy4gSWYgcG9zc2libGUsIGRvIHRoaXMgd2l0aGluIHRoZSBwaXBpbmcgZW52aXJvbm1lbnQgd2l0aG91dCBzYXZpbmcgdGhlIGRhdGFzZXQgYXMgYW4gb2JqZWN0LgpgYGAKCmBgYAoKCiMgQ2FsY3VsYXRpb25zIG9uIHRhYmxlcwoKRm9yIHRoaXMsIHdlJ2xsIHVzZSBgYmlub21pYWwtZGF0YS5jc3ZgLgpgYGB7cn0KZGF0YSA8LSByZWFkLmNzdigiLi4vZGF0YS9iaW5vbWlhbC1kYXRhLmNzdiIpCmBgYAoKUGVyc29uYWxseSwgSSBwcmVmZXIgdG8gbm90IHNhdmUgZGF0YSBpbnRvIG5ldyB2YXJpYWJsZXMgaWYgSSBjYW4gYXZvaWQgaXQuIEhvd2V2ZXIsIHRoaXMgbWFrZXMgZG9pbmcgc3RhdGlzdGljYWwgYW5hbHlzZXMgbW9yZSBjb21saWNhdGVkLiBXZSdsbCB0YWxrIG1vcmUgYWJvdXQgdGhpcyBvbiBUaHVyc2RheSwgYnV0IGZvciBub3cgaGVyZSBhcmUgc29tZSBuaWNlIHRyaWNrcyB0aGF0IHdpbGwgYmUgZ29vZCB0byBrbm93IGdvaW5nIGZvcndhcmQuCgojIyBQdWxsIGFuZCBzZWxlY3QKClNvbWUgZnVuY3Rpb25zIGluIHRpZHl2ZXJzZSBhcmUgbm90IGdvb2QgYXQgaXNvbGF0aW5nIHNpbmdsZSBjb2x1bW5zIGZyb20gYSBkYXRhc2V0IGJ1dCBhcmUgc3RpbGwgc2Vuc2l0aXZlIHRvIGBncm91cF9ieWAgZmxhZ3MuIElmIHlvdSB3YW50IHRvIGlzb2xhdGUgYSBzaW5nbGUgY29sdW1uLCBgcHVsbGAgZG9lcyB0aGUgdHJpY2suCgpgcHVsbGAgdGhlIGNvbHVtbiBgZXh0cmFgIGZyb20gYHNsZWVwYDoKYGBgCgpgYGAKCmBzZWxlY3RgIGRvZXMgdGhlIHNhbWUgdGhpbmcsIGJ1dCB5b3UgY2FuIHNlbGVjdCBtb3JlIHRoYW4gb25lIGNvbHVtbiwgb3Igc3BlY2lmeSBhIGNvbHVtbiB0byByZW1vdmUuIAoKVGhlcmUgYXJlIHNvbWUgb3RoZXIgZGlmZmVyZW5jZXMsIGV2ZW4gd2hlbiBzZWxlY3Rpbmcgb25lIGNvbHVtbi4gQ2FuIHlvdSB0ZWxsPwpgYGAKIyBqdXN0IGBleHRyYWAKYGBgCgpgYGAKIyBldmVyeXRoaW5nIGV4Y2VwdCBgZXh0cmFgCmBgYAoKYGBgCiMgYElEYCBhbmQgYGV4dHJhYApgYGAKCiMjIyBUcnkgaXQgb3V0CgpDcmVhdGUgYSBzdWJzZXQgb2YgYHF1YWtlc2AgdGhhdCBpbmNsdWRlcyBvbmx5IGxhdGl0dWRlIGFuZCBsb25naXR1ZGUgb2YgcXVha2VzIHdpdGggYSBtYXhpbXVtIG1hZ25pdHVkZSBvZiA1IGFuZCBubyBmZXdlciB0aGFuIDMwIHN0YXRpb25zIHJlcG9ydGluZy4KYGBge3J9CnF1YWtlcyAlPiUgCiAgZmlsdGVyKOKAouKAouKAoikgJT4lIAogIHNlbGVjdCjigKLigKLigKIpCmBgYAoKVXNpbmcgdGhpcyBzdWJzZXQsIHN1bW1hcmlzZSB0aGUgZGF0YSB0byBjb3VudCB0aGUgbnVtYmVyIG9mIHF1YWtlcyB0aGF0IG9jY3VyIHRvIHRoZSBlYXN0IGFuZCB3ZXN0IG9mIDE4MMuaIGxvbmdpdHVkZS4KYGBge3J9CnF1YWtlcyAlPiUgCiAgZmlsdGVyKOKAouKAouKAoikgJT4lIAogIHNlbGVjdCjigKLigKLigKIpICU+JSAKICBtdXRhdGUo4oCi4oCi4oCiKSAlPiUgCiAgZ3JvdXBfYnko4oCi4oCi4oCiKSAlPiUgCiAgc3VtbWFyaXNlKOKAouKAouKAoikKYGBgCgojIyBEbyAoZm9yIG5vdykKClRoZSBmdW5jdGlvbiBgZG9gIGlzIGFwcGFyZW50bHkgb24gaXRzIHdheSBvdXQsIHRvIGJlIHJlcGxhY2VkIGJ5IGBtYXBgIChpbiBgcHVycnJgKSwgYnV0IGZvciBub3cgd2UnbGwgdGFsayBhYm91dCBgZG9gIGFuZCB5b3UgY2FuIHVzZSB5b3VyIG5ld2ZvdW5kIHNraWxscyB0byB0ZWFjaCB5b3Vyc2VsZiBgbWFwYCB3aGVuIGl0IGJlY29tZXMgYXZhaWxhYmxlIQoKYGRvYCBsaXRlcmFsbHkganVzdCBtZWFucyAiZG8gdGhlIG9wZXJhdGlvbiBJJ20gdGVsbGluZyB5b3UgdG8gZG8gb24gc29tZSBkYXRhc2V0Iiwgd2hpY2ggaXMgc3VwZXJmaWNpYWxseSBub3QgdXNlZnVsLgpgYGB7cn0KcXVha2VzICU+JQogIGRvKGhlYWQoLikpICMgLiBtZWFucyAndGhlIGRhdGFzZXQgd2UndmUgYmVlbiBwaXBpbmcgdGhyb3VnaCB0aGlzIGNodW5rJwpgYGAKCkhvd2V2ZXIsIG5vdGljZSBob3cgd2UgKm5lZWQqIHRvIGhhdmUgc29tZXRoaW5nIHdpdGhpbiB0aGUgYnJhY2VzIGZvciBgaGVhZGAgaGVyZSwgd2hlbiB3ZSB3b3VsZG4ndCBoYXZlIG5lZWRlZCBpdCB3aXRob3V0IGBkb2AuCmBgYAoKYGBgCgpUaGlzIGlzIHRoZSBpbXBvcnRhbnQgcGFydCBvZiBgZG9gOiBpdCBhbGxvd3MgdXMgdG8gc3BlY2lmeSB3aGljaCBkYXRhc2V0IHdlIHdhbnQgdG8gKmRvKiBzb21ldGhpbmcgdG8sIGV2ZW4gd2l0aGluIGEgcGlwaW5nIGVudmlyb25tZW50LiBUaGF0IGlzLCB3ZSBjYW4gbmVzdCBvcGVyYXRpb25zIG9mIG1vcmUgdGhhbiBvbmUgZGF0YXNldCB3aXRoaW4gYSBjaHVuayBieSB1c2luZyBgZG9gLiAKCiMjIGBicm9vbWAKCmBgYHtyfQpsaWJyYXJ5KGJyb29tKQpgYGAKCkknbGwgaW50cm9kdWNlIHRoZSBwYWNrYWdlIGBicm9vbWAgbm93LCBidXQgd2UnbGwgY29tZSBiYWNrIHRvIGl0IG9uIFRodXJzZGF5LiBSaWdodCBub3csIHdlIHdpbGwgb25seSB1c2UgaXQgZm9yIHRoZSBmdW5jdGlvbiBgdGlkeWAsIHdoaWNoIHR1cm5zIHRoZSBvdXRwdXQgb2YgYSBmdW5jdGlvbiB0byBhIHRpZHkgdGliYmxlIGlmIHBvc3NpYmxlLgoKVXNpbmcgYmFzZSBSLCBgY29yLnRlc3RgIHByb3ZpZGVzIHRoZSByZXN1bHRzIG9mIGEgdGVzdCBmb3IgY29ycmVsYXRpb24gYmV0d2VlbiBwYWlyZWQgc2FtcGxlcywgZGVmYXVsdGluZyB0byBQZWFyc29uJ3MgcHJvZHVjdCBtb21lbnQuIEl0IHRha2VzIHR3byBhcmd1bWVudHMgKGVhY2ggaXMgb25lIG9mIHRoZSBwYWlyZWQgc2FtcGxlcykuCmBgYAoKYGBgCgpIZXJlJ3MgaG93IHdlIGNhbiB1c2UgYGRvYCBhbmQgYHRpZHlgIHRvIHByb2R1Y2UgYW4gb3V0cHV0IHRoYXQgaXMgZWFzaWVyIHRvIGZvcm1hdCwgdGh1cyBlYXNpZXIgdG8gdXNlIGluIGxpdGVyYXRlIHByb2dyYW1taW5nLgpgYGAKCmBgYAoKSWYgd2Ugc2F2ZSB0aGUgdGlkaWVkIG91dHB1dCAoanVzdCB0aGlzIG9uY2XigKYpLCB3ZSBjYW4gcHV0IGl0IGRpcmVjdGx5IGludG8gdGhlIHRleHQgb2YgdGhlIC5SbWQgZmlsZS4KYGBgCgpgYGAKClNlZT8gCgo+IFRoZSAkXGJldGEkIHZhbHVlIG9mIHRoaXMgY29ycmVsYXRpb24gaXMgYHIgcm91bmQodGlkeS5jb3IkZXN0aW1hdGUsMylgIHVzaW5nIHRoZSBgciB0aWR5LmNvciRtZXRob2RgIG1ldGhvZCBvZiBhbmFseXNpcy4=