---
title: "ADA1: Class 13, Inference"
author: anonymous
date: "10/04/2016"
output:
html_document:
toc: true
---
Include your answers in this document in the sections below the rubric.
# Rubric
Answer the questions with the two data examples.
---
# Guess the ages example
## Read and reshape the data to produce summary table and plot
Erik has written a lot of code here to reshape, summarize, and combine the data
in order to create the table and plot.
You are encouraged to look at the code, run each line at a time and examine the results
to understand the steps.
The steps used are a common set of transformations for basic analysis.
```{R, cache = TRUE, echo = FALSE}
# install.packages("gsheet")
# Read Ages data from google spreadsheet
library(gsheet)
dat.ages.url <- "docs.google.com/spreadsheets/d/1lhkApsOF1UKYZVXCK3FzxMVUGk9YucyWeeazI1lxN_Y"
# convert the spreadsheet to csv-formatted test
dat.ages.all <- gsheet2text(dat.ages.url)
```
```{R, echo = FALSE}
# process data, reshape and organize
# read body of table as data (first two rows have other information
dat.ages <- read.table(text=dat.ages.all, skip = 2, header = TRUE, sep = ",", stringsAsFactors = TRUE)
# convert gender letters to capital
dat.ages$GenderOfGuesser <- toupper(dat.ages$GenderOfGuesser)
# extract top two rows for true ages and genders
dat.ages.true <- read.table(text=dat.ages.all, nrows = 2, header = FALSE, sep = ",", stringsAsFactors = FALSE)
colnames(dat.ages.true) <- c("Image", colnames(dat.ages)[-1])
#str(dat.ages.true)
# guessed ages
#str(dat.ages)
# reshape into long format
library(reshape2)
dat.ages.true.long <-
data.frame(Image = as.character(colnames(dat.ages.true[,-1]))
, Age = as.numeric(dat.ages.true[1,-1])
, Gender = as.character(dat.ages.true[2,-1]))
dat.ages.long <- melt(dat.ages
, id.vars = colnames(dat.ages)[1]
, variable.name = "Image"
, value.name = "Age")
# order images by age
dat.ages.true.long$Image <- factor(dat.ages.true.long$Image
, ordered = TRUE
, levels = levels(reorder(dat.ages.true.long$Image, dat.ages.true.long$Age))
)
dat.ages.long$Image <- factor(dat.ages.long$Image
, ordered = TRUE
, levels = levels(reorder(dat.ages.true.long$Image, dat.ages.true.long$Age))
)
# join datasets to include image Gender with guesses
library(plyr)
dat.ages.long <- join(dat.ages.long
, subset(dat.ages.true.long, select = -Age)
, by = "Image")
# calcuate mean, sd, and CI from data for each image
library(plyr)
## .X <- subset(dat.ages.long, Image == "X1")
dat.ages.est.long <- ddply(dat.ages.long
, "Image"
, function (.X) {
t.result <- t.test(.X$Age, conf.level = 0.95)
sd.age <- sd(.X$Age)
out <- data.frame(Age = t.result$estimate
, Gender = .X[1,"Gender"]
, SD.Age = sd.age
, CI.lower = t.result$conf.int[1]
, CI.upper = t.result$conf.int[2]
)
return(out)
}
)
#$
# To create a table, create a "TrueAge" column, and join the true with est table
dat.ages.true.long.temp <- dat.ages.true.long
colnames(dat.ages.true.long.temp)[which(colnames(dat.ages.true.long.temp) == "Age")] <- "TrueAge"
library(plyr)
dat.ages.est.long <- join(dat.ages.est.long
, dat.ages.true.long.temp
)
dat.ages.est.long$Bias <- dat.ages.est.long$Age - dat.ages.est.long$TrueAge
#$
# sort by Gender and TrueAge so in same order as plot
dat.ages.est.long <- arrange(dat.ages.est.long, Gender, TrueAge)
# reorder columns
dat.ages.est.long <- dat.ages.est.long[, c("Image", "Gender", "TrueAge", "Age"
, "Bias", "SD.Age", "CI.lower", "CI.upper")]
```
Below is a summary table for each image, and the associated plot.
We will display the original images in class for comparisons.
```{R}
# finally, display beautiful table
dat.ages.est.long
```
```{R, fig.width = 7, fig.height = 7}
# plot data and create table
library(ggplot2)
p <- ggplot(dat.ages.long, aes(x = Age))
p <- p + geom_histogram(aes(fill = GenderOfGuesser), position="stack", binwidth = 2, alpha = 1)
p <- p + geom_rug(alpha = 1/8)
# true ages
p <- p + geom_vline(data = dat.ages.true.long, aes(xintercept = Age)
, colour = "red", linetype = "dotted", size = 1)
# est ages
p <- p + geom_vline(data = dat.ages.est.long, aes(xintercept = Age)
, colour = "blue", size = 1)
p <- p + geom_rect(data = dat.ages.est.long, aes(xmin = CI.lower, xmax = CI.upper, ymin = -1, ymax = 0)
, fill = "blue", alpha = 1)
p <- p + facet_grid(Gender + Image ~ ., space = "free")
# Legend: Put top-right corner of legend box in top -right corner of graph
p <- p + theme(legend.justification=c(1,1), legend.position=c(1,1))
p <- p + labs(title = "Age guesses\nred dotted = true, blue = est +- 95% CI")
print(p)
```
## Questions to answer
1. (1 p) From the table or plot,
what is the overall pattern of age guesses based on the Gender and Age of the Image
based on the Gender of the guesser?
2. (1 p) The sample who took the Guess the Ages survey were
"Self-selected undergraduate and graduate students in
Stat 427/527 this semester".
Define the population this sample was taken from.
3. (1 p) The population parameter being tested is
"The mean age $\mu_j$ that people in the population would assess for each image $j=1, \ldots, 10$."
(Note, the population parameter is _not_ the true age, rather, it is the _mean assessed age_.
The bias difference of these two is an indicator of whether a person looks young or old for their age.)
The sample statistic is $\bar{Y}_j$, the sample mean for each image.
Give the name for and interpret the standard deviation for this sample statistic.
4. (2 p) Report and interpret the confidence interval for image X1.
---
# Lego example
## Draw samples, estimate the mean "cells" of 50 Lego assemblages
A "cell" is defined as a one-unit high square that includes a single lego circle.
Some assemblages have a 1/5-high base; ignore this, it is only for structure.
Procedure:
a. Select a "representative or random sample" of 5 assemblages out of the bag.
b. Count the number of cells for each assemblage.
c. Calculate the sample mean of the 5 (to estimate population mean of the 50); you can use this R code by replacing the numbers: `mean(c(1, 2, 3, 4, 5))`.
d. Write your mean cell-count estimate on a portable white board in big numbers (3 decimal places) and lay it on your table (so other tables don't see).
e. When all tables are done, hang up your board so everyone can see each table's cell-count estimate.
f. Record all the estimates in the R code chunk below and plot the estimate of the sampling distribution of the mean cell count based on $n=5$.
```{R}
# enter list of cells means here
#sam.lego <- data.frame(mean.cells = c(0, 0, 0, 0, 0, 0
# , 0, 0, 0, 0, 0, 0))
sam.lego <- data.frame(mean.cells =
c(33.2, 32.4, 29.2, 19.6, 18.4
, 32.4, 21.6, 41.2, 48, 53.6
, 58.8, 35.2))
# we'll fill this in after data is collected
#true.mean.cells = 0
true.mean.cells = 11.86
summary(sam.lego$mean.cells)
```
```{R}
# plot data and create table
library(ggplot2)
p <- ggplot(sam.lego, aes(x = mean.cells))
p <- p + geom_histogram(binwidth = 5)
p <- p + geom_rug(alpha = 1/2)
# est mean cells
p <- p + geom_vline(aes(xintercept = mean(sam.lego$mean.cells))
, colour = "blue", size = 1)
# true mean cells
p <- p + geom_vline(aes(xintercept = true.mean.cells)
, colour = "red", linetype = "dotted", size = 1)
p <- p + labs(title = "Sampling distribution of mean cells of 50 assemblages\nn=5, red dotted = true, blue = est")
print(p)
```
## Questions to answer
5. (1 p) Adjust the histogram `binwidth=` to provide an informative representation of the distribution.
6. (1 p) Describe the sampling distribution: mention the center, spread, and any outliers in the plot.
7. (1 p) Are the estimates larger or smaller than the actual mean cells of the assemblages,
and what might cause this bias?
8. (1 p) What would happen to the standard error and the bias if we increased the sample size from 5 to 10 or 20?
For example, write: "As the sample size increases, we expect the standard error to ... and the bias to ...".
9. (1 p) Does the Central Limit Theorem apply to the lego example? Why or why not?