Overview

StreetEasy (owned by Zillow) is well-known to anyone who’s been NYC apartment-hunting in the past 10 years and even more-so for folks searching in the past 5 years. They’ve posted close to half a million rental listings on an annual basis since 2015.

Contents

After downloading the All.zip file, unzipping, and then unzipping the included files discountShare_All.zip, medianAskingRent_All.zip, and rentalInventory_all.zip, combining and converting into tidy format in a tibble called df, the data will look like this:

glimpse(df)

## Rows: 76,032
## Columns: 6
## $ areaName <chr> "All Downtown", "All Downtown", "All Downtown", "All Midtow~
## $ Borough  <chr> "Manhattan", "Manhattan", "Manhattan", "Manhattan", "Manhat~
## $ areaType <chr> "submarket", "submarket", "submarket", "submarket", "submar~
## $ setname  <chr> "Discount Share", "Median Asking Rent", "Rental Inventory",~
## $ yyyymm   <date> 2010-01-01, 2010-01-01, 2010-01-01, 2010-01-01, 2010-01-01~
## $ value    <dbl> 0.114, 3200.000, 4268.000, 0.106, 2875.000, 3065.000, 0.135~

The data contains several features and a time-frame starting in 2010 but today’s post is going to focus on a subset of the data, plot_dat, with the follow criteria:

1. areaType == 'neighborhood': Just the neighborhood names (to exclude borough-level counts)
2. yyyymm >= 2018: Listings later than and including 2018 through August 2020
3. setname %in% c("Rental Inventory", "Median Asking Rent", "Discount Share")

Before making any assumptions about the following plots, please review the data consideratons under the reference section at the end of this post.

Rental Inventory by Month

The number of rental listings available on StreetEasy across all neighborhoods, monthly since 2018, by NYC borough.

rental_plot <- plot_dat %>% 
  filter(setname == "Rental Inventory") %>% 
  ggplot(aes(x = yyyymm, y = value, 
             group = yyyymm))+
  geom_boxplot(aes(color=Borough), 
               alpha = .4, 
               outlier.color = "gray",
               outlier.alpha = .50, 
               outlier.size = .5)+
  geom_vline(xintercept = as.Date("2020-03-17"),
             linetype = 3, 
             alpha = .8, 
             color = "blue",
             size = .75) +
  scale_y_log10(labels=scales::label_comma(accuracy = 1))+
  facet_wrap(~Borough, scales = "free_y")+
  my_theme

rental_plot

Each box plot represents a month and the value range is across all neighborhood listings for that month and the vertical, blue-dotted line indicates the beginning of NYC’s official shutdown, March 17th, 2020.

In each borough, there appears to be an increase in the median number of listings posted. Both Brooklyn and Manhattan neighborhoods appear to be flattening but Queens neighborhoods are still showing growth as of the data available at posting, through August 2020.

Median Asking Rent

The plot code remains the same except I filter setname on "Median Asking Rent", and I focus on values strictly greater than $50 per month. Further, I introduce plotly’s ggplotly function and convert the median rent plot, medrent_plot, into an interactive plot:

ggplotly(medrent_plot, dynamicTicks = TRUE) 

Median rents seem to be dropping in Brooklyn, Manhattan, and Queens neighborhoods. I am less clear on the impact to the Bronx.

On a technical note, being able to zoom-in on the plot with plotly to see the actual values extends and improves the usability of this plot. I wish the formatting was just a little closer to standard ggplot, however.

Discount Share

The share of all active rental listings on StreetEasy that had a reduction in asking rent during that month across all neighborhoods. I filter like previously except I set setname on "Discount Share", and I focus on values strictly greater than 0.1% per month:

The rise in Discount Share after the shutdown is most-prominent in Manhattan and Brooklyn neighborhoods - this aligns with my previous observations in the Inventory and Median Asking Rent plots. I was bit surprised about how tightly packed the middle 50% of the Manhattan values remained month-to-month prior to the shutdown, too.

On a technical note, for pure exploration, like when I’m first looking into a new data set, I prefer the standard, out-of-the-box ggplot plots over the output from ggplotly.

Prep Data

First I create a data set called ddf that is filtered as described in the Contents and add hover-over label column called text:

ddf <- df %>% 
  filter(areaType == "neighborhood", value > 0, yyyy >= 2018) %>% 
  mutate(value_preso = ifelse(
    grepl("disc", tolower(setname)), scales::percent(round(value, 4)),
    ifelse(grepl("media", tolower(setname)), 
           glue("${scales::comma(value)}"), 
           ifelse(grepl("inven", tolower(setname)), 
                  scales::comma(value), ""))
    ),
    text = glue("{setname},<br><b>{areaName}</b>:
                <b>{value_preso}</b>
                {as.character(yyyy)}-{as.character(mm)}"),
    setname = as.factor(setname)) 

ddf$setname <- fct_relevel(ddf$setname, 
                           levels = c("Rental Inventory", 
                                      "Median Asking Rent", 
                                      "Discount Share"))

glimpse(ddf)

## Rows: 12,777
## Columns: 10
## $ areaName    <chr> "Astoria", "Astoria", "Astoria", "Auburndale", "Auburnda~
## $ Borough     <chr> "Queens", "Queens", "Queens", "Queens", "Queens", "Brook~
## $ areaType    <chr> "neighborhood", "neighborhood", "neighborhood", "neighbo~
## $ setname     <fct> Discount Share, Median Asking Rent, Rental Inventory, Me~
## $ yyyymm      <date> 2018-01-01, 2018-01-01, 2018-01-01, 2018-01-01, 2018-01~
## $ value       <dbl> 0.197, 2150.000, 1560.000, 2300.000, 17.000, 0.088, 1875~
## $ yyyy        <chr> "2018", "2018", "2018", "2018", "2018", "2018", "2018", ~
## $ mm          <dbl> 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,~
## $ value_preso <chr> "19.700%", "$2,150.00000", "1,560.00000", "$2,300.00000"~
## $ text        <glue> "Discount Share,<br><b>Astoria</b>:\n<b>19.700%</b>\n20~

Create highlight_key

Next I convert the ddf into a highlight_key that will then be transformed into a shared data object. See the comments in the code chunk below for additional context:

ddf_plot <- ddf %>% 
  # set shared data key to `areaName`, ask user to select
  # note the open brace after the pipe: 
  highlight_key(~areaName, "Select a Neighborhood")  %>% {
  # the facet_wrap is baked inside the highlight_key: 
  ggplot(., aes(x = yyyymm, y = value, group = areaName, 
                color = Borough, text = text))+
  geom_line(alpha = .5)+
  facet_wrap(~setname, scales = "free_y", nrow = 3)+
  theme(legend.position = "none",
        axis.title.x = element_blank(),
        axis.title.y = element_blank())
  } %>% 
  # convert the ggplot into a ggplotly object &
  # add in the crosstalk element `hightlight()`
  ggplotly(dynamicTicks = T, tooltip = "text") %>% 
  highlight(dynamic = TRUE, 
            selectize = TRUE, 
            selected = attrs_selected(opacity = .8, 
                                      size = 3)) %>% 
  layout(showlegend = FALSE) 

Hold the SHIFT key to select more than one neighborhood at a time (trace or dropdown) and also note the ability to switch brush colors between selections:

ddf_plot

Using crosstalk is relatively straight-forward if you’re already familiar with plotly but it can seem finicky to plotly newbs. My reccommendation is to get a solid foundation with plotly and perhaps try working with shiny to understand the shared data object before diving into crosstalk.

Links and Sources

• crosstalk
• StreetEasy’s All Rental Data
• NYC Neighborhood Map via mapcruzin.com
• ggplot
• Interactive web-based data visualization with R, plotly, and shiny

Data Considerations

While StreetEasy’s data is probably a decent source of information on the current state of NYC’s apartment rental market - it is popular, regularly refreshed, and contains thousands of listings - it does come with limitations that mean it cannot tell the whole story. Here are some points of consideration:

• Staten Island is not included in StreetEasy listings.
• Bronx listings appear underrepresented in the data across all years (since 2010) with rarely over a 1,000 listings in a month. Whether this is a function of unpopularity of the site to Bronx landlords, lower economic mobility of residents, high-demand, or a combination of several factors is unknown.
• There exists several alternative resources to find apartments in NYC - no site will have access to all available rental listings.
• Anyone who’s ever apartment-hunted in NYC can tell you that neighborhood boundaries can be stretched to the point of meaninglessness by marketers. E.g. An apartment on the “Upper West Side” according to the listing may actually be located in what may be better-known as Morningside Heights, Washington Heights, or Harlem.

Returnt to Data Contents

StreetEasy Definitions

• Discount Share: The share of all active rental listings on StreetEasy that had a reduction in asking rent during that month/quarter/year.
• Median Asking Rent: The exact middle asking rent among all rental listings available on StreetEasy at any point during the month/quarter/year. In general, median values are more accurate than average values, which may be skewed by price outliers (a few rentals that are extremely expensive or extremely inexpensive).
• Rental Inventory: The number of rental listings available on StreetEasy at any point during the month/quarter/year.

Neighborhood Map

This doesn’t show all the neighborhood names but it’s a good reference for those that may not be familiar with any:

StreetEasy/Zillow used to share their neighborhood shape files but it seems like they’ve since stopped as this link now directs you to a contact form. I picked up this shape file from this website but I cannot vouch for its authenticity. I know that it must be at least a bit older because a few of the more-modern NYC neighborhoods aren’t mentioned (e.g.DUMBO).