Showing posts with label Lahman database. Show all posts
Showing posts with label Lahman database. Show all posts

July 23, 2014

Left-handed catchers


Benny Distefano – 1985 Donruss #166
(source: baseball-almanac.com)
We are approaching the twenty-fifth anniversary of the last time a left-handed throwing catcher appeared behind the plate in a Major League Baseball game; on August 18, 1989 Benny Distefano made his third and final appearance as a catcher for the Pirates. Distefano’s accomplishment was celebrated five years ago, in Alan Schwarz’s “Left-Handed and Left Out” (New York Times, 2009-08-15).

Jack Moore, writing on the site Sports on Earth in 2013 (“Why no left-handed catchers?”), points out that lack of left-handed catchers goes back a long way. One interesting piece of evidence is a 1948 Ripley’s “Believe It Or Not” item with a left-handed catcher Dick Bernard (you can read more about Bernard’s signing in the July 1, 1948 edition of the Tuscaloosa News). Bernard didn’t make the majors, and doesn’t appear in any of the minor league records that are available on-line either.


Dick Bernard in Ripley’s “Believe It or Not”, 1948-12-30
(source: 
sportsonearth.com)


There are a variety of hypotheses why there are no left-handed catchers, all of which are summarized in John Walsh’s “Top 10 Left-Handed Catchers for 2006” (a tongue-in-cheek title if ever there were) at The Hardball Times. A compelling explanation, and one supported by both Bill James and J.C. Bradbury (in his book The Baseball Economist) is natural selection; a left-handed little league player who can throw well will be groomed as a pitcher.

Throwing hand by fielding position as an example of a categorical variable



I was looking for some examples of categorical variables to display visually, and the lack of left-handed throwing catchers, compared to other positions, came to mind. The following uses R, and the Lahman database package.

The analysis requires merging the Master and Fielding tables in the Lahman database – the Master table gives the player's name and his throwing hand, and Fielding tells us how many games at each position they played. For the purpose of this analysis, we’ll look at the seasons 1954 (the first year in the Lahman database that has the outfield positions split into left, centre, and right) through 2012.

You may note that for the merging of the two tables, I used the new dplyr package. I tested the system.time of the basic version of “merge” to combine the two tables, and the “inner_join” in dplyr. The latter is substantially faster: my aging computer ran “merge” in about 5.5 seconds, compared to 0.17 seconds with dplyr.


# load the required packages
library(Lahman)
library(dplyr)
#

The first step is to create a new data table that merges the Fielding and Master tables, based on the common variable “playerID”. This new table has one row for each player, by position and season; we use the dim function to show the dimensions of the table.

Then, select only those seasons since 1954 and omit the records that are Designated Hitter (DH) and the summary of outfield positions (OF) (i.e. leave the RF, CF, and LF).

MasterFielding <- inner_join(Fielding, Master, by="playerID")
dim(MasterFielding)
## [1] 164903     52
#
MasterFielding <- filter(MasterFielding, POS != "OF" & POS != "DH" & yearID > "1953")
dim(MasterFielding)
## [1] 91214    52

This table needs to be summarized one step further – a single row for each player, counting how many games played at each position.

Player_games <- MasterFielding %.%
  group_by(playerID, nameFirst, nameLast, POS, throws) %.%
  summarise(gamecount = sum(G)) %.%
  arrange(desc(gamecount)) 
dim(Player_games)
## [1] 19501     6
head(Player_games)
## Source: local data frame [6 x 6]
## Groups: playerID, nameFirst, nameLast, POS
## 
##    playerID nameFirst nameLast POS throws gamecount
## 1 robinbr01    Brooks Robinson  3B      R      2870
## 2 bondsba01     Barry    Bonds  LF      L      2715
## 3 vizquom01      Omar  Vizquel  SS      R      2709
## 4  mayswi01    Willie     Mays  CF      R      2677
## 5 aparilu01      Luis Aparicio  SS      R      2583
## 6 jeterde01     Derek    Jeter  SS      R      2531

This table shows the career records for the most games played at the positions (for 1954-2012). We see that Brooks Robinson leads the way with 2,870 games played at third base, and the fact that Derek Jeter, at the end of the 2012 season, was closing in on Omar Vizquel’s career record for games played as a shortstop.

Cross-tab Tables


The next step is to prepare a simple cross-tab table (also known as contingency or pivot tables) showing the number of players cross-tabulated by position (POS) and throwing hand (throws).

Here, I’ll demonstrate two ways to do this: first with dplyr’s “group_by” and “summarise” (with a bit of help from reshape2), and then the “table” function in gmodels.

# first method - dplyr
Player_POS <- Player_games %.%
  group_by(POS, throws) %.%
  summarise(playercount = length(gamecount))
Player_POS
## Source: local data frame [17 x 3]
## Groups: POS
## 
##    POS throws playercount
## 1   1B      L         411
## 2   1B      R        1515
## 3   2B      L           4
## 4   2B      R        1560
## 5   3B      L           4
## 6   3B      R        1889
## 7    C      L           4
## 8    C      R         980
## 9   CF      L         393
## 10  CF      R        1252
## 11  LF      L         544
## 12  LF      R        2161
## 13   P      L        1452
## 14   P      R        3623
## 15  RF      L         520
## 16  RF      R        1893
## 17  SS      R        1296

To transform this long-form table into a traditional cross-tab shape we can use the “dcast” function in reshape2.

library(reshape2)
## Loading required package: reshape2
dcast(Player_POS, POS ~ throws, value.var = "playercount")
##   POS    L    R
## 1  1B  411 1515
## 2  2B    4 1560
## 3  3B    4 1889
## 4   C    4  980
## 5  CF  393 1252
## 6  LF  544 2161
## 7   P 1452 3623
## 8  RF  520 1893
## 9  SS   NA 1296

A second method to get the same result is to use the “table” function in the gmodels package.

library(gmodels)
## Loading required package: gmodels
throwPOS <- with(Player_games, table(POS, throws))
throwPOS
##     throws
## POS     L    R
##   1B  411 1515
##   2B    4 1560
##   3B    4 1889
##   C     4  980
##   CF  393 1252
##   LF  544 2161
##   P  1452 3623
##   RF  520 1893
##   SS    0 1296

A more elaborate table can be created using gmodels package. In this case, we’ll use the CrossTable function to generate a table with row percentages. You’ll note that the format is set to SPSS, so the table output resembles that software’s display style.

CrossTable(Player_games$POS, Player_games$throws, 
           digits=2, format="SPSS",
           prop.r=TRUE, prop.c=FALSE, prop.t=FALSE, prop.chisq=FALSE,  # keeping the row proportions
           chisq=TRUE)                                                 # adding the ChiSquare statistic
## 
##    Cell Contents
## |-------------------------|
## |                   Count |
## |             Row Percent |
## |-------------------------|
## 
## Total Observations in Table:  19501 
## 
##                  | Player_games$throws 
## Player_games$POS |        L  |        R  | Row Total | 
## -----------------|-----------|-----------|-----------|
##               1B |      411  |     1515  |     1926  | 
##                  |    21.34% |    78.66% |     9.88% | 
## -----------------|-----------|-----------|-----------|
##               2B |        4  |     1560  |     1564  | 
##                  |     0.26% |    99.74% |     8.02% | 
## -----------------|-----------|-----------|-----------|
##               3B |        4  |     1889  |     1893  | 
##                  |     0.21% |    99.79% |     9.71% | 
## -----------------|-----------|-----------|-----------|
##                C |        4  |      980  |      984  | 
##                  |     0.41% |    99.59% |     5.05% | 
## -----------------|-----------|-----------|-----------|
##               CF |      393  |     1252  |     1645  | 
##                  |    23.89% |    76.11% |     8.44% | 
## -----------------|-----------|-----------|-----------|
##               LF |      544  |     2161  |     2705  | 
##                  |    20.11% |    79.89% |    13.87% | 
## -----------------|-----------|-----------|-----------|
##                P |     1452  |     3623  |     5075  | 
##                  |    28.61% |    71.39% |    26.02% | 
## -----------------|-----------|-----------|-----------|
##               RF |      520  |     1893  |     2413  | 
##                  |    21.55% |    78.45% |    12.37% | 
## -----------------|-----------|-----------|-----------|
##               SS |        0  |     1296  |     1296  | 
##                  |     0.00% |   100.00% |     6.65% | 
## -----------------|-----------|-----------|-----------|
##     Column Total |     3332  |    16169  |    19501  | 
## -----------------|-----------|-----------|-----------|
## 
##  
## Statistics for All Table Factors
## 
## 
## Pearson's Chi-squared test 
## ------------------------------------------------------------
## Chi^2 =  1759     d.f. =  8     p =  0 
## 
## 
##  
##        Minimum expected frequency: 168.1

Mosaic Plot


A mosaic plot is an effective way to graphically represent the contents of the summary tables. Note that the length (left to right) dimension of each bar is constant, comparing proportions, while the height of the bar (top to bottom) varies depending on the absolute number of cases. The mosaic plot function is in the vcd package.

library(vcd)
## Loading required package: vcd
## Loading required package: grid
mosaic(throwPOS, highlighting = "throws", highlighting_fill=c("darkgrey", "white"))


Conclusion

The clear result is that it’s not just catchers that are overwhelmingly right-handed throwers, it’s also infielders (except first base). There have been very few southpaws playing second and third base – and there have been absolutely no left-handed throwing shortstops in this period.

As J.G. Preston puts it in the blog post “Left-handed throwing second basemen, shortstops and third basemen”,
While right-handed throwers can be found at any of the nine positions on a baseball field, left-handers are, in practice, restricted to five of them.

So who are these left-handed oddities? Using the filter function, it’s easy to find out:

# catchers
filter(Player_games, POS == "C", throws == "L")
## Source: local data frame [4 x 6]
## Groups: playerID, nameFirst, nameLast, POS
## 
##    playerID nameFirst  nameLast POS throws gamecount
## 1 distebe01     Benny Distefano   C      L         3
## 2  longda02      Dale      Long   C      L         2
## 3 squirmi01      Mike   Squires   C      L         2
## 4 shortch02     Chris     Short   C      L         1

# second base
filter(Player_games, POS == "2B", throws == "L")
## Source: local data frame [4 x 6]
## Groups: playerID, nameFirst, nameLast, POS
## 
##    playerID nameFirst  nameLast POS throws gamecount
## 1 marqugo01   Gonzalo   Marquez  2B      L         2
## 2 crowege01    George     Crowe  2B      L         1
## 3 mattido01       Don Mattingly  2B      L         1
## 4 mcdowsa01       Sam  McDowell  2B      L         1

# third base
filter(Player_games, POS == "3B", throws == "L")
## Source: local data frame [4 x 6]
## Groups: playerID, nameFirst, nameLast, POS
## 
##    playerID nameFirst  nameLast POS throws gamecount
## 1 squirmi01      Mike   Squires  3B      L        14
## 2 mattido01       Don Mattingly  3B      L         3
## 3 francte01     Terry  Francona  3B      L         1
## 4 valdema02     Mario    Valdez  3B      L         1

My github file for this entry in Markdown is here: [https://github.com/MonkmanMH/Bayesball/blob/master/LeftHandedCatchers.md]

-30-

June 4, 2013

Major League Baseball run scoring trends with R's Lahman package

The statistical software R has an ever-expanding array of packages that provide pre-programmed functions and datasets. One such package is named Lahman, bundling the contents of the Lahman database into a quick-and-easy resource for R users. In addition to the data tables, the package resources also contain a variety of analyses and graphics undertaken using the package, providing some examples of how the package can be used.

Full disclosure: I am now one of the Lahman package project members.

This is my first blog post using the Lahman package, and as a first step I will simply recreate the league run scoring trends graphs that I generated previously. Originally, I had used data from Baseball Reference, for the simple reason that the Lahman database does not, in its source form, contain any league-level aggregations.

The process for loading the Lahman package is as simple as any other R package; this simplicity is even greater if you are using an IDE such as RStudio. Once loaded, you have access to all the tables in the database, without any of the futzing that is sometimes required in tidying up a raw flat file (I find that variable names are sometimes lost or changed in translation).

The code (available as a gist here, downloadable as an R script file) creates a pair of tables, calculating each league's run scoring rates by year. Then, recycling my earlier code, it calculates a series of trend lines using the loess method, and graphs those trend lines. For simplicity's sake, only the final version of each graph is shown.

Step 1: install the package (if you haven't already), access the library, and load the data table “Teams”.


# load the package into R, and open the data table 'Teams' into the
# workspace
library("Lahman")
data(Teams)
#

The second step is to use the individual team season results to calculate the aggregate of each league's year. We start with 1901, the year the American League was formed. Once those tables are created, the loess function is used to calculate trend lines for each league's run scoring environment.

# ===== CREATE LEAGUE SUMMARY TABLES
# 
# select a sub-set of teams from 1901 [the establishment of the American
# League] forward to 2012
Teams_sub <- as.data.frame(subset(Teams, yearID > 1900))
# calculate each team's average runs and runs allowed per game
Teams_sub$RPG <- Teams_sub$R/Teams_sub$G
Teams_sub$RAPG <- Teams_sub$RA/Teams_sub$G
# create new data frame with season totals for each league
LG_RPG <- aggregate(cbind(R, RA, G) ~ yearID + lgID, data = Teams_sub, sum)
# calculate league + season runs and runs allowed per game
LG_RPG$LG_RPG <- LG_RPG$R/LG_RPG$G
LG_RPG$LG_RAPG <- LG_RPG$RA/LG_RPG$G
# select a sub-set of teams from 1901 [the establishment of the American
# League] forward to 2012 read the data into separate league tables
ALseason <- (subset(LG_RPG, yearID > 1900 & lgID == "AL"))
NLseason <- (subset(LG_RPG, yearID > 1900 & lgID == "NL"))
#
# ===== TRENDS: RUNS SCORED PER GAME
# 
# AMERICAN LEAGUE create new object ALRunScore.LO for loess model
ALRunScore.LO <- loess(ALseason$LG_RPG ~ ALseason$yearID)
ALRunScore.LO.predict <- predict(ALRunScore.LO)
# create new objects RunScore.Lo.XX for loess models with 'span' control
# span = 0.25
ALRunScore.LO.25 <- loess(ALseason$LG_RPG ~ ALseason$yearID, span = 0.25)
ALRunScore.LO.25.predict <- predict(ALRunScore.LO.25)
# span = 0.5
ALRunScore.LO.5 <- loess(ALseason$LG_RPG ~ ALseason$yearID, span = 0.5)
ALRunScore.LO.5.predict <- predict(ALRunScore.LO.5)
# NATIONAL LEAGUE create new object RunScore.LO for loess model
NLRunScore.LO <- loess(NLseason$LG_RPG ~ NLseason$yearID)
NLRunScore.LO.predict <- predict(NLRunScore.LO)
# loess models
NLRunScore.LO.25 <- loess(NLseason$LG_RPG ~ NLseason$yearID, span = 0.25)
NLRunScore.LO.25.predict <- predict(NLRunScore.LO.25)
NLRunScore.LO.5 <- loess(NLseason$LG_RPG ~ NLseason$yearID, span = 0.5)
NLRunScore.LO.5.predict <- predict(NLRunScore.LO.5)
#

Now that we have calculated the league averages and trend lines (using the loess method), we can start the plots. First, a simple plot of the actual values:

# MULTI-PLOT -- MERGING AL AND NL RESULTS plot individual years as lines
ylim <- c(3, 6)
# start with AL line
plot(ALseason$LG_RPG ~ ALseason$yearID, type = "l", lty = "solid", col = "red", 
    lwd = 2, main = "Runs per team per game, 1901-2012", ylim = ylim, xlab = "year", 
    ylab = "runs per game")
# add NL line
lines(NLseason$yearID, NLseason$LG_RPG, lty = "solid", col = "blue", lwd = 2)
# chart additions
grid()
legend(1900, 3.5, c("AL", "NL"), lty = c("solid", "solid"), col = c("red", "blue"), 
    lwd = c(2, 2))


Next, comparing the league trends.

# plot multiple loess curves (span=0.50 and 0.25)
ylim <- c(3, 6)
# start with AL line
plot(ALRunScore.LO.5.predict ~ ALseason$yearID, type = "l", lty = "solid", col = "red", 
    lwd = 2, main = "Runs per team per game, 1901-2012", ylim = ylim, xlab = "year", 
    ylab = "runs per game")
# add NL line
lines(NLseason$yearID, NLRunScore.LO.5.predict, lty = "solid", col = "blue", 
    lwd = 2)
# add 0.25 lines
lines(ALseason$yearID, ALRunScore.LO.25.predict, lty = "dashed", col = "red", 
    lwd = 2)
lines(NLseason$yearID, NLRunScore.LO.25.predict, lty = "dashed", col = "blue", 
    lwd = 2)
# chart additions
legend(1900, 3.5, c("AL (span=0.50)", "NL (span=0.50)", "AL (span=0.25)", "NL (span=0.25)"), 
    lty = c("solid", "solid", "dashed", "dashed"), col = c("red", "blue", "red", 
        "blue"), lwd = c(2, 2, 2, 2))
grid()


Next, calculate the difference between the two leagues – both the absolute difference and the difference in the loess trend lines.

# 1. absolute
RunDiff <- (ALseason$LG_RPG - NLseason$LG_RPG)
# 2. LOESS span=0.25
RunDiffLO <- (ALRunScore.LO.25.predict - NLRunScore.LO.25.predict)
#

And plot the differences.

# plot each year absolute difference as bar, difference in trend as line
ylim <- c(-1, 1.5)
plot(RunDiff ~ ALseason$yearID, type = "h", lty = "solid", col = "blue", lwd = 2, 
    main = "Run scoring trend: AL difference from NL, 1901-2012", ylim = ylim, 
    xlab = "year", ylab = "runs per game")
# add RunDiff line
lines(ALseason$yearID, RunDiffLO, lty = "solid", col = "black", lwd = 2)
# add line at zero
abline(h = 0, lty = "dotdash")
# chart additions
grid()
legend(1900, 1.5, c("AL difference from NL: absolute", "AL difference from NL, LOESS (span=0.25)"), 
    lty = c("solid", "solid"), col = c("blue", "black"), lwd = c(2, 2))
#


For the next “using R” post, I'll take a look at the ways to plot the residuals from the loess method.

The one after that: ggplot2 versions of the graphs.

-30-

February 24, 2013

MLB runs allowed by team

Or, How good were the Maddux/Glavine-era Braves?

In this on-going series of posts about run scoring in Major League Baseball, for this installment I'll turn the equation around and look at runs allowed.  In order to account for the changing run scoring environments, the runs allowed by individual teams is compared to the league average for that season, creating an index where 100 is the league average. In this formulation, a score below 100 is a good thing; a team with an index score of 95 allowed runs at a rate 5 percentage points below the league average.

Having written the original code in R, it's now a very simple process to change a few variable names and create the equivalent of the earlier runs scored analysis, but looking at runs allowed. This is one of the most important benefits of a code/syntax environment, an option that doesn't exist if  you are using a point-and-click GUI interface.

February 17, 2013

Run production, one team at a time


In a previous post, I used R to process data from the Lahman database to calculate index values that compare a team's run production to the league average for that year.  For the purpose of that exercise, I started the sequence at 1947, but for what follows I re-ran the code with the time period 1901-2012.

The R code I used can be found at this Github gist. Instead of boring you here with the ins and outs of what the code is doing, I've embedded that as documentation in the gist. The R code assumes that you've got a data frame called "Teams.merge" already in your workspace.  This can be achieved by running the previous code, or if you've done that before, you'll have created a csv file with the name "Teams.merge.csv", and now have the option to read that file as a data frame "Teams.merge".

The first step is to choose one of the current teams, and create a data frame that contains just that club's history.  Once this has been done, the code then creates trend lines (using the LOESS method, as I did with the leagues in previous posts), and then plot them.

February 2, 2013

Comparing individual team run production

Or, The 2010 Mariners: How Bad Were They?

In earlier posts, I used the statistical software R to plot the trends in league average run scoring since 1901. This was the first step to answering other questions I had on my mind:
  1. How poor was the offensive performance of the 2010 Seattle Mariners?
  2. Are they showing any signs of improvement?
  3. And how can I use R to tabulate the data to answer these questions?
So, to answer Question #1.  It is well-established that the 2010 Mariners were not very good, at least offensively. (For fans of the team the well-deserved Cy Young award won by Felix Hernandez is surely the highlight of the season.) But I wanted a form of relative measure that would be comparable across time, to accommodate the various fluctuations in run scoring that were the subject of that earlier post.

As I started into this, the first decision was to draw a line in the historical record. I opted to use the eras described in Bill James' "Dividing Baseball History into Eras" article (behind a pay wall – but chances are if you're reading my blog, you already a Bill James subscriber):
  • Era 1 (The Pioneer Era), 1871-1892 
  • Era 2 (The Spitball Era), 1893-1919 
  • Era 3 (The Landis Era), 1920-1946 
  • Era 4 (The Baby Boomers Era), 1947-1968 
  • Era 5 (The Artifical Turf Era), 1969-1992 
  • Era 6 (The Camden Yards Era), 1993-2012
Based on these groupings, I opted to use the range of seasons 1947-2012 inclusive. This yields 1,580 team seasons of National League and American League baseball.

The second step was to calculate a runs per game (RPG) for each team, by year. This corrects for the longer regular season in the post-expansion period, the strike-shortened seasons, and will give us a common denominator to compare the results so far in 2012.

To do this, I accessed the 2012 edition of the Lahman database. Once I had downloaded and extracted the comma-delimted version of the files, I read the "teams" file into R.

January 12, 2013

Latest Lahman database now available

One of the most important resources for baseball researchers, the Lahman database, has been updated with the 2012 Major League Baseball data.  It's an amazing compendium, with complete statistics for teams and players from 1871 to 2012.

You can access the dataset (in a variety of formats) here.

And I encourage you to make a financial contribution through the donations page to help support this endeavour.

-30-