Indexing: Going, going… gone?

Baseball teams and the lotteries have more in common than other industries. Both are monopolies. A new upstart baseball team is not going to appear in Washington, D.C. to compete against the Nationals, just as a new lottery is not going to appear to compete against the D.C. Lottery. Additionally, both sell entertainment that used to dominate their markets, but now are finding it harder to attract Millennials’ interest. In a recent study of 1,500 people conducted by the Texas Lottery Commission, the average age for lottery players was 55.7. The average age for a baseball fan and lottery player is identical. “[The 2014] World Series had a viewer with a median age of 55.6, a new high. Just five years prior, it had been sub-50 at 49.9, and was in the 44-46 range during the early 1990’s,” Mark Fahey, CNBC Data Journalist, said in an online article.

Due to some similarities, lotteries might need to take a keen look at Moneyball: The Art of Winning an Unfair Game by Michael Lewis, which details how Billy Beane, manager of the Oakland A’s, revolutionized baseball with the use of sabermetrics. In 2002, the New York Yankees spent $144 million but the A’s could only spend $44 million. However, the A’s won the American League West because of how they valued their players.

Prior to the 2002 season, baseball scouts judged players on statistics like batting averages, which were poor predictors of winning games. The A’s recognized other variables were better indicators of how many runs a player would score. They began recruiting players who excelled in these variables, paying less attention to batting averages. They were able to capitalize on a market inefficiency in baseball.

It wasn’t that batting average was a poor indicator of a player’s skill. It had merit, but it was overvalued by managers because it was a statistic that had always been used to judge a player’s worth. Those long-standing biases are present in every industry, including the lottery. So the question becomes, what is the lottery industry’s batting average?

Our Batting Average?

The brilliance of indexing is that internally, for any lottery, it can encapsulate complicated information in a way that is very easy to understand and compare very quickly. But problems quickly arise when states begin looking at other states’ index numbers. “Indexing is one of a number of variables that we use to forecast or measure success. If we’re looking for new games, and they index really well in one state, it’s not a guarantee that they’re going to index well here.  While it does provide us with considerable insight, we always weigh it with other factors,” Lauren Luttrell, Instant Product Manager, Massachusetts Lottery, said.

There are multiple reasons for that. The primary issue is that indexing can also be easily manipulated. “A comparison group of average-to-poor selling tickets will make a game look like it’s performing better than it really is; a comparison group made up of high-performing tickets has the opposite effect. I prefer to use a couple of different comparison groups when calculating indexes. One group could be games of a similar play style, another could be games with the same theme. This gives me a view of how a game is performing from several different perspectives. One of the misleading things about indexes is, it depends on what the comparison group is. You can make a game look really good, if you want to compare it to games that performed poorly,” Glenn Strong, Instant Games Product Manager, Michigan Lottery, said.

A major factor in comparing apples to apples is the type of game that is being sold. A holiday game has a different sales pattern than an extended play game. An extended play game can have a different sales pattern than a match 3 game. All three of these games attract different audiences.

Environmental factors can also have a drastic effect on sales performance. For instance, one state sold two identical games in 2015 and 2016. Both had the same art, same color, same format, same play style, and were released in January. When compared with similar games, the 2015 game indexed in the first eight weeks at 166. In 2016, over the same time period, the game indexed at 66. The cause could be that the public was tired of the game. Or perhaps the reason the game indexed so drastically different was because the 2016 game was released on January 5, 2016 during the rollup of the largest jackpot in the history of the global lottery industry. When the time frame is extended from eight weeks to 20 weeks, the index of the 2015 game normalizes to 128, and the 2016 game indexes at 93.

Sometimes the lottery itself gives one game an unfair advantage to index better over another game. TV, radio and even POS can really alter how a game performs. It’s important to understand the imbalanced advantage it gives to some games over others. For instance, in one state, when comparing four games that were promoted at the $2 price point, they indexed respectively at 157, 115, 107, and 89. However, when the games were compared against only $2 promoted games across multiple years, the index changed drastically: 134, 98, 92, and 76.

Indexing can cause us to fall into the same trap that baseball managers fell into in 2002. Indexing is generated using only sales data. While important, this may not be the end result that lotteries want. For baseball, teams want to score runs. Lotteries want to increase their profits to good causes. They also have a second responsibility in that they must acquire new players for the short and long term.

Indexing can throw a wrench into things because not only does it not reflect profits to good causes or player acquisition, but can, in fact, overshadow those two criteria.  For instance, if a niche game has a high index, it does not guarantee that the game had reached a new player base. The game could have just attracted the core player for a multitude of reasons. Or a game could have a low indexing number, but have reached an entirely new player. Without the proper analytics, marketers and product managers are left in the dark attempting to guess what happened at retail, which means niche tickets focused on different audiences can be risky to put in the product portfolio.

“We’ve steered away from focusing on smaller audiences with our strategy,” Justin Rock, Director of Product and Research, Florida Lottery said. “Our goal is to offer games with broad/mass appeal and take calculated risks with branded products. Every launch is important. Every game is important. We can’t risk flyers as we move forward.”

Even if a lottery were to undergo a massive effort to convert new players, the issue remains it would be difficult to prove it was successful. There are limited metrics to ultimately prove tickets reached new audiences and that those new audiences continued to play.  The research project required to quantify that information would have to be massive and expensive.

“I am not aware of anyone who is able to quantify how many players you are converting when you offer a niche game.  It’s often argued that ‘these products are good for the brand overall, even if it’s not a strong selling ticket, we’re reaching more people and that’s going to boost sales across the board.’ While I certainly agree there is a reach that you are getting when you put a product out there that is building off another brand’s equity, I can’t tell you we’re doing a game because we expect to convert x amount of new players into core lottery players.”  Rock stated.

For other lotteries, there are still immediate benefits that have little to do with converting long-term players. “Within our oversized annual scratch game portfolio, we typically include a number of games that might not traditionally ‘index’ that well, but are necessary and effective in either building the Texas Lottery brand of the Texas Lottery, raising our profile among the many other consumer products available at retail, or just simply garnering headlines with a new game launch, which can morph the routine issuance of a new scratch ticket game into an advertising campaign for the Texas Lottery,” Gary Grief, Executive Director, Texas Lottery, said.

These benefits, including brand awareness, are critical to lotteries’ success. They remind lapsed players to play again, drive active players to retail, even siphon brand equity off other brands. But as Grief mentioned, these games provide benefits that sales cannot reflect, and therefore are not reflected in indexing at all.

All of this is not to say that lotteries  should abandon indexing. It is an important resource that lotteries should use. But indexing should not always be the deciding factor. It is simply one variable of many that can suggest if a ticket will perform well or poorly in another state. So many influences can affect an indexing number: advertising support, planogramming, seasonality, draw game’s jackpots, holographic material, call outs, art, play style, game mechanics like multipliers, amount of similar price point tickets on sale, prize value, and a plethora of other factors.

“Where (indexing) is very useful is when you look at a non-licensed, non-spotlight game. If the indexing is really poor or really good, it gives you a pretty good indicator. We’re about to do a money themed $1 game, and when you look at the indexing in the states where it has run, scores are off the chart, 229, 191 and 179. So that’s a good indicator,” said Rock.

As long as indexing remains the primary data point, however, it could be more hindrance than a help. A high index number can distract from other disquieting variables that may go unnoticed. For instance, in Michigan, after a second chance promotion on a instant game, Strong polled all new registrants to Michigan Lottery’s Players’ Club to see if the ticket had been purchased by players who had never purchased a ticket before. “To answer this question I needed to 1) find players who had never purchased an instant ticket until seeing the licensed game in question (let’s call it Brand A), and 2) send these people a questionnaire to verify that they are indeed a new player. Finding those people who had never played an instant ticket until coming across Brand A was obviously a significant challenge and the only way I could think of to find them was through our Players Club. Members of the club can earn reward points by entering the validation code from non-winning tickets. My thought was that the only chance I had to find these potential ‘new players’ was to isolate those people who had entered (and presumably purchased) one or more non-winning Brand A tickets AND didn’t join Players Club until after Brand A launched. Although not perfect, this seemed to be the best way available to find an audience of potential new players created by Brand A. We emailed a survey invitation to these people and learned that 100% of those who responded were instant ticket buyers before the launch of Brand A. We ran the same survey for another licensed game (Brand B) and saw the same result,” Strong said. Few would argue that the 300 people who took part in the combined two surveys could be considered statistically projectable, especially onto a national stage, but the severe nature of the results is cause for concern. If those numbers are even close to being accurate and lotteries aren’t cultivating new players, then there could be dire consequences for the industry down the road.

The Texas Lottery is fully aware of the potential consequences: “We have experienced unprecedented growth in the sales of our scratch ticket products over the last several years in Texas and, in doing so, we’ve determined that traditional “indexing” as a way to measure the success of an individual scratch ticket game is a flawed approach… How an individual game indexes in the traditional sense is only one of several data points, some of them more subjective than objective, that must be examined to assess the true performance of any individual scratch game. Seasonality, game novelty, current events, advertising support, multi-state game jackpot levels, print size; all these factors and more must be included in any game performance analysis… This gives you a flavor as to the complex set of variables and goals that need to be considered whenever we evaluate a scratch game’s overall performance. We are working with IGT, our Lottery Operator, to incorporate these variables and goals into our ongoing scratch ticket game analysis process,” Grief said.

The Texas Lottery, one of the largest lotteries in the U.S., is once again leading the charge in innovative ways to improve the industry. It is imperative that all lotteries review their system when it is released. However, the industry should not wait for the Texas Lottery. Large or small, lottery or vendor, why not pursue new ways to evaluate the instant ticket market? It is anyone’s guess who will come up with the next set of Key Performance Indicators  that will transform the industry. It will take as much intelligence as imagination to come up with. But we cannot forget it wasn’t the imposing superstar New York Yankees who revolutionized baseball, it was the underdog Oakland A’s.