Excerpt

T
e
x
t
 Preface
The Power of
Dynamic Data

The most valuable commodity I know of is

information.
—Michael Douglas as Gordon Gekko, Wall Street

O
n August 7, 2015, the mood at Chipotle headquarters in Denver,
Colorado, was jovial. The stock (NYSE: CMG) of the chain of “fast
casual” Mexican restaurants had just reached an all-time high of
$749.12. Sure, the company faced its fair share of challenges (includ-
ing an alarmingly high number of lawsuits), but today was a day to
celebrate.
Fast-forward six months. As so often is the case these days, things
had changed very quickly.
A series of food-borne illnesses came to light at the end of 2015—
and not just a few mild stomachaches caused by a batch of bad salsa.
The true culprit: E. coli. As the Centers for Disease Control and Preven-
tion (CDC) announced on December 2, 2015, “52 people from nine
states have been sickened, 20 have been hospitalized, and there are
no deaths.”1
By April 16, 2016, Chipotle’s stock was in free fall, dropping 40
percent from its high to $444. Things continued to spiral downward
for the chain. The stock hit $370 on December 9 of that year. In August
2016, nearly 10,000 employees sued the company for unpaid wages.
In September, a 16-year-old girl won a $7.65 million lawsuit against
the company for sexual harassment. One of the victim’s attorneys

xvii

fpref xvii 22 May 2017 8:32 AM

 xviii ▸ PREFACE

described the situation as “a brothel that just served food.”2 Damning

words to be sure.
Sensing opportunity, activist investor Bill Ackman started gobbling
up Chipotle equities. His hedge fund, Pershing Square Capital Man-
agement (PSCM), purchased large quantities of options trades, “nor-
mal” stock buys, and equity swaps. Rumor had it that Ackman wasn’t
just looking to make a buck; he wanted seats on the Chipotle board
and a significant say in the company’s long-term and daily manage-
ment. And PSCM wasn’t the only hedge fund betting long on CMG in
2016. Plenty of others were taking notice.3
Ackman is an interesting cat and a mixed blessing to the Chipotles
of the world.* Over the years, he has earned a reputation as a thorn in
the side of many distraught companies and their boards of directors.
Still, Chipotle executives knew that his hedge fund was keeping their
portfolios healthy. No doubt that CMG would have fallen further if
PSCM and other funds weren’t buying so aggressively.
Why were hedge funds buying Chipotle’s shares on the cheap in
2016? You don’t need to be Warren Buffett to see what was happen-
ing. The heads of these funds believed in the long-term value of the
stock. Chipotle would eventually recover, they reasoned, so why not
make a few bucks? In a way, Ackman and his ilk are no different from
Homer Simpson. The patriarch of the iconic cartoon family once sum-
marized his remarkably facile investment philosophy in the following
seven words, “Buy low. Sell high. That’s my motto.”†
This begs the natural question: On what basis do these folks make
their multibillion-dollar bets?
At a high level, sharks such as Ackman operate via a combination
of instinct and analysis. With regard to the latter, hedge funds have
always coveted highly quantitative employees—aka quants. As Scott
Patterson writes in The Quants: How a New Breed of Math Whizzes Con-
quered Wall Street and Nearly Destroyed It, their complex and proprietary
models factor in dozens or even hundreds of variables in attempting to
predict stock prices and place large wagers.
New and unexpected data sources could be worth a fortune.

* Watch an interview with him on Charlie Rose at http://bit.ly/2mTzWKv.

†
From “Burns Verkaufen der Kraftwerk,” one of my very favorite episodes of The Simpson.

fpref xviii 22 May 2017 8:32 AM fpref

 PREFACE ◂ xix

FOURSQUARE’S RISE, FALL, AND DATA-DRIVEN

SECOND ACT

Although Facebook beat it by five years, Foursquare still arrived rela-

tively early at the social-media party. Launched in March 2009 as a
“local search-and-discovery service mobile app,” it didn’t take long for
the company to approach unicorn status. Cofounder and CEO Dennis
Crowley became a bona fide rock star. Millions of people used the app
to check in to restaurants and bars. Of course, none of this would have
been possible as late as 2006. By 2009, though, the smartphone revo-
lution was in full swing. Foursquare could piggyback on the ubiquity
of iPhones and Droids.
Crowley and Foursquare allowed anyone to download and use the
app for free. Millions of people did. Oodles of active users, however,
do not a business model make. There’s a world of difference between
a user and a customer.
At some point, like all enterprises, Foursquare needed to make
money. By growing its user base, Foursquare hoped to expand its cus-
tomer base: local businesses that could create highly targeted ads that
its millions of users would see and, it was hoped, act on.
Foursquare was about to take location-based advertising into the
smartphone age. No longer would a city pub owner or restaurateur
need to pay someone to interrupt passersby on the street and hand out
cards that advertised two-for-one drink specials. Via Foursquare, eat-
eries could reach potential customers in a way never before possible.
At least that was the theory.
Foursquare’s promise has always exceeded its financial results.
For all of its users and hype, the company has never reported earning
a profit.4 At different points in 2012, both Marissa Mayer’s acquisi-
tion-happy Yahoo! and Facebook reportedly flirted with acquiring
Foursquare. In the end, though, the parties never consummated
a deal.5 Yahoo! remained a mess, and Facebook didn’t really need
Foursquare. Its network was enormous, and it wasn’t as if the idea of
a check-in had never occurred to Mark Zuckerberg. (In August 2010,
the social network launched Places, a feature “not unlike” Four-
square.6) Determined to remain relevant, Crowley and his troops
soldiered on.

:32 AM fpref xix 22 May 2017 8:32 AM

 xx ▸ PREFACE

Version 2.0: Two Apps Are Better Than One

On May 15, 2014, Foursquare launched a spin-off app called Swarm.

The new app allowed users to broadcast their locations to their friends
on social networks such as Facebook and Twitter. The main Four-
square app would still exist, but with a new focus. It would attempt to
wean market share from Yelp. Writing for The Verge, Ben Popper and
Ellis Hamburger explained the two apps’ different purposes:

Swarm will be a social heat map, helping users find friends

nearby and check in to share their location. The new
Foursquare will ditch the check-in and focus solely on
exploration and discovery, finally positioning itself as a
true Yelp-killer in the battle to provide great local search.7

Splitting Swarm from the Foursquare app has not turned out to
be a panacea. Over the past few years, many industry analysts have
doubted its long-term financial solvency. Foursquare has lost its status
as an it company. In Figure P.1, Google Trends shows just how far the
company has fallen.
Something had to give.
On January 14, 2016, COO Jeff Glueck replaced Crowley as CEO.
On the same day that that long-rumored change of leadership took
place, Foursquare bit the bullet and announced a new investor lifeline
at a fraction of its prior valuation. Yes, the company and its employees
had to endure the ignominy of the dreaded down round. As Mike Isaac
wrote for the New York Times:

Foursquare said it had raised $45 million in a new round

of venture funding, as it tries to bolster its location
data-based advertising and developer businesses. The
financing pegs Foursquare’s valuation at roughly half of the
approximately $650 million that it was valued at in its last
round in 2013, according to three people with knowledge
of the deal’s terms, who spoke on the condition of
anonymity.8 [Emphasis mine.]

Despite Foursquare’s well-documented struggles, the app still

sports a reported 50 million monthly active users.9 As Bloomberg TV’s
Cory Johnson is fond of saying, “That ain’t nothin’.” Was it possible
that Foursquare’s next and ultimately best business model was staring
its management in the face?

fpref xx 22 May 2017 8:32 AM fpref

 PREFACE ◂ xxi

100

75

50

25

March 1, 2009 August 1, 2011 January 1, 2014 June 1, 2016

Figure P.1 Foursquare Interest over Time, March 1, 2009, to March 29, 2017
Source: Google Trends.

Version 3.0: A Data-Induced “Pivot”

On April 12, 2016, Glueck penned a fascinating post on Medium10 that

qualified as bragging or at least posturing. The Foursquare CEO revealed
how his company collated user check-in data and other variables to
accurately predict Chipotle’s first-quarter sales. (The number dropped
nearly 30 percent compared to the fourth quarter of 2015.)
As anyone who has studied retail knows, foot traffic isn’t a terribly
innovative concept these days. Brick-and-mortar retailers have known
for many decades that it can serve as a valuable proxy for sales and
revenue. All else being equal, there’s a direct relationship between the
former and the latter. Still, Glueck’s lengthy data- and graph-laden
article illustrated the power of “digital” foot traffic. Figure P.2 shows
one of the post’s charts.

10%

7.5%
E. coli
Frequent Chipotle Goers Outbreak Employees
Sick
5%

2.5%
Infrequent Chipotle Visitors
5

15

6
01

01
20
2

2
n.

n.
ly
Ju
Ja

Ja

Figure P.2 Chipotle Share of Restaurant Foot Traffic (Week over Week)
Source: Foursquare Medium feed.

:32 AM fpref xxi 22 May 2017 8:32 AM

 xxii ▸ PREFACE

Glueck’s post does not formally ask the following questions, but it
certainly implies them:
■ What if Foursquare could harness this type of data en masse and
tie it to detailed user demographic information? (Foursquare
allows its users to log in via Facebook, arguably the world’s rich-
est data trove.)
■ Would companies with physical presences be willing to pony
up for this kind of information? (Better question: Why wouldn’t
they be willing to pay handsomely for it?)
■ What about opportunistic hedge-fund managers looking to out-
smart the market? What about the Bill Ackmans of the world?
■ Could Chipotle use location-based information to offer differ-
ent deals and coupons to its growing number of ex-customers?
Could Foursquare help Chipotle rescue customers? Could it be
a means to an end?
■ Would any chain restaurants be willing to pay Foursquare not
to release this type of damning information? (Admittedly, this
might qualify as blackmail or at least as unethical.)
Glueck wasn’t just speculating about what his company could theo-
retically offer. As it turns out, Foursquare no longer just sells in-app ads
to local bars and restaurants; an unknown but evidently increasing rev-
enue stream for the company involves “renting” its data to interested
parties. Foursquare’s new Place Insights service purportedly offers:
■ Insights from the world’s largest opt-in foot traffic panel
■ Overnight analysis of a global, cross-category dataset
■ Translation of real-world behavior into business health, trend
detection, and consumer insights*
Put differently, Glueck is attempting to redefine Foursquare as a
data-licensing company. Just look at the company’s website copy in
early 2010:

Foursquare on your phone gives you & your friends new

ways of exploring your city. Earn points & unlock badges
for discovering new things.†

* See https://enterprise.foursquare.com/insights.
†
See http://bit.ly/2oImTsS.

fpref xxii 22 May 2017 8:32 AM fpref

 PREFACE ◂ xxiii

The focus is clearly on the consumer/user. At the time, many com-

panies employed gamification strategies. Now contrast that one with
the company’s business-first message today:

Foursquare is a technology company that uses location

intelligence to build meaningful consumer experiences
and business solutions.*

The differences between Foursquare’s early message and its con-

temporary one could not be more stark. Nothing against Joe Sixpack
or Melissa Millennial, but they probably don’t understand what loca-
tion intelligence and meaningful consumer experiences even mean. I doubt
that they would sign up for them. It’s a moot point, though, because
Foursquare isn’t chasing the Joes and Melissas anymore. They don’t
pay the bills, at least directly.
Given Foursquare’s history, its decision to rebrand is no coinci-
dence, nor is its new message isolated to its website. Foursquare now
consistently refers to itself as a “location intelligence company.” Just
view its official Twitter† and Medium‡ feeds. Its blog posts, while cer-
tainly informative, are meant to plant a very specific seed in the heads
of prospective customers—that is, firms that would benefit from using
Foursquare’s data. For instance, in a post on Medium dated August 4,
2016, Foursquare claimed that it knew precisely how many women
(justifiably) avoided Trump properties during the 2016 presidential
election.§ Other posts boast previously successful predictions of Apple
iPhone 6 sales and the impact of the decision by McDonald’s to sell
breakfast all day long.
To use Eric Reis’s now-hackneyed term, if Foursquare successfully
pivots, it would be neither the first in history nor the craziest. YouTube
began as “Tune In Hook Up,” a dating site redolent of HotorNot.**
Instagram used to be Burbn, a location-based gaming and social net-
working app nearly identical to Foursquare.11 Before finding its footing
with photos, Flickr focused on gaming.

* See https://foursquare.com/about.
†
See https://twitter.com/@foursquare.
‡
See https://medium.com/@foursquare.
§
See http://bit.ly/2lOtPaE.
** See http://mashable.com/2011/02/19/youtube-facts.

:32 AM fpref xxiii 22 May 2017 8:32 AM

 xxiv ▸ PREFACE

Chapter 3 will return to Foursquare in the context of streaming

data and application program interfaces. For now, suffice it to say that
the current, data-oriented incarnation of Foursquare seems closer
than ever to finally capitalizing on its promise.

CHAPTER REVIEW AND DISCUSSION QUESTIONS

■ Foot traffic has always mattered. What’s different about it today?

■ Why would hedge funds and even individual investors be interested in
data related to “digital” foot traffic? Can you think of any other uses for
this data?
■ Why has Foursquare struggled to meet its financial goals? What is it trying to do
to finally meet them? Do you think that the company will succeed?

NEXT

Do stories, ideas, questions, and issues such as these interest you? Do

you wish that you could use data and analytics in this way? If so, then
keep reading. You have found the right book.

NOTES
1. “CDC Update: Chipotle-Linked E. Coli Outbreak Case Count Now at 52,” Food Safety
News, December 4, 2015, http://bit.ly/2na5c8C.
2. Virginia Chamlee, “Teen Chipotle Worker Wins $7.65M in Sexual Harassment
Suit,” Eater, September 29, 2016, http://bit.ly/2mMEstY.
3. John Maxfield, “Hedge Funds Gobbled Up Chipotle’s Stock Last Quarter,” The Motley
Fool, February 25, 2017, http://bit.ly/2nq54h7.
4. Matthew Lynley, “How Foursquare Hopes to Hit Profitability,” Tech Crunch, May 9,
2016, http://tcrn.ch/2naP2f1.
5. Alyson Shontell, “Remember, Dennis Crowley Could Have Sold Foursquare for
$120 Million,” Business Insider, January 11, 2013, http://read.bi/2mTtgMn.
6. Ryan Singel, “Facebook Launches ‘Check-In’ Service to Connect People in Real
Space,” Wired, August 18, 2010, http://bit.ly/2mvAraS.
7. Ben Popper and Ellis Hamburger, “Meet Swarm: Foursquare’s Ambitious Plan to
Split Its App in Two,” The Verge, May 1, 2014, http://bit.ly/1hh0Dvd.

fpref xxiv 22 May 2017 8:32 AM fpref

 PREFACE ◂ xxv

8. Mike Isaac, “Foursquare Raises $45 Million, Cutting Its Valuation Nearly in Half,”
New York Times, January 14, 2016, http://nyti.ms/2md5S9c.
9. Ken Yeung, “Foursquare Users Have Checked In over 10 Billion Times,” VentureBeat,
September 13, 2016, http://bit.ly/2mP7Wbx.
10. Jeff Glueck, “Foursquare Predicts Chipotle’s Q1 Sales Down Nearly 30%; Foot
Traffic Reveals the Start of a Mixed Recovery,” Medium, April 12, 2016, http://bit
.ly/2ngij51.
11. Megan Garber, “Instagram Was First Called ‘Burbn,’” The Atlantic, July 2, 2014,
http://theatln.tc/2ohL9BI.

:32 AM fpref xxv 22 May 2017 8:32 AM

 Introduction
It Didn’t Used to Be
This Way

The value of an idea lies in the using of it.

—Thomas Edison

S
o how did Foursquare predict Chipotle’s sales for the first quarter
of 2016 with such scary accuracy?
Permit me four answers to this question.
Here’s the really short one: data.
Here’s the second, just-plain-short one: Foursquare collected accu-
rate and real-time data on Chipotle check-ins over time. Equipped with
this information, the company’s data scientists built a model. That’s it.
Now, I don’t mean to oversimplify or to diminish Foursquare, its
employees, or what it was able to do here. As explained in the preface,
Foursquare merely answered a question by using the technology and
data available to it with a considerable tip of the hat to:

■ The hardware of third-party smartphone manufacturers such as

Apple, Samsung, and others.
■ Powerful software such as iOS and Android.
■ Related tools in the form of software development kits and
application program interfaces.

cintro 1 22 May 2017 9:31 AM

 2 ▸ ANALYTICS

■ The massive investments of Verizon, AT&T, and others to build

their carrier networks.
■ Government research and infrastructure projects.*

That is, Foursquare built something very impressive, but not

entirely unprecedented—at least in today’s environment—and not
without considerable assistance. Jeff Bezos of Amazon has made the
same point: Yes, he worked very hard, but his company did not need
to build a national transportation system. It merely took advantage of
the existing one.†
The third and longer answer is: I’m not exactly sure. Like all but
a few people, I can’t tell you precisely how the company worked its
magic. For all sorts of valid reasons, Foursquare doesn’t make its code
base and user data freely available to the general public.‡ It’s not an
open-source project à la Atom, Github’s “hackable text editor.” I don’t
know Foursquare’s technical specifications, nor have I studied the ins
and outs of its application program interface.§ Finally, if I asked any-
one in the know at Foursquare to fill me in, I wouldn’t get very far.
Why tell me? Ex-employees in the know have most likely signed non-
disclosure agreements, anyway.
The fourth and final answer is concurrently both more ambiguous
and more definitive. Without fear of accurate contradiction, I can tell
you that Foursquare derived these insights by not following the path of
so many of its predecessors.

A LITTLE HISTORY LESSON

Many companies have historically attempted to glean insights very

methodically. Large and midsized firms in particular would slowly
build and integrate their enterprise systems, data warehouses, data
hubs, and data marts. As for small businesses, their owners typically
had neither the time nor the expertise to cobble together data from a

* Many people are quick to point out government’s flaws, but reluctant to acknowledge
successes such as the ARPANET, the predecessor to the modern-day Internet.
†
For a fascinating interview with the man, see https://charlierose.com/videos/29412.
‡
Nor do many of the companies profiled in this book.
§
Here’s the starting point: https://developer.foursquare.com.

cintro 2 22 May 2017 9:31 AM cintro

 INTRODUCTION ◂ 3

bunch of disparate sources. Even if they did, most couldn’t afford the
six- or even seven-figure price tags of software vendors’ best-of-breed
solutions.
For instance, consider a typical project plan for a new business
intelligence (BI) application in 2002. It typically involved the steps and
approximate time frames listed in Table I.1.

Table I.1 Project Plan for Launch of Generic BI Application

Phase Description Start Date End Date

1 Evaluate proposals from software vendors, check 2/1/02 5/31/02
references, and perform general due diligence.
2 Select winning bid. Negotiate terms and sign contract. 6/1/02 7/31/02
3 Extract data from legacy systems, clean up errors, and 8/1/02 8/31/02
deduplicate records.
4 Implement and customize software, typically with help of 9/1/02 10/31/02
expensive consultants.
5 Train users on new application. 11/1/02 2/28/03
6 Load purified data into BI application and address errors. 3/1/03 3/31/03
7 Launch application and squash bugs. 4/1/03 4/30/03
8 Engage vendor in on-site or remote application support. 5/1/03 6/30/03
Source: Phil Simon.

Think about it: More than a year would pass from the project’s
formal kickoff until employees actually used the application in a pro-
duction environment—and that’s if everything went according to plan. As I
wrote in Why New Systems Fail, more than half of the time that doesn’t
happen. On the contrary, these types of projects routinely exceed their
budgets (often by ghastly amounts), take longer than expected, don’t
deliver required or expected functionality, or experience some combi-
nation of all of these.
There are terms for these types of traditional, rigid, and ultimately
unsuccessful information technology (IT) projects: phase-gate or Water-
fall. In a nutshell, a new phase or stage cannot begin until the team
has completed the prior one. You don’t need to be a certified project
manager professional to see the limitations of this approach, outlined
in Table I.1.

:31 AM cintro 3 22 May 2017 9:31 AM

 4 ▸ ANALYTICS

For instance, what if the people gathering the business require-

AU: Something ments miss a few key ones? This is a massive problem. In 2014, the
amiss in this Project Management Institute (PMI) released its Pulse of the Profession
sentence,
probably report. PMI found that “37 percent of all organizations reported inac-
to do with
curate requirements as the primary reason for project failure.” Less
“requirements
management” than half of organizations possess the resources to properly manage
phrasing. Maybe
a “from” between
their requirements. Only one-third of organizations’ leaders consider
these two words? requirements management to be critical.1

Beyond that, other questions abound:

■ What if a user fails to disclose an essential data source?

■ What if a key employee leaves the company?
AU: As in ■ What if there’s a bug in the software?
Table I.1....
■ What if step 3 (data conversion, cleansing, and deduplicating)
Marking not
clear takes longer than expected? What if this isn’t nearly as simple as
the software vendor and/or consulting firm intimated?

The answer to each of these questions is the same: The project

won’t hit its date without some type of concession. Typically, these
take the form of increased resources, reduced functionality, or new
(read: postponed) deadlines.
Lest you think that the aforementioned scenarios are rare, I’ve
seen all of them happen multiple times. In fact, Waterfall projects typi-
cally suffer from multiple issues at each phase. Their cumulative effects
have resulted in untold billions in wasteful spending, high-profile law-
suits, and executives looking for new jobs.
But I’m going to give you the benefit of the doubt. What if you
never experience the problems endemic to Waterfall projects? What if
all employees on the project predict with eerie certainty how long their
tasks take? Everyone is on board, and organizational politics doesn’t
impede the project’s progress. Is the new BI application guaranteed—
or even likely—to produce the kinds of lofty results that the software
vendor promised before money changed hands?
What happens when—not if—the world just changes? One year or
more represents an eternity, especially today. Put differently, no lon-
ger do just about all organizations and their leaders think of analytics
in a virtually identical manner.

cintro 4 22 May 2017 9:31 AM cintro

 INTRODUCTION ◂ 5

Prominent consulting firms have taken notice. To this end, they

have begun offering “rapid deployment” options. For instance, on
September 13, 2016, analytics company Teradata debuted something
called Rapid Analytic Consulting Engagement. From the company’s
press release:

Teradata (NYSE: TDC) . . . today announced RACE,

or Rapid Analytic Consulting Engagement, an agile,
technology-agnostic methodology that gives clients insight
into the potential business value of analytic solutions before
an investment is made. Teradata’s process fully delivers on
commitments in just six to ten weeks, rather than months.2

In point of fact, Teradata is hardly the only firm to realize the futil-
ity of yearlong engagements without any payoff in the near future.
Irrespective of moniker, the futures of Agile methods and of analytics
are inextricably intertwined.

ANALYTICS AND THE NEED FOR SPEED

In most mature organizations of certain sizes, the process of devel-

oping any analytics remains downright painful, let alone meaning-
ful ones. These projects resemble death marches, and their existence
couldn’t be more inopportune. Never before has the world moved as
fast as it does today.
To be sure, many people feel this way. You may, and I certainly do,
but forget what we think and feel. Let’s look at some data.
Celebrating the 25-year anniversary of the World Wide Web in
March 2014, The Economist examined the increasingly rapid adoption
of new technologies. From the article:

It took only seven years from the first web pages in 1991
for the web to be used by a quarter of the American
population. That compares with 46 years for electricity,
35 years for the phone and 26 years for television. The
web, just 25 years old, is still at the start of its life.3

In important and perhaps irreversible ways, the speed of both

business and life has intensified considerably over the past 10 years.

:31 AM cintro 5 22 May 2017 9:31 AM

 6 ▸ ANALYTICS

For instance, up until fairly recently, the notion of a flash sale didn’t
exist. Ditto for viral videos and trending tweets. If you happen to see
the front page of the morning’s newspaper (remember those?), it’s a
good bet that you’ve already heard about the top stories. Thanks to
the ubiquity of contemporary technology, all signs point toward these
trends continuing for the foreseeable future.

S P E E D A ND S TR ATE G Y
First-mover advantage has served some firms exceptionally well. Amazon
is a classic case study. From its inception, Jeff Bezos eschewed profits
for sales and scale. No one can credibly argue with his strategy and
its results. Kozmo, Pets.com, and other dot-com start-ups squandered
hundreds of millions of dollars when they should have followed leaner,
more capital-efficient approaches.
Other firms have done well by following their peers quickly. Facebook
was hardly the first social network. Many people forget that Goto.com
pioneered the pay-per-click model for monetizing search engines, not
Google. UPS carefully watches what rival FedEx does and learns from the
latter’s mistakes. UPS then creates similar offerings.
Examples abound of companies that waited too long to enter markets.
Consider Bing in search, the very definition of an also-ran. Amazon’s
2014 Fire Phone failed to gain any traction. For a more obscure example,
consider Ello. Launched in March 2014, the company arrived at the end of
the social network soiree. Despite its initial hype, Ello today is very much
a niche product,4 hardly the Facebook rival that it had imagined.
The point here is one management gambit or type of product launch isn’t
innately better than another. These brief examples prove that one size
does not fit all. As I’ve said many times before, you don’t need to be first,
but you certainly can’t be last.* Try launching a social network, search
engine, or smart phone today. Good luck with that.

*
http://bit.ly/2mSyKDJ.

How Fast Is Fast Enough?

The answer is neither simple nor universal. At the risk of sounding like
a stereotypical consultant, it depends. Retailers need to precisely track

cintro 6 22 May 2017 9:31 AM cintro

 INTRODUCTION ◂ 7

inventory in the period around—and certainly during—Black Friday.

Do they possess that same need during a slow Saturday in July? What
about for a pharmaceutical company or a nonprofit?
As you’re waiting in your doctor’s office, you might think that
relatively stable industries like health care don’t require real-time ana-
lytics—and you’d often be wrong. As Google has shown, by aggregat-
ing search results, data can prevent the flu and even the spread of
infectious diseases.5
Franz Kafka once said, “Better to have, and not need, than to
need, and not have.” Although he died nearly a century ago, that
axiom is particularly relevant today. The ability to move quickly is
more essential than ever—even if it’s not necessary at a given time.
For instance, for decades Cisco has had the capability to close its
books at the end of every business day. This is no small feat for such
a large enterprise. Does the company always do this? Of course not,
but the ability to easily accomplish this feat can be absolutely invalu-
able for diagnosing problems, assessing profitability, and answering
business questions before the end of a day, week, month, quarter, or
fiscal year.
Let’s return to the question: With regard to analytics, how fast is
fast enough today? Before answering, consider the following queries
that will help you address your organization’s need for speed:

■ How quickly do your employees act on currently existing num-

bers? Real-time analytics don’t mean a hill of beans if an orga-
nization’s employees never use data to make decisions.
■ Are there perils to acting too fast? In the race for speed, are you
losing time to reflect on what the data isn’t telling you?
■ Does your organization’s culture tolerate failure? If you act
faster, then you are able to benefit in key areas. At the same
time, though, you are also likely to make more mistakes.

No one can give you a simple, all-encompassing answer to these

questions—myself included. I can say, however, that traditional meth-
ods to obtain analytics are almost always far too slow.
Fortunately, there’s good news. Many folks are waking up. They
have realized that the Waterfall method simply cannot meet the
demands of modern business and life. They are eschewing drawn-out

:31 AM cintro 7 22 May 2017 9:31 AM

 8 ▸ ANALYTICS

IT death marches. In their place, organizations are adopting Agile meth-

ods to further their analytics goals.

Automation: Still the Exception That Proves the Rule

In his 2015 book Flash Boys, Michael Lewis of Moneyball fame details
how Wall Street firms are spending billions of dollars to lay fiber. No,
they are not doing pro bono public-works projects; their motivations
are far less benevolent. They are trying to shave a few milliseconds off
the time required to trade securities and derivatives. This phenom-
enon goes by the name high-frequency trading (HFT). Make no mistake:
HFT means big money. Beyond these sizable investments, hedge-fund
managers have filed lawsuits against one another for poaching key
employees who have intimate knowledge of these proprietary, com-
plicated, and very profitable algorithms.
At their core, these algorithms make trades far faster than we
humans can. Still, traffic in a slow lane can mean billions of dollars in
lost profits. Those few milliseconds can make all the difference.
Along these lines, programmatic ad buying refers to the increas-
ing use of sophisticated software that automatically purchases digital
advertising based on a panoply of rules. The days of slick presentations
by Don Draper of Mad Men appear to be waning. Writing for digital-
marketing agency Digiday, Jack Marshall noted:

It’s impossible to tell what portion of advertising is now

traded programmatically, but it’s definitely on the rise.
Some agencies now say they’re eager to buy as much
media as possible through programmatic channels, and
some major brands have even built out in-house teams to
handle their programmatic ad buying as they spend more
of their marketing budgets that way.6

HFT and programmatic ad buying give plenty of policy wonks

and everyday people pause about the future of jobs. Driverless cars
are coming, and, in general, artificial intelligence (AI) continues to
encroach on our lives. As Nicholas Carr manifests in his book The Glass
Cage, this trend shows no signs of abating. More automation isn’t nec-
essarily a good thing for society.

cintro 8 22 May 2017 9:31 AM cintro

 INTRODUCTION ◂ 9

Even instances in which humans have replaced machines haven’t

always gone smoothly. Consider Facebook. It has struggled for years
in determining which information NewsFeed ought to present to its
users and in which order. NewsFeed initially required people to select
which stories its billion-plus users would see, but it was only a matter
of time before Mark Zuckerberg would let its vaunted algorithm finally
assume the reins. On August 29, 2016, Zuckerberg decided that it was
time for machine to replace man. The social network unceremoni-
ously canned its trending team. It didn’t take long for its human-free
algorithm to go “crazy.”7
At least you can take some solace in the fact that AI represents the
exception that proves the following rule: Despite the fascinating and
often unexpected insights that they reveal, analytics and data gener-
ally do not make decisions by themselves. For the most part, human
beings do. At least for now, we generally and actively decide what—
if anything—we do when confronted with analytics. And it is here
where the wheat separates itself from the chaff.

TIP
Data and analytics rarely make decisions by themselves. People still
matter, and they can routinely increase their odds of success by intel-
ligently using data and analytics.

BOOK SCOPE, APPROACH, AND STYLE

I’m a big believer in the concept of truth in advertising. To this end,

allow me to explain what Analytics: The Agile Way attempts to achieve
and how the book attempts to achieve it.
At a high level, the following pages explore how firms are eschew-
ing dated Waterfall or phase-gate methods to make sense out of vast
amounts of data and an increasing array of data sources. They are
turning to nimbler methods such as Scrum. More than ever, there’s
tremendous upside to using analytics to act quickly—and significant
risks to waiting too long.

:31 AM cintro 9 22 May 2017 9:31 AM

 10 ▸ ANALYTICS

To achieve this goal, these pages cover quite a bit of ground. As I

have in all of my books, I again will attempt to synthesize seemingly
disparate business technology and events and trends into a cohesive
narrative. Analytics: The Agile Way is no exception. Along these lines,
expect far more practice than theory.

Breadth over Depth

By design, Analytics: The Agile Way is a multidimensional book. It does

not explore all of the ins and outs of any one topic. For instance, con-
sider the topic of Chapter 5: Scrum. A quick Amazon search reveals
nearly 1,800 books on that subject alone. Analytics: The Agile Way
doesn’t purport to cover Scrum in nearly the same level of detail as
many of those tomes.
And it doesn’t stop there. You can find plenty of insightful books
on just about all of the other subjects in this book: building statistical
models, learning how to program in different languages, and develop-
ing a variety of analytics.* To varying degrees, Analytics: The Agile Way
delves into these subjects. Those looking for an in-depth, 10-step pro-
cess for using analytics to transform specific industries such as health
care, however, will be disappointed.
This is a conscious choice and, because of it, I reference quite a
few valuable texts, videos, blog posts, and articles. You can ignore the
footnotes, notes, and bibliography if you like, but they are rife with
informative and complementary resources. In other words, I hope that
this book is a starting point to further exploration.

Methodology: Guidelines > Rules

Analytics: The Agile Way does not advocate strictly adhering to any one
development method. Yes, Part Two introduces Agile methods (spe-
cifically Scrum) as well as a simple six-step framework for deriving
analytics. Still, this does not mean that achieving success with data and
analytics is tantamount to baking a cake. It’s not.

*
Amazon currently sells 166 books related to “healthcare analytics” alone.

cintro 10 22 May 2017 9:31 AM cintro

 INTRODUCTION ◂ 11

To glean profound insights via data and analytics—especially when

employing Agile methods—requires guidelines more than proper
rules. To borrow from the title of the 2016 book by Ben Lindbergh
and Sam Miller on applying analytics to baseball, the only rule is that
it has to work. Different business problems, industries, environmental
factors, budgets, employee skills, scenarios, and types of data often
mandate wildly different approaches. A given technique in one con-
text may be ill advised. In another, it may be completely appropriate
and even necessary. What worked for Company A often does not work
for Company B.*

Technical Sophistication

As I know all too well, poorly configured or antiquated systems

directly impede firms’ efforts to harness one of their most valuable
assets: data. To keep this book at a manageable length, I largely omit
the technical and often arcane side of enterprise data. Readers won’t
find much advice on data modeling, normalization, and warehousing.†
Aside from the occasional recommendation, the same holds true for
related and important subjects such as database schemas and design,
master data management, enterprise architecture, and technical pro-
cesses such as extract, transform, and load. Each of these meaty topics
is certainly important. Anyone who doesn’t believe that they affect
enterprise data is sorely mistaken. Nevertheless, they lie outside of the
scope of this book.

Vendor Agnosticism

On many levels, ours is a unique era. We have never had more data at
our disposal—often an overwhelming amount flying toward us with
increased speed. Data may or may not be “the new oil,” but we cannot
dispute its value when used correctly.‡ Ditto for analytics.

* Chapter 4 will return to this subject.

†
If you want to know more here, see Ken Collier’s Agile Analytics: A Value-Driven Ap-
proach to Business Intelligence and Data Warehousing.
‡
For my thoughts on the matter, see http://bit.ly/2naMMSc.

:31 AM cintro 11 22 May 2017 9:31 AM

 12 ▸ ANALYTICS

Fortunately, there have never been more ways to capture, buy, or

rent this data. We can use powerful and affordable tools to analyze it,
interpret it, and act on its insights. (Whether or not we do this remains
to be seen.) Relatively new data-analysis programs, tools, and frame-
works such as Tableau, R, Stata, D3, Hadoop, Hive, Pig, import.io, and
countless others have exploded in popularity. Today, these accompany
stalwarts such as Microsoft Access and Excel, SPSS, SAS Enterprise
Miner and JMP, Minitab, and others.
Some of these tools are doubtless better suited for specific uses than
others are. No one tool, programming language, or development frame-
work can do everything. Still, this book is vendor-agnostic by design.
It does not advocate any one software application over another. (Don’t
let the Teradata press release from earlier in the chapter fool you.)
The objectively “best” or “perfect” analytics tool—assuming that
one even exists—will not guarantee successful results by itself. The
firms and individuals best poised to find the signals in the noise need
more than the latest tech and data tchotchkes. An Agile mind-set and
a willingness to adapt quickly to changing circumstances are essential
for little victories, never mind the big ones.
Yes, we’ll cover examples of individual applications. By no means,
though, am I implying that one is inherently better than another.
Anyone looking for specific product recommendations will be disap-
pointed, as will those seeking detailed application how-to guides and
tutorials.

INTENDED AUDIENCE

More than ever, we need to do more than merely understand the

insights that data can offer. We need to act on them. I wrote Analytics:
The Agile Way for individuals interested in learning how to system-
atically turn data into something meaningful: analytics. More specifi-
cally, the book’s intended audiences include the following four groups:

1. Undergraduate and graduate students new to the practice

of analytics. (The questions at the end of each chapter are
designed to provoke independent thought and in-class discus-
sions more than supposedly correct answers.)

cintro 12 22 May 2017 9:31 AM cintro

 INTRODUCTION ◂ 13

2. Functional users supporting different lines of business. These

include entry-level analysts and senior executives. Many of
these folks have only developed and deployed analytics in a
more traditional manner.
3. Traditional analytics professionals who are rightfully curious
about Agile methods.
4. Employees in technical fields who want to support their non-
technical, internal clients better.

Regardless of your current role, though, if you’re curious about

trends in data and analytics and open to new ideas, then you should
enjoy this book.

A R E C O MME ND ATIO N F O R EM PL OYE E S AT

STA RT-U PS
Analytics: The Agile Way is not geared toward a particular industry
or company size. I like to think that its lessons apply irrespective of
where you work. If you’re looking for a book about the specific analytics
challenges that start-ups face, check out Lean Analytics: Use Data to
Build a Better Startup Faster by Alistair Croll and Benjamin Yoskovitz.

PLAN OF ATTACK

Analytics: The Agile Way consists of five parts. Part One (Background
and Trends) lays the groundwork for the entire book. It describes the
current data- and tech-centric business environment and how we
arrived here. It also includes foundational chapters on data and ana-
lytics. After all, success begins with a common understanding of terms.
Part Two (Agile Methods and Analytics) begins with a short chap-
ter on Agile principles. It then moves on to Agile methods with a par-
ticular focus on Scrum. It next introduces a simple six-step framework
for Agile analytics that readers can apply to any number of analytics
projects.
Part Three (Analytics in Action) provides windows into the analyt-
ics efforts of specific organizations. Here it’s time for case studies. We’ll

:31 AM cintro 13 22 May 2017 9:31 AM

 14 ▸ ANALYTICS

see how progressive firms are rapidly and incrementally understand-

ing their business problems—and developing solutions to them. That
is, they have broken the inimical patterns that plague so many tradi-
tional analytics efforts.
Part Four (“Making the Most Out of Agile Analytics”) takes a step
back. It explores the lessons, benefits, and limitations of Agile analyt-
ics. You’ll learn that the types of data and analytics available to you
often hinge on product design. As such, Part Four concludes with a
chapter on essential design considerations.
Part Five (“Conclusions and Next Steps”) concludes the book. It
asks some pointed questions about issues related to the growing use
of Agile analytics. It covers new challenges with respect to data gover-
nance and data exhaust, privacy, and security.

CHAPTER REVIEW AND DISCUSSION QUESTIONS

■ What are Waterfall or phase-gate projects?

■ Do they lend themselves to successful outcomes? Why or why not?
■ What are a few examples of automated decision making? Is this currently the
exception or the rule? Do you expect that to change?

NEXT

Now that you know what this book covers and how, it’s time to answer
another big question: Why? The next chapter places analytics into a
contemporary business context. It describes the major trends that have
elevated analytics from optional to essential.

NOTES
1. Elizabeth Larson, “I Still Don’t Have Time to Manage Requirements: My Pro-
ject Is Later Than Ever,” Project Management Institute, October 2014, http://bit
.ly/2mjU6KR.
2. “Teradata Debuts Agile Analytics Business Consulting,” September 13, 2016, http://
tinyurl.com/zna6lan.

cintro 14 22 May 2017 9:31 AM cintro

 INTRODUCTION ◂ 15

3. “Happy Birthday World Wide Web,” The Economist, March 12, 2014, http://tinyurl
.com/jm2ltqw.
4. Charley Lock, “Remember Ello? You Abandoned It, but Artists Didn’t,” May 17,
2106, Wired, http://bit.ly/2eOhUmf, retrieved March 22, 2017.
5. Mary Chris Jaklevic, “Disease Sleuths Analyze Google Searches to Stop Infections,”
NPR, December 10, 2015, http://n.pr/1SQF742.
6. Jack Marshall, “WTF Is Programmatic Advertising?” Digiday, February 20, 2014,
http://tinyurl.com/z37vql5.
7. Sam Thielman, “Facebook Fires Trending Team, and Algorithm without Humans
Goes Crazy,” The Guardian, August 29, 2016, http://bit.ly/2bMBfVv.

:31 AM cintro 15 22 May 2017 9:31 AM