Brad Hargreaves | Building Things

Brad Hargreaves on entrepreneurship, community and life

Archive for the ‘science’ tag

Experimenting on Human Beings

without comments

Since the Renaissance, scientific experiments were the domain of Science. And Science had its own way of doing things. Science derived questions from first principles, proposed hypotheses that may answer those questions, and designed the methodologies that would prove or disprove those hypotheses.

That’s how Science worked, and nothing else worked quite like it. Business was art; Business was Don Draper doodling his visions on a cocktail napkin. Politics was even further afield, an art buried in the fog of war. Science and Science alone ran real experiments.

But then some Bad Things were done in the name of Science. Things that shook the foundations of Science and threatened to bring down the whole operation. To prevent those Bad Things from happening again, Science began regulating itself, introducing concepts like Informed Consent and requiring experiments to be approved by Institutional Review Boards. Most people agree that those changes were for the better.

But then other people started figuring out what Science had been doing all this time. Finance was probably the first to find it, bringing experimental methodologies to trading in the late ’80s and early ’90s. Advertising soon followed suit and has become overwhelmingly sophisticated in the past ten years. Obama’s 2008 campaign was a watershed moment for Politics, with rigorous message testing and voter data methodologies at work for the first time.

One by one, domains of Business and Politics adopted the Scientific Method. Their subjects, naturally, were human beings. But since Science only regulated itself – not everyone who used its methods – these new domains don’t have the same procedures and oversights. Any organization with an audience could run experiments on them to see what message, user flow, landing page, ad campaign or button color worked the best. Some of these experiments could have a meaningful and lasting impact on their subjects – when an OkCupid test works, more people hook up, resulting in marriage, kids, STDs, emotional trauma and all kinds of chronic effects that would put any IRB into a tizzy.

And now Facebook is getting FTC attention for doing something every savvy web-based company has done in the past fifteen years. And OkCupid and others are defending them, claiming (correctly) that everyone does it.

Now everyone doing it doesn’t make it okay, per se. But it does raise questions of what “informed consent” means. One could argue that signing up for Facebook and accepting friend requests is consenting to seeing whatever those friends may post, even if Facebook is presenting them in a way that is likely to evoke certain emotions. And the informed consent argument is even stronger with OkCupid; they’re not trying to hide the purpose of their site from prospective users.

Of course, the specific methodologies of informed consent aren’t anywhere in play. There’s no paperwork or review, just a ToS that no one ever reads.

Perhaps the problem here is not that these companies are running experiments without informed consent, but that the implementation of informed consent needs to be totally re-thought when applied to Business and Politics. Signed forms and IRBs aren’t going to work – we need a methodology that fits companies that run thousands of experiments every day and test hundreds of variables, the vast majority of which have little to no lasting impact on their users.

Perhaps rather than going after Facebook, informed consent advocates would be better off tackling the issues with ToS and getting companies (and users) to care about the documentation that already exists rather than adapting new business practices to an old scheme that was created to solve different problems.

Written by Brad Hargreaves

July 28th, 2014 at 9:23 am

Posted in Uncategorized

Tagged with , ,

An Agile Approach to Science Education

with 5 comments

As an entrepreneur, a big part of my job is figuring out what people want and building products that meet those needs. Even if I think a product is really cool, I’m not going to invest time and money making it better if the market doesn’t seem to care. There’s a slim chance that I’ll fiddle around with the product long enough that I can get people to understand what they didn’t know they needed, but such is a fool’s errand unless there’s a clear path to success.

Unsurprisingly, people with backgrounds in iterative software development aren’t running education in America. It’s a shame, really, because I think consumer web startups could provide some good lessons to improving K-12 science education. Let me start with one premise: Our nation’s cultural values, especially in middle and high school environments, are strongly aligned against science and technology. And most distressingly — and has Dean Kamen has recognized — this is contagious. When a student’s most respected peer is the football captain, they are likely to realign their interests away from science and education and towards things that are less productive to society.

Yet like an entrepreneur without a good grasp of the audience, we continue to focus on shifting the product — fiddling around with different ways to present information — rather than the market. While there’s certainly value in iteration and superior presentation, I can’t really envision a secular change in performance and output taking place without a fundamental change in the market’s attitude toward science. We have to make science sexy to high-potential K-12 kids. All the product iteration in the world is for moot unless we can figure out a way to make smart students actually care about science, math and engineering.

Logically, there are two ways to make this happen:

Change the attitudes of society as a whole. This is Dean Kamen’s strategy with FIRST — turn science into a sport, engaging larger segments of the populace by framing science students in the same verbiage as football players.

Change the attitude of a subset of society, and immerse qualified science students in that subset. This is a controversial one, and — other than a few specialty schools such as TJHSST — isn’t commonly employed in a meaningful way.

While I love what FIRST is doing, I’m not convinced that the former is feasible. Getting hundreds of thousands of high school students engaged in competitive science — as FIRST has done — is awesome. But it’s not changing our culture’s attitude toward science as an unpopular, unsexy, geeky, male-dominated field. And there’s a decent argument to be made that such stereotypes are hard to dispel because they’re true. Fixing that problem — well, that’s another debate. Regardless, I don’t see brilliant science students gaining the fame, notoriety and sexiness of their peer athletes in my lifetime. And if you rely on the societal change model, this is a massive problem: science and engineering students have been promised a reward in exchange for their work that they’re not going to get. In other words, FIRST could be selling a lie.

Disturbingly, our best bet to stay competitive as a nation may be to ghettoize high-performing students, placing those with real potential to be our nation’s next generation of scientists and engineers in environments where their interests won’t be misaligned by the skewed perspectives of a nation fascinated by D1 college football and Justin Bieber. There are a lot of downsides to this proposition — namely, the fact that a majority of students are stuck in downward-spiraling groups of non-qualifying kids. But is this significantly different than our nation’s current private school structure, except with academic performance as opposed to financial means as the selector?

Regardless of the method used, I’d love to see our nation’s policymakers and educators focus a bit more on the market they’re trying to reach.

Written by Brad Hargreaves

April 25th, 2010 at 7:35 am