Archive by Author | ali swanson

Wander over to Wildebeest Watch!

Wildebeest Watch Home

Can’t get enough of these gnarly gnus? Head on over to our new spinoff project, Wildebeest Watch!

In collaboration with Dr Andrew Berdhal from the Santa Fe Institute, and Dr Allison Shaw at the University of Minnesota, we are taking a closer look at what the wildebeest are doing in the Snapshot Serengeti images to try and better understand the details of the world’s largest mammal migration.

Every year, 1.3 million wildebeest chase the rain and fresh grass growth down from the northern edge of the ecosystem down to the short grass plains in the southeast. We have a broad-scale understanding of where they are moving across the landscape, but don’t understand how they make these detailed decisions of where and when to move on a moment-to-moment basis. Wildebeest as individuals aren’t known for being particularly smart — so we want to know how they use the “wisdom of the crowd” to make herd-level decisions that get them where they need to go.

So while you’re waiting for more photos of lions, hyenas, and other sharp-toothed beasts, why not wander over to Wildebeest Watch to help us understand the collective social behavior of these countless critters?

Snapshot Serengeti’s first scientific publication — today!

Yay! Says cheetah.

“Yay!” Says cheetah.

Champagne corks will be popping tonight. Snapshot Serengeti’s first peer reviewed scientific publication comes out today in Nature’s Scientific Data journal. Please give yourselves a round of applause, because we’d never have been able to do this without you.

The paper is a “data descriptor” instead of a traditional research article, meaning that we describe the detailed methods that led to the Snapshot Serengeti consensus dataset. In addition to describing all the excrutiating details of how we set the cameras in the field, we talk about the design of Snapshot Serengeti, setting retirement rules and aggregation algorithms to combine all of our answers into a single expert-quality dataset. We don’t talk about the cool ecological results just yet (those are still only published in my dissertation), but we do talk about all the cool things we hope the dataset will lead to. The dataset is publicly available here. Anyone can use it — to ask ecological questions about Serengeti species, evaluate better aggregation algorithms for citizen science research, or — we get this a lot — use the images plus consensus data to train and test better computer recognition algorithms.

Feel free to download the dataset and explore the data on your own. We’d love to hear what you find!

Getting Good Data, Part II (of many)

Okay, so by now you’ve heard dozens and dozens of times that you guys produce really good data: your aggregated answers are 97% correct overall (see here and here and here). But we also know that not all images are equally easy. More specifically, not all species are equally easy. It’s a lot easier to identify a giraffe or zebra than it is to decide between an aardwolf and striped hyena.

The plot below shows the different error rates for each species. Note that error comes in two forms. You can have a “false negative” which means you miss a species given that it’s truly there. And then you can have a “false positive,” in which you report a species as being there when it really isn’t. Error is a proportion from 0 to 1.

Species specific error rates.

Species specific error rates.

We calculated this by comparing the consensus data to the gold standard dataset that Margaret collated last year. Note that at the bottom of the chart there are a handful of species that don’t have any values for false negatives. That’s because, for statistical reasons, we could only calculate false negative error rates from completely randomly sampled images, and those species are so rare that they didn’t appear in the gold standard dataset. But for false positives, we could randomly sample images from any consensus classification – so I gathered a bunch of images that had been identified as these rare species and checked them to calculate false positive rates.

Now, if a species has really low rates of false negatives and really low rates of false positives, then it’s one people are really good at identifying all the time. Note that in general, species have pretty low rates of both types of error. Furthermore, species with lower rates of false negatives have higher rates of false positives. There aren’t really any species with high rates of both types of error. Take rhinos, for example: folks often identify a rhino when it’s not actually there, but never miss a rhino if it is there.

Also: we see that rare species are just generally harder to identify correctly than common species. The plot below shows the same false negative and false positive error rates plotted against the total number of pictures for every species. Even though there is some noise, those lines reflect  significant trends: in general, the more pictures of an animal, the more often folks get it right!

Error rates vs. species commonness, measured by the total number of pictures of that species

Error rates vs. species commonness, measured by the total number of pictures of that species

This makes intuitive sense. It’s really hard to get a good “search image” for something you never see. But also folks are especially excited to see something rare. You can see this if you search the talk pages for “rhino” or “zorilla.” Both of these have high false positive rates, meaning people say it’s a rhino or zorilla when it’s really not. Thus, most of the images that show up tagged as these really rare creatures aren’t.

But that’s okay for the science. Because recall that we can assess how confident we are in an answer based on the evenness score, fraction support, and fraction blanks. Because such critters are so rare, we want to be really sure that those IDs are right — but because those animals are so rare, and because you have high levels of agreement on the vast majority of images, it makes it really easy to review any “uncertain” image that’s been ID’d as a rare species.

Pretty cool, huh?

Getting good data: part 1 (of many)

Despite the site being quiet, there’s a lot going on behind the scenes on Snapshot Serengeti at the moment. Season 8 is all prepped and currently being uploaded and should be online next week! And on my end, I’ve been busily evaluating Snapshot Serengeti data quality to try and develop some generalisable guidelines to producing expert quality data from a citizen science project. These will actually be submitted as a journal article in a special section of the journal Conservation Biology, but as that is a slowwwwwww process, I thought I’d share them with you in the meanwhile.
So! Recall that we use a “plurality” algorithm to turn your many different answers into a “consensus dataset” — this has one final answer of what is in any given image, as well as various measures of certainty about that image. For example, back in 2013, Margaret described how we calculate an “evenness score” for each image: higher evenness means more disagreement about what is in an image, which typically means that an images is hard. For example, everyone who looks at this photo
would probably say there is 1 giraffe , but we’d expect a lot more diversity of answers for this photo:
(It’s a warthog, btw.)
To test how good the plurality algorithm answers were, we created a “gold-standard dataset” by asking experts to identify ~4,000 images. Overall, we found that consensus answers from your contributions agreed with experts nearly 97% of the time!  Which is awesome. But now I want to take a closer look.  So I took all the images that had gold standard data and I looked at Evenness, Number of “nothing here” responses, and %support for final species, and evaluated how those measures related to whether the answer was right or wrong (or, impossible). Even though we don’t have an “impossible” button on the SS interface, some images simply are impossible, and we let the experts tell us so so these wouldn’t get counted as just plain “wrong.”
A note on boxplots: If you’re not familiar with a boxplot, what you need to know is this: the dark line in the middle shows the median value for that variable; the top and bottom of the boxes shows the 25 and 75 percentiles; and the “whiskers” out the ends show the main range of values (calculated as 1.5 * interquartile range, details here). Any outliers are presented as dots.
Evenness: The boxplot below shows the mean “Evenness” score described above vs. how the consensus answer matched the gold standard answer.   What you can see below is that the average evenness score for “correct” answers is about 0.25, and the average evenness score for wrong and impossible answers is 0.75. Although there are some correct answers with high evenness scores, there are almost no wrong/impossible answers with evenness scores below 0.5.
 EvennessVsAccuracy
Percent Support: This number tells us how many people voted for the final answer out of the total number of classifiers. So, if 9 out of 10 classifiers said something was a giraffe, it would have 90% support. It’s similar to evenness, but simpler, and it shows essentially the same trend. Correct answers tended to have more votes for what was ultimately decided as the final species.
PercentSupport
NumBlanks: So with the evenness and percent support scores, we can do a decent job of predicting whether the consensus answer for an image is likely to be right or wrong. But with Number of Blanks we can get a sense of whether it is identifiable at all. Margaret noticed a while back that people sometimes say “nothing here” if they aren’t sure about an animal, so the number of “nothing here” votes for an image ultimately classified as an animal also reflects how hard it is. We see that there isn’t a huge difference in the number of “nothing here” answers on images that are right or wrong — but images that experts ultimately said were impossible have much higher average numbers of “nothing here” answers.
 NumBlanksVsAccuracy
So, what does this tell us? Well, we can these metrics on the entire dataset to target images that are likely to be incorrect. In any given analysis, we might limit our dataset to just those images with >0.50 evenness, or go back through all those images with >0.05% evenness to see if we can come up with a final answer. We can’t go through the millions of Snapshot Serengeti images on our own, but we can take a second look at a few thousand really tricky ones.
There’s all sorts of cool analyses still to come — what species are the hardest, and what’s most often mistaken for what. So stay tuned!

Everybody dance now!

Formerly titled: I love the Zooniverse, and all its ridiculousness.

Some of you might be familiar with the silly manoeuvres we undertake to trigger the camera traps in the field. The sensors on the cameras are best at capturing movement side-to-side, hence ridiculousness such as this:

flcd4

The Stan Dance

and this…

flcgb

Craig surprising the camera

and this.

“The Ali Dance”

Well, fellow Zoo-ites Rob and Grant have now generously attempted their own version of this last one. Check it out on the Zooniverse Advent Calendar here!! And, if you’re feeling especially brave, send in your own recreation to team@zooniverse.org!

The predator of my predator is my friend.

If you’re a prey animal, you spend an awful lot of your time trying to not wind up like this:

lions_ngm_0813_006_brnd

 

As we’ve talked about an awful lot on this blog (here, here, and here, for example), the same holds true for a lot of predators. Just because you kill and eat other animals, doesn’t mean you don’t have to worry about being killed yourself (as this hyena so unceremoniously discovered).

But, what we haven’t talked so much about, is that the same holds true for plants. If you’re a plant, you get eaten by these terrifying animals:

But, just like prey animals and mesopredators can change their behaviour to minimise the risk of being killed, plants have a few tricks up their sleeves. They can spend a lot of energy growing big thorns, for example, that makes them less delectable.

Or! They can grow in places that their predators avoid — the places where their predators’ predators hang out. Got that?  It’s a trickle-down landscape of fear, which had, until now, only been really well documented in small experimental systems with critters like spiders and grasshoppers. But researcher Dr. Adam Ford and colleagues just published an elegant paper in Science showing that leopards and African wild dogs can make the Kenyan savanna less thorny through this cascade. Basically, leopards and wild dogs eat impala. Impala eat Acacia trees. Impala much prefer to eat acacias with fewer thorns (because really, who doesn’t?) – and, if given the opportunity, impala will can eat these small-thorned acacias so much that they can suppress the acacia population.

But! Leopards and wild dogs seem to be offering these tasty small-thorned acacias a refuge. Leopards and wild dogs spend most of their time in denser thickets, where they have more cover to hunt. Impala avoid these thickets and rarely venture in —  when they do, however, they have a much higher probability of being killed. And this creates this spiral – those tasty small-thorned trees survive and grow in these thickets because predators scare impala away.

So it’s a trickle down landscape of fear – a compelling and really exciting story. But, what sets Adam’s paper apart from many other attempts to document this effect in large predators, is the series of elegant experiments in which he and colleagues explicitly tested each step in this cascade.  Controlling for habitat use to confirm that impala aren’t getting killed in the woods simply because they spend more time there (and in fact, they get killed more even though the spend less time there). Adding and removing thorns to acacias to see if it was really the thorns that mattered. Creating herbivore exclosures to measure whether impala could really suppress acacia density. I spent my entire time reading the article alternating between saying “This is so cool!” and “I am so jealous!” It’s an amazing story. Read more about it here (or here, or or here)!

 

Zooniverse advent calendar

At the Zooniverse, we love any excuse to have fun, be festive, and highlight cool things from our awesome projects. Check out the Christmas countdown Advent Calendar at https://www.zooniverse.org/advent!

Welcome Darren

Darren, our new Community Builder

Darren, our new Community Builder

Hey everyone – I just wanted to introduce you to one of the Zooniverse’s newest members, Darren McRoy, who is our new Community Builder. 

As Community Builder, Darren serves as the Zooniverse’s liaison with its citizen science community, including Snapshot Serengeti. He also assists with Zooniverse’s general communications efforts and is working closely with designers and developers on the next generation of the Talk discussion system.

Darren is a 2010 graduate of the Medill School of Journalism at Northwestern University and has a background in journalism and digital communications. He is a resident of the northern Chicago suburbs, and enjoys golf, volleyball, fiction writing, gaming, and participating in a variety of online communities.

Darren can be reached at darren@zooniverse.org, and posts on Talk under the handle “DZM.” Please feel free to get in touch with him if you have any thoughts, comments, questions, etc about how the Zooniverse communicates with its community — you!

Groovin’

Just for fun. I was hanging around on Talk today and stumbled across this grooving Kori Bustard. Kind of makes me want to dance…

dancin' kori bustard

dancin’ kori bustard

Serengeti Lions win Wildlife Photo of the Year award!

lions_ngm_0813_006_brnd

You might remember that National Geographic did a big story on “our” lions last year. David Quammen spent a while being bounced around in our land rovers and Nick Nichols and his crew spent months on end camping in the southeastern corner of our study area, following the Vumbi pride’s every move.

Well, one of Nick’s pictures from that trip has just won him the prized Wildlife Photographer of the Year award!   The competition, co-run by London’s Natural History Museum and the BBC, has just turned 50 years old, and is a pretty big deal (you can read about it here). Nick’s winning piece is a black and white photo of the Vumbi pride sprawled in rather epic fashion over the kopjes. We can’t post the picture here for copyright purposes, unfortunately, but go check it out!  And go check out some of the other fantastic runners-up here while you’re at it.