Curbing Idealistic Images and CDV for the Sake of Lion Conservancy

Meredith: Our brilliant team of Snapshot Serengeti undergraduate volunteers at the University of Minnesota are perhaps even more on top of the lion literature than I am! This week, we have a guest post from one such student, Clayton Mazur, describing some recent work of Dr. Packer’s on lion disease spread in Serengeti Park. This post is a synopsis of a scientific paper that can be accessed in full here

S3_L11_R11_ASG0008jkx_1

I propose we play a word-association game. I will offer a word and you think of what comes to mind. “Africa.” Did you imagine Mt. Kilimanjaro, or the towering, lush rain-forests of the Congo? “Wildlife.” Did you envision the sprawling savannas of Tanzania, home to hundreds of thousands of migrating wildebeest? If so, I would bet that your savanna also included the enigma of Africa: the African lion. Was he a graceful figure standing upon Pride Rock looking out over his kingdom? Perhaps he was laying in the shade, his dark mane flowing in the breeze as he waits for the females to return with a kill. I would argue that elegant images such as these are what come to mind for the majority the public. The portrayals of lions in the media- from Lion King to the MGM Lion- support this notion.  As elegant as these images are, reality is less than elegant for the lions living in Serengeti National Park in Tanzania, Africa.

African lions have unique social structures that help them brave the tough conditions of the savanna. Lions live in families called prides; one to two male lions rule a pride. The roles of female and male lions within a single pride are vastly different.  While females hunt, raise the cubs, and reproduce, male lions defend the pride from attack by other predators such as hyenas. Living in a large family group offers lions protection, but it also comes with costs. One cost is that females need to supply a large number of individuals with enough food for survival. Female lions coordinate hunts whereby they stalk prey and then give chase, but this technique only yields about 26% success. The low success rate forces large prides to split or starve.  Perhaps a more interesting difficulty of living in a large pride is the spread of disease within lion populations. As Dr. Craig Packer has found, disease prevalence is a threat to the current lion population.

A fatal disease that persists in the carnivores of Serengeti National Park is Canine Distemper Virus (CDV). CDV infects a range of carnivores, from dolphins to rodents and even some primates. The viral infection causes encephalitis, pneumonia, anorexia and eventually, death. You may be familiar with CDV if you own a dog, for many owners in the US vaccinate their dogs for CDV. In Serengeti National Park, where local villages cannot afford to vaccinate their dogs, CDV is a conservation concern for African Lions who contract CDV from domestic dogs. To try to remedy the concern, an intense vaccination regime started in 2003. Packer and colleagues attempted to characterize the progression of CDV in both domestic dogs and African Lions. Their goal was to determine if domestic dogs were responsible for the infections observed in African Lions. The team also wanted to determine if the 2003 vaccination program had any effect at reducing CDV in the domestic dogs and/or African Lions.

The scientists worked with blood plasma collected from both domestic dogs (obtained from 1992-2012) and from African lions (obtained from 1984-2012).  After collecting the blood samples, the team ran serological tests to detect for the presence of the CDV virus in individual dogs or lions. Using a Bayesian model, the scientists then calculated the probability that an individual lion or dog would contract CDV in one year. The scientists also used sensitivity models to determine the extent at which domestic dogs transmit CDV to lions. From the results of these models, the scientists were able to comment on the fate of the lions with regard to CDV.

The research team drew results by interpretation of the two models. They found that CDV had persisted in the populations of both dogs and lions for more than 25 years. Outbreaks of CDV occurred in 1981, possibly in 1976, and in 1993. Not only do these results suggest a historic presence of the fatal disease in the national park, the dynamics of each outbreak of the disease was unique. The scientists found that the year in which CDV infected the most dogs differed from the year in which CDV infected the most lions. The researchers proposed that this pattern identified domestic dogs as initial vectors for CDV in the park. After a 1994 outbreak in the lion population, the dynamics of the outbreaks become more disjoint. The disjointed dynamics between the domestic dog and lion populations suggest that after 1994, infections cycled through the dog population and the lion population separately. However, the spread of CDV between dogs and lions was not eliminated after 1994.

With domestic dogs established as an initial vector of CDV, the scientists wanted to know transmission rates between domestic dogs and lions. Again, they used a sensitivity model to predict this factor. The scientists found that domestic dogs were ten times more likely to spread CDV to lions than lions were to spread CDV to domestic dogs. With such a high prevalence of CDV in the domestic dog population and the tendency for CDV to spread from dog to lion, the effects of the 2003 vaccination effort were an important factor to analyze for the conservation of the Serengeti National Park lions. The scientists first analyzed the effects of the vaccine on the domestic dog population. Before 2003, there had been very sparse vaccination in villages surrounding Serengeti National Park. As was expected, this vaccination effort did little to curb CDV infection in either lions or domestic dogs. It was not until after 2003 when all villages to the east of Serengeti National Park and all villages within 10 km to the west of Serengeti National Park vaccinated their dogs against CDV did there exist a decrease (~5%) in CDV infections.

With CDV slightly decreased in domestic dogs due to the vaccination effort, was there a similar decrease in CDV infections in the lion populations?  Unfortunately, the sample size of lion serum that the scientists could obtain was not enough to comment on the updated magnitude of dog-lion CDV transmission. Overall, the scientists determined that CDV was still able to cycle in the lion population with very little reduction in the prevalence of the disease. However, not all is hopeless for the lion populations of Serengeti National Park. Dr. Packer’s team suggests that direct vaccination of lions may be more effective at preventing the disease. Additionally, the team suggests that advances in serological techniques would allow for increased accuracy when researching episodic diseases, such as CDV. Implementation of safe vaccines coupled with more accurate serological tests could minimize the effects of CDV outbreaks and ensure the health of the Serengeti lions.

As evident from the work of Dr. Packer and colleagues, there are threats to the conservation of the lion populations of Serengeti National Park. Not only do prides run the risk of individuals starving to death, splitting, and human-lion conflict, disease is another risk factor of living in a pride. Yet, these prides, these perfect families, come to mind when the public thinks about lions. Idealistic images of cubs play fighting or suckling from their mother are important for generating interest and compassion for African lions. One can be content with the image of the brave, courageous, elegant male lion standing on Pride Rock overlooking his kingdom, but one must simultaneously recognize the reality of the lion’s plight. Only then will conservation for lions be truly feasible.

Back to the Field!

photo (2)

Good news: thanks to funding from National Geographic, we’re heading back out to Tanzania with some new camera traps for Snapshot Serengeti!

It’s a bit short notice, but I’ll be heading back out to the field in just under two weeks to dive back into camera maintenance and data collection. I’ve been frantically ordering field equipment and gathering together all the supplies I need in Serengeti, including 50 cameras and what feels like twice my weight in rechargeable batteries. I’ll be adding new cameras back in to the grid to replace those that have been damaged or stolen, in addition to following up on some playback experiments I conducted last summer and continuing to monitor changes in the habitat around each of our camera sites. Some new data that we’ll be picking up this year include examining changes in the soil quality throughout the camera trap set-up and characterizing diversity in the plant communities in the immediate vicinity of our camera traps. Both of these factors contribute to forage quality for our ungulates and affect how appealing a particular site is for different animal species. I might even attempt to collect samples of dung (ah, the glamour of field work) from around our cameras to see whether we’re actually catching in our photos all the animals hanging out in these areas.

After a few months in Tanzania, I’ll be heading down to South Africa to conduct additional experiments in a small private reserve in the Kalahari. Look forward to updates from the field, and wish me luck!

 

Getting Good Data, Part II (of many)

Okay, so by now you’ve heard dozens and dozens of times that you guys produce really good data: your aggregated answers are 97% correct overall (see here and here and here). But we also know that not all images are equally easy. More specifically, not all species are equally easy. It’s a lot easier to identify a giraffe or zebra than it is to decide between an aardwolf and striped hyena.

The plot below shows the different error rates for each species. Note that error comes in two forms. You can have a “false negative” which means you miss a species given that it’s truly there. And then you can have a “false positive,” in which you report a species as being there when it really isn’t. Error is a proportion from 0 to 1.

Species specific error rates.

Species specific error rates.

We calculated this by comparing the consensus data to the gold standard dataset that Margaret collated last year. Note that at the bottom of the chart there are a handful of species that don’t have any values for false negatives. That’s because, for statistical reasons, we could only calculate false negative error rates from completely randomly sampled images, and those species are so rare that they didn’t appear in the gold standard dataset. But for false positives, we could randomly sample images from any consensus classification – so I gathered a bunch of images that had been identified as these rare species and checked them to calculate false positive rates.

Now, if a species has really low rates of false negatives and really low rates of false positives, then it’s one people are really good at identifying all the time. Note that in general, species have pretty low rates of both types of error. Furthermore, species with lower rates of false negatives have higher rates of false positives. There aren’t really any species with high rates of both types of error. Take rhinos, for example: folks often identify a rhino when it’s not actually there, but never miss a rhino if it is there.

Also: we see that rare species are just generally harder to identify correctly than common species. The plot below shows the same false negative and false positive error rates plotted against the total number of pictures for every species. Even though there is some noise, those lines reflect  significant trends: in general, the more pictures of an animal, the more often folks get it right!

Error rates vs. species commonness, measured by the total number of pictures of that species

Error rates vs. species commonness, measured by the total number of pictures of that species

This makes intuitive sense. It’s really hard to get a good “search image” for something you never see. But also folks are especially excited to see something rare. You can see this if you search the talk pages for “rhino” or “zorilla.” Both of these have high false positive rates, meaning people say it’s a rhino or zorilla when it’s really not. Thus, most of the images that show up tagged as these really rare creatures aren’t.

But that’s okay for the science. Because recall that we can assess how confident we are in an answer based on the evenness score, fraction support, and fraction blanks. Because such critters are so rare, we want to be really sure that those IDs are right — but because those animals are so rare, and because you have high levels of agreement on the vast majority of images, it makes it really easy to review any “uncertain” image that’s been ID’d as a rare species.

Pretty cool, huh?

Check out Mammal March Madness!

For some reason I missed this in 2013 and 2014. Maybe it was because I was finishing up my dissertation the first time and then recovering from a cross-country move the second time. But now I am totally excited about 2015’s

Mammal March Madness

What is Mammal March Madness? I’ll let organizer Katie Hinde explain:

In honor of the NCAA College Basketball March Madness Championship Tournament, Mammals Suck is featuring *simulated* combat competition among mammals. … Battle outcome is a function of the two species’ attributes within the battle environment. Attributes considered in calculating battle outcome include temperament, weaponry, armor, body mass, fight style, and other fun facts that are relevant to the outcome.

As a spectator to Mammal March Madness, you fill out a bracket and then follow along on Twitter or on the Mammals Suck … Milk! blog. The first game is on Monday, March 9, and direct elimination games continue until the championship on March 26.

I’ll note that the 2014 winner was Hyena who defeated Orca in the championship game. This year, we’ve got some Serengeti representation as well. But with Lion, Baboon, and Vervet monkey ranked just 8th, 12th, and 13th in the ‘Sexy Beasts’ division, they’re going to need all the cheering-on they can get.

So head on over, print out a bracket, and tell me who you think will make it all the way to the top this year.

(And just to be clear, I am not involved in Mammal March Madness in any way except as a participant. But it looks fun!)

Notes from the Lab

Meredith: Not all of our exciting research takes place in the field — there’s a lot going on behind the scenes in our lab, and we rely on an invaluable group of undergraduate research assistants to help us go through the massive amounts of Snapshot data you guys provide! Jess has been working with us for the last few semesters and has some insight on what it’s like to work with this Serengeti data set. 

Hello everyone! My name is Jessica Dewey, and I am currently an undergraduate student at the University of Minnesota working in the research lab that runs this project! Cool, right? I’m new around here so I thought my first post should be an introduction of myself and how I got involved in this lab.

Imagine me a few years ago: a young high school student, undoubtedly procrastinating in some way, suddenly stumbling upon a website called “Snapshot Serengeti”. At the time, I was only certain of two things — I loved animals and I loved research – so this discovery was perfect for me! I spent most of my evening identifying animals, and continued to go back to procrastinate even more.

Now flash forward to last semester, when I get an email from one of the university biology clubs saying that Dr. Craig Packer, head of the Serengeti Lion Project, will be giving a talk about his work. Well I HAVE to go! I sit and listen intently, eager to learn all about research being done with lions. Near the end of his talk he then mentions a website (Snapshot Serengeti, of course) where all of the images from the field get uploaded for the public to identify, and I’m immediately floored. How did I get so lucky, to go to the very University that uses those pictures I spent time identifying years ago in their research? The best moment was when the graduate students working with Craig said that they were looking for undergraduates to help them with their research. I took the opportunity to introduce myself to Meredith, and so far my experience in this lab has been amazing.

I’ve learned a lot about how field research is done, how data is collected and analyzed, and what it takes for someone to actually be a researcher in the field.  Not everything I do is as fun as going through tons of pictures a day, but all of the work in this lab is interesting and meaningful, and that’s what really matters to me. One of the major projects the lab has been working on with Meredith is trying to characterize changes in habitat at the camera trap sites by looking at the Snapshot pictures. We have been going through the giant list of data to find pictures to use for this characterization. We haven’t been going through the images themselves – rather the metadata, or the data ABOUT the data (it’s literally the biggest Excel sheet I’ve ever seen). It can get monotonous at times, but what keeps me going is the thought that when we finish picking out all of these random images, we will get to look at them and use them for this research project.

I hope that was a thorough enough introduction for you all, but let me say one last thing: THANK YOU!  Without the time you all put in to identifying these pictures, the research we are doing would not be happening at the pace it is.

Here is one of my favorite images I’ve seen so far:

pic1

 

Help us find animal selfies!

We’re partnering with National Geographic to put together a photo book of animal selfies from Snapshot Serengeti. We’ve got some selfies already from the first seven seasons, but because no one has looked through Season 8 yet, we don’t know what great selfies might be in there.

You can help! If you find an animal selfie, please tag it as #selfie in Talk. (Click the ‘Discuss’ button after you’ve classified the image and then enter #selfie below the image on the Talk page. You can get back to classifying using the button in the upper right.)

All proceeds from book sales will go to supporting Snapshot Serengeti. We’re planning for a fall 2016 publication date, so it will be a while. But we’re excited to get working on it.

tommie

Season 8 Movie Time

Looks like everyone is sinking their teeth into Season 8! As a reminder, feel free to ask questions or chat with us through the Snapshot Serengeti Discussion board or in the comments of any of our blog posts.

Now, there’s some data from this new season that hasn’t made it online — sometimes, instead of taking pictures, our cameras accidentally switch into “video” mode and capture 10-second clips of animals doing their Serengeti thing. While this isn’t very good for us in terms of data collection (although we’ve been tossing around the idea of setting up a Snapshot Serengeti: Video Edition!…), it gives you a unique perspective on the lives of these animals.

(Okay, so it’s mostly animals eating grass. They eat a lot of grass. Perhaps not the most “unique” insight on their behaviors, but they’re still pretty fun to watch). Here’s some of my favorite accidental movies from our new Season!

 

(If you want more videos, Margaret has blogged some of her favorite movie clips from past seasons here and here)

Season 8 Release!

And now, the moment you’ve all been waiting for …  Can I present to you:

IMAG0335

SEASON 8!

I’m particularly proud of this, the first season that I’ve helped to bring all the way from the field to your computers. We’ve got a lot of data here, and I can’t wait for you guys to discover a whole host of exciting things in this new season.

This season is accompanied by IMPORTANT changes to our interface!

There’s a few more bits of data we think we can pull out of the camera trap photos this time around, in addition to all the great information we already get. One thing we’re particularly interested in is the occurrence of fire. Now, fire is no fun for camera traps (because they tend to melt), but these wildfires are incredibly important to the cycle of ecosystem functioning in Serengeti. Burns refresh the soil and encourage new grass growth, which attracts herbivores and may in turn draw in the predators. We have added a fire checkbox for you to tick if things look hot. Now, because we’re looking for things other than just animals, we replaced your option to click on “nothing there” with “no animals visible“, just to avoid confusion.

Some of the more savvy creature-identifiers among you may have noticed that there are a few Serengeti animals that wander into our pictures that we didn’t have options for. For this new season, we’ve added six new animal choices: duiker, steenbok, cattle, bat, insect/spider, and vultures. Keep an eye out for the following:

duikersteenbokcattle

batspidervulture

This season runs all the way from September 2013 until July 2014, when I retrieved them this summer, my first field season. Our field assistants, Norbert and Daniel, were invaluable (and inhumanly patient) in helping me learn to navigate the plains, ford dry river beds, and avoid, as much as possible, driving the truck into too many holes. Together, we set out new cameras, patched up some holes in our camera trap grid, and spent some amazing nights camped out in the bush.

Once I got the hang of the field, I spend my mornings running around to a subset of the cameras conducting a pilot playback experiment to see if I could artificially “elevate” the predation risk in an area by making it seem as though it were frequented by lions (I’m interested in the reactions of the lion’s prey, and to see whether they change their behaviors in these areas and how long it takes them to go back to normal). I’m more than a bit camera-shy (and put a lot of effort into carefully sneaking up around the cameras’ blind spots) but perhaps you’ll catch a rare glimpse of me waving my bullhorn around blaring lion roars…

Back in the lab, there’s been a multi-continental collaboration to get these data cleaned up and ready for identification. We’ve been making some changes to the way we store our data, and the restructuring, sorting, and preparing process has been possible only through the substantial efforts of Margaret, over here with me in the States, and Ali, all the way across the pond, running things from the Zooniverse itself!

But for now, our hard work on this season is over – it’s your turn! Dig in!

P.S. Our awesome developers have added some fancy code, so the site looks great even on small phone and tablet screens. Check it out!

Getting good data: part 1 (of many)

Despite the site being quiet, there’s a lot going on behind the scenes on Snapshot Serengeti at the moment. Season 8 is all prepped and currently being uploaded and should be online next week! And on my end, I’ve been busily evaluating Snapshot Serengeti data quality to try and develop some generalisable guidelines to producing expert quality data from a citizen science project. These will actually be submitted as a journal article in a special section of the journal Conservation Biology, but as that is a slowwwwwww process, I thought I’d share them with you in the meanwhile.
So! Recall that we use a “plurality” algorithm to turn your many different answers into a “consensus dataset” — this has one final answer of what is in any given image, as well as various measures of certainty about that image. For example, back in 2013, Margaret described how we calculate an “evenness score” for each image: higher evenness means more disagreement about what is in an image, which typically means that an images is hard. For example, everyone who looks at this photo
would probably say there is 1 giraffe , but we’d expect a lot more diversity of answers for this photo:
(It’s a warthog, btw.)
To test how good the plurality algorithm answers were, we created a “gold-standard dataset” by asking experts to identify ~4,000 images. Overall, we found that consensus answers from your contributions agreed with experts nearly 97% of the time!  Which is awesome. But now I want to take a closer look.  So I took all the images that had gold standard data and I looked at Evenness, Number of “nothing here” responses, and %support for final species, and evaluated how those measures related to whether the answer was right or wrong (or, impossible). Even though we don’t have an “impossible” button on the SS interface, some images simply are impossible, and we let the experts tell us so so these wouldn’t get counted as just plain “wrong.”
A note on boxplots: If you’re not familiar with a boxplot, what you need to know is this: the dark line in the middle shows the median value for that variable; the top and bottom of the boxes shows the 25 and 75 percentiles; and the “whiskers” out the ends show the main range of values (calculated as 1.5 * interquartile range, details here). Any outliers are presented as dots.
Evenness: The boxplot below shows the mean “Evenness” score described above vs. how the consensus answer matched the gold standard answer.   What you can see below is that the average evenness score for “correct” answers is about 0.25, and the average evenness score for wrong and impossible answers is 0.75. Although there are some correct answers with high evenness scores, there are almost no wrong/impossible answers with evenness scores below 0.5.
 EvennessVsAccuracy
Percent Support: This number tells us how many people voted for the final answer out of the total number of classifiers. So, if 9 out of 10 classifiers said something was a giraffe, it would have 90% support. It’s similar to evenness, but simpler, and it shows essentially the same trend. Correct answers tended to have more votes for what was ultimately decided as the final species.
PercentSupport
NumBlanks: So with the evenness and percent support scores, we can do a decent job of predicting whether the consensus answer for an image is likely to be right or wrong. But with Number of Blanks we can get a sense of whether it is identifiable at all. Margaret noticed a while back that people sometimes say “nothing here” if they aren’t sure about an animal, so the number of “nothing here” votes for an image ultimately classified as an animal also reflects how hard it is. We see that there isn’t a huge difference in the number of “nothing here” answers on images that are right or wrong — but images that experts ultimately said were impossible have much higher average numbers of “nothing here” answers.
 NumBlanksVsAccuracy
So, what does this tell us? Well, we can these metrics on the entire dataset to target images that are likely to be incorrect. In any given analysis, we might limit our dataset to just those images with >0.50 evenness, or go back through all those images with >0.05% evenness to see if we can come up with a final answer. We can’t go through the millions of Snapshot Serengeti images on our own, but we can take a second look at a few thousand really tricky ones.
There’s all sorts of cool analyses still to come — what species are the hardest, and what’s most often mistaken for what. So stay tuned!
Follow

Get every new post delivered to your Inbox.

Join 4,816 other followers