* Sarah Huebner, who heads up the Snapshot Safari team has written the following blog to give all participants of Snapshot Safari projects the low down on machine learning advances that are being introduced today*
In the era of Big Data, when equipment allows us to collect data faster than we assess it, researchers are always looking for ways to enhance and accelerate the process between data collection and analysis. We here at Snapshot Safari are proud to have been the first camera trapping project partnering with citizen scientists on Zooniverse when we introduced Snapshot Serengeti, and to have expanded that model from one African park to dozens. Now we hope to improve the data pipeline once again by integrating machine learning to reduce the amount of volunteer effort required to classify data from our participating sites.
‘Machine learning’ refers to Artificial Intelligence algorithms that have been trained for a specific task or purpose. These algorithms are fed millions of images labeled with their correct names and are ‘trained’ to recognize those animals again in different settings. These models generate ‘predictions’ based on the training they’ve received and provide confidence levels to let us know how sure they are that is the correct label. Because Snapshot Serengeti has been running since 2010, it has generated millions of images over the years, which make a perfect training dataset for machine learning (ML) algorithms. We are employing ML models to drastically reduce the effort required to retire empty images (no animals present) and to retire images of common animals like wildebeest and zebra.
First, our ML models have become quite good at telling us whether animals are present or not. This helps us to more easily spot cameras where vegetation has grown in front of the lens, resulting in hundreds of pictures of grass blowing in the wind. Pretty, but not quite what we’re after, so we can eliminate those prior to upload. Secondly, we have modified the retirement rules on Snapshot projects (implemented starting today as new seasons are launched) so that only two volunteers need to confirm the computer’s prediction of ‘empty’. This means instead of 10 or even 20 people viewing those photos, only two people will see them and can push them out of the dataset quickly.
Those of you who have been working on this project for a while know that the wildlife you’re most likely to see are zebras and wildebeest, and you all are great at identifying those! Because those are easy identifications, they too will retire with fewer views than before. What this means practically is that you should see more images of rare and cryptic species like predators and fewer blank images. We have implemented a number of retirement rules behind the scenes to make this happen, based on varying confidence levels produced by the algorithm. For example, our simulations have proven that even at only 50% confidence, the computer is right 99.6% of the time when it tells us that an image is empty. Therefore, any ‘empty’ prediction with confidence of 50% or more will only need two human views to confirm that the computer is correct. Likewise, if the model tells us that it’s a human with a confidence level of 80% or higher, we will retire with just two confirmations.
We will continue to improve the algorithm’s capabilities by using our most valuable asset—all of you! We hope that you will be as interested as we are in advancing the use of ML to make the classifying process more fun and satisfying. The algorithm is pretty good at species, but now we need to improve its ability to count animals, so we will soon be introducing a special project, ‘Snapshot Focus’, which will feature images the algorithm has reviewed and marked each animal with a bounding box. We will ask you to tell us whether the ML model got it right. Stay tuned for that and other special projects!
We are launching three new sites today—Camdeboo National Park, Kgalagadi Transfrontier Park, and DeHoop Nature Reserve, all from South Africa. These three projects have the new retirement rules in place, as will Season 12 of Snapshot Serengeti, which will launch in June. As new seasons or new projects come online, they will be set up with these rules and perhaps more as we refine the data pipeline. Let us and the moderators know how it goes. We are so thankful for your efforts and support, which help us to return data to our collaborators at reserves in Africa quickly and with confidence that it is correct thanks to the combination of citizen science and machine learning. Happy classifying!
Research Manager, Snapshot Safari
May 28, 2019
For more information about the machine learning algorithms created using Snapshot Serengeti images, see:
Willi, Marco, Pitman, Ross Tyzack, Cardoso, Annabelle W., Locke, Christina, Swanson, Alexandra, Boyer, Amy, Veldthuis, Marten, and Fortson, Lucy. (2019) Identifying animal species in camera trap images using deep learning and citizen science. Methods in Ecology and Evolution 10(1):80-91.
Norouzzadeh, Mohammad Sadegh, Nguyen, Anh, Kosmala, Margaret, Swanson, Alexandra, Palmer, Meredith S., Packer, Craig, and Clune, Jeff. (2017) Automatically identifying, counting, and describing wild animals in camera-trap images with deep learning. Proceedings of the National Academy of Sciences 115(25):E5716-E5725.
To read about how algorithms make decisions in comparison to humans, see:
Miao, Z., Gaynor, K.M., Wang, J., Liu, Z., Muellerklein, O., Norouzzadeh, M.S., McInturff, A., Bowie, R.C., Nathon, R., Stella, X.Y. and Getz, W.M. (2018) A comparison of visual features used by humans and machines to classify wildlife. bioRxiv, p.450189.
It has been just over a year now since Snapshot Safari was launched. Snapshot Serengeti as the original Zooniverse citizen science camera trap project has remained the flagship project but there are now several other projects under the Safari banner.
One of the good things about joining forces with all these other projects is that collaboration tends to bring perks that operating on your own doesn’t. For example, different areas of expertise that some of us would just never have thought of, being somewhat far from the mind of field researchers and ecologists.
I am talking about the computer learning side of things, ML (Machine Learning) in particular. The team behind Snapshot Safari have been working hard on this aspect essentially to help speed up the rate of classification needed now that there is so much data across different projects.
They have recently engaged a specialist in machine learning at the University of Minnesota who is helping them to develop ML for use with the Snapshot Safari projects.
The idea is to run an algorithm on the data prior to uploading it to the Zooniverse, this will identify most of the misfire and vegetation only images meaning us volunteers will be left with more of the good stuff, the actual animal images.
The algorithm has been trained on millions of images from Snapshot Serengeti, now for these latest batches the team have asked the ‘machine’ to predict which species it is seeing, how many and what the behaviour of the animals is. Don’t panic we really are in the early stages and the idea is to compare what it came up with against the actual results from the volunteers whom we already know are pretty darn accurate. The team doesn’t expect great things yet from the ML and doesn’t foresee computers taking over any time soon but it will be interesting to see how the ML goes.
So, for now things will be continuing as normal, no changes in how you classify or what you do but hopefully there will be fewer blank images with no animals. As the team stresses, they are a looonnng way from machines taking over.
For those of you who enjoy being part of this developmental side of the project the Safari team will soon be launching a side project called Snapshot Focus. It is designed to also look at how well the ML is at recognising animals in trickier images, especially those with multiple animals. If you feel like helping out as a break from the usual work flow it couldn’t be simpler. All you have to do is answer yes/no as to whether the computer has managed to put a bounding box around every animal in the image.
Look out for updates from the team over the coming months to learn more about these developments but meanwhile there are still lots of images to classify on Snapshot Serengeti.
It seems as though when it comes to lion ecology most of the experts seem to agree that male coalitions are usually the most successful at holding on to females and siring cubs. Certainly when take-overs happen it is usually the coalition with more members that wins the day and lone males find it hard to stand their ground. Numbers seem to count.
Of course that isn’t to say that single males can’t have fun or success. Take Kalamas, a male known to us in the Ngorongoro Conservation Area. He is a nomadic male who wanders far and wide across the area even daring to take trips into the Crater itself, an area that in terms of lions is heavily defended by resident males, competition is very strong there and really not a place for a none resident lone male to be seen.
So what is so different about Kalamas. Well firstly he doesn’t seem concerned about the competition. Earlier this year whilst monitoring the Crater Lion’s Ingela Jansson of KopeLion project spotted a very distinctive dark maned male that she recognised as Kalamas. The last time he had been spotted in the Crater was in November 2015 but this time there he was in full view lounging around mating with one of the Lakes pride females. In the background four contesting males could be heard roaring their presence.
Kalamas ignored them and had the audacity to stay put in the crater with the female for three days before walking back up the steep Crater slope and out onto the open plains of the Ngorongoro Conservation Area where despite being surrounded by Maasai herders and their live stock he managed to stay out of trouble.
A few weeks later however we received a frantic message early one morning to say that Kalamas was sitting out in the open with many herders starting to gather. Fearing some sort of conflict our team rushed to the area, luckily fairly close to headquarters. Once there we could see that people were sensibly keeping a safe distance and approaching by car it was evident that Kalamas had been fighting. He showed deep wounds characteristic of fighting male lions and there was much blood splattered around.
He attempted to stand but couldn’t quite make it and so we feared he may have been fatally injured.
We decided to stay around and monitor the situation; Kalamas just lay there for many hours. As the sun climbed to its midday point kalamas managed to drag himself into the shade of our vehicle where he remained for the rest of the day. Just as we were starting to wonder what on earth to do next, with night fall approaching, Kalamas took us totally by surprise and stood straight up, shook himself gently and, rather shakily started to walk towards the safety of a well treed gully. Satisfied that we had done all we could and that Kalamas would take care of himself we left.
For the next few weeks we monitored his movements and he seemed to lay low, recovering, but you can’t keep a good lion down. Far from learning his lesson about encroaching on other male’s territories he has since been seen in the presence of other females from the Crater rim. His modus operandi seems to be to hang around on the periphery and entice the ladies away for a few days at a time. They just don’t seem to be able to get enough of him. Something about that Jon Snowesque mane of dark shaggy dark hair.
It is an interesting tactic. We ponder whether perhaps mating with pride females belonging to other males in this sneaky way may mean that when (and if) they give birth the resident male is duped into believing that Kalamas’s offspring are their own.
It is certainly a great way for Kalamas to get as many females as possible but not have the burden of looking after any of the offspring.
It is certainly an unusual story and far from the norm. It remains to be seen if Kalamas was at all successful or if the resident males were harder to fool than he imagined. We are looking out for cubs with dark manes though.
I have been a little quiet recently and for that I must apologise but my excuse is good. I have been relocating to Tanzania where I am going to be based for the next three months working with Kopelion in the Ngorongoro Conservation Area. One of Snapshot Serengeti’s partners kopelion (Korongoro People’s Lion Initiative) is a conservation organisation and research project that focuses on human-lion coexistence in the multi use landscape of Ngorongoro. I have written about the project in these blogs if you want to read more https://blog.snapshotserengeti.org/2017/02/23/meet-the-people-2/
After a two week intensive language course in nearby Moshi I have finally made it to base camp on the crater rim. The office is perched at 2300m looking down on the Crater Lake and has one of the best office views I have ever had which makes up for being stuck indoors when you would rather be in the field.
It’s not all office based thank goodness and I have already had the pleasure of three days out in the Ndutu area learning about the work the project does. Although there has been some rain it is still in the grip of the dry season here and the scenery for the most part is a dry and dusty yellow. The lions are hungrily awaiting the rains that will bring a welcome flush of green that will draw the wildebeest in vast numbers and thus plenty of prey.
Saturday was spent following up on reports of lion spoor (tracks) found near to an area that Maasai bring their cattle to drink. We turned up early morning to start tracking the spoor to see if we could figure out if the lion where still in the area; if this turns out to be the case a lion guardian or Ilchokuti will stay put in the vicinity to warn herders about the lion presence and hopefully avoid an encounter.
It was obvious that several lion had been in the area, you could see depressions in the sand where they had lay down for a bit of a nap. The tracks lead alongside a small water drainage channel and the lions had wandered down to drink in a few spots. Further along the water channel the tracks of individual lions suddenly converged on one access path down to the water. Clearly something had excited their interest. After a careful look around we descended the same route to investigate. Lying in the mud at the edge of the water we found the body of a young spotted hyena, teeth marks around its throat and the surrounding tracks told the story. Most likely the youngster was drinking when the lions ambushed it, its small size meant it didn’t stand a chance and lions probably quickly dispatched it.
Despite the fact that the lions in the area are somewhat lean at the moment they made no attempt to eat the hyena. This is normal behaviour for lions; they will not tolerate other predators in their territory and will kill them if the chance arises. There was a lack of other hyena spoor in the area so this youngster was probably on its own, why we cannot say but it became an easy target for the lions.
It is a great privilege to walk into an area that has such a story written out for you in the sand and mud. In this instant the presence of a body left little to interpret but the trackers here are capable of reading far less obvious stories and it is this skill that is helping to mitigate lion-human conflict by acting as an early warning system to the people who live side by side with lions.
Our camera-trapping efforts afford us an unparalleled view into the lives of the Serengeti ecosystems animals but the work of conservation has many aspects and I hope to bring you a good view of what is going on here over the next few months.
Photo Credit: Edward Lopatto
These incredible images of a major lion turf war have been taken by the team in the Serengeti and come with the fantastic announcement that the long running Serengeti Lion Project is back up and running.
Although the camera-trap aspect of the project has continued without pause, the main work of the Serengeti Lion project has been on hiatus for the past few years. Now, it is finally being restored and the priority is to sort out who’s who in all the study area prides. Comparing existing id’s and adding new ones is going to take some time.
Looks like these boys are trying to shake up the genes even more. Two coalitions both looking strong have clashed over ownership of prime real estate. The team report that all the males involved looked strong and healthy so this is probably not the definitive battle.
We will have more news for you soon on how the work is going as well as reports from the field, so stay tuned. Meanwhile enjoy these stunning images.
The saying goes a picture is worth a thousand words but here we have a picture that doesn’t tell us enough.
What happened here? We see a beautiful male lion strolling through the Serengeti but he has quite a gash on his back. Other wise he looks unharmed and he has a full looking belly so what do you think? A fight, a hunting injury, miss judged a low branch?
This picture leaves us hanging and is a few words short of the sentence. What’s your thoughts?
Snapshot Serengeti is in the limelight again!
A new paper titled “No respect for apex carnivores: Distribution and activity patterns of honey badgers in the Serengeti” has been published by a team from the University of Wisconsin and University of Ljubljana using the Snapshot Serengeti data classified by our citizen scientists.
Honey badgers are surprisingly understudied. Although extremely charismatic the fact that they have large territories, up to 541km2 for an adult male in the Kalahari, and no clear habitat preferences makes it hard to predict where to find and study them.
The Snapshot Serengeti data of course is a dream come true to many researchers enabling them to ask scientific questions without having to wait potentially years to collect data themselves.The team took advantage of the open access data, courtesy of Snapshot Serengeti to look at what they could learn about honey badgers and how they live alongside other predators. Ferocious as they are honey badgers are killed by lion, hyena and leopard and so the team wanted to know whether they avoided areas where these large carnivores were active.
Well it seems that despite ending up as an occasional victim the honey badger is quite happy living alongside the larger carnivores, at least in the Serengeti anyway according to the authors. It appears as if the honey badger actively seeks out the same habitats as the large carnivores. The authors modelled a variety of different explanatory scenarios to see which would be the best fit to explain honey badger distribution across the Serengeti study area. Included where variables such as habitat preference, water availability, cover availability, lion abundance, and leopard abundance. Their best models showed that the presence of all three large carnivores coincided with the presence of honey badgers and that there was also a positive correlation temporally between leopard, hyena and honey badger showing that they use the same habitat at the same time.
It’s interesting stuff. The authors do point out that although the data set was huge there was actually very few incidence of honey badger over the 3 year period covered by their work and so their sample size was small. It does however show just how valuable the data collected by Snapshot Serengeti and the other Snapshot Safari projects can be, if nothing else to give scientists a relatively inexpensive way to explore questions before undertaking more specific research work themselves.
You can read the paper here, although it is not open access unfortunately: https://www.sciencedirect.com/science/article/pii/S1616504717302720
These two images illustrate the point nicely, you can clearly see the same camera has captured honey badger and spotted hyena with in 13 days of each other. Interestingly both in day light.
I promised I would have some news about what the Serengeti team has been up to recently in the field. Our beloved camera-trap grid is still being cared for, cards downloaded, batteries replaced and cameras given the once over. So all is well on that front but what is the latest question being asked by the team.
Well thanks to the spatial occupancy modelling of the Snapshot Serengeti camera-trap grid we have learned a lot about how the animals share the environment. What we can’t derive from the camera trap images is the details of what the different species are doing when they are in those spaces and how so many large herbivores can exist together. It could be that they simply facilitate each others foraging or maybe they are using different resources. Scientists have identified what is known as niche partitioning, a mechanism that sees different species specialising in eating different proportions of grasses verses non-grasses; pure grazers and pure browsers and a sliding scale between the two. A second mechanism sees different species eating different parts of the same plant.
These two mechanisms seem to make perfect sense but it is not understood to what extent these two truly affect coexistence of large herbivores. This is where the Snapshot Serengeti team research comes in.
Under our own Dr Michael Anderson they have teamed up with Dr Rob Pringle and researchers at Princeton University in using a revolutionary new analysis method known as DNA metabarcoding to see what exactly each animal is eating.
Up until recently scientists studying herbivore diet had two choices, they could watch their subjects and try to identify what they were eating or they could use microhistology, whereby plant parts in faeces are visually identified. As you can imagine these methods are fine for differentiating between, say, grasses and trees but don’t allow scientists to classify down to individual plant species. With DNA metabarcoding they now have that ability and it should tell us a whole lot more about how the animals divide their resources in space and time.
So that’s the science but how does the team collect this data. Well as with microhistology it involves dung. Our intrepid scientists are roaming the Serengeti collecting poop from as many different herbivores as they can and then it all has to be shipped back to the labs for analysis.
If you are thinking that our team must be highly skilled detectives able to identify a wide variety of brown pellets in the savannah grasses then think again. That’s not to say they can’t but this work relies on 100% knowing which species produced said dung its sex and age as well as a sample that has not been contaminated in anyway. The method of collection relies then, on stealthy observation waiting for an individual to lift its tail and sprinkle the ground with brown pellets before running in with your sample jar at the ready to collect the freshly deposited “clean” offerings. I have some experience with this work and believe me it does feel slightly odd to be observing animals in this way, willing them on to have a bowel movement so you can move on to the next species. It is also a little risky as you can get so engrossed at watching your target animal that you forget there are predators there watching and waiting. At least in this project it is only herbivores the team are interested in, to do the same with predator’s faeces, that’s a whole lot more smelly.
The study is still in its early stages but the team reports they are already seeing some noteworthy things.
Spoiler alert, early results suggest that there are only two ‘pure’ grazers in Serengeti (zebra and warthog) and lots of variation between wet and dry season.
We will bring you further updates once the team has finished their analysis work and have the full results. It promises to be exciting stuff. In the meantime you can think on the glamorous job a field scientist has whilst you stay clean at home helping with the job of classification.
Those of you who have been with us for some time will probably have noticed that the image quality since we switched to the Snapshot Safari platform has reduced, sometimes dramatically. Before I go any further, we are trying hard to fix this but in the meantime I thought I would try and explain what the issues are in a hope that it may induce a little more patience from you. I am afraid that I really am technically challenged when it comes to computer stuff so I am going to be a little vague here but please, if there is anyone out there with more knowledge who can either help explain more appropriately or better still offer our team help don’t hesitate to get in touch.
So the trouble all started when Snapshot Serengeti joined the bigger Snapshot Safari platform at the start of this year. At this time Zooniverse was having a big overhaul with older projects operating on Ouroboros moving over to the Panoptes format. Essentially Ouroboros and Panoptes are both software packages which enable projects to build their pages and run them.
Of course Snapshot Serengeti being one of the oldest Zooniverse projects was designed using Ouroboros and has had some teething problems with the switch over. One thing to remember is that the teams involved with bringing all the camera trap images to the Snapshot Serengeti platform are for the most part unpaid graduate and undergraduate students studying ecology. They are not experts in computer programming yet have to keep the platforms running and fix all the problems.
In the old days the University of Minnesota based team would upload the batches of images from the camera traps and send them to Zooniverse who would process and upload them to the platforms. That was when there were a dozen or so projects. There are now over 50 active projects. Can you imagine how long it would take for Zooniverse to do all the uploading? To address this problem they have asked individual projects to manage the uploading themselves. To complicate this process a little more they have also placed a 600GB maximum file size on the images.
This all means that the team of ecologists at Minnesota have to engage computer code developers to write custom scripts enabling their super computers to interact with the Zooniverse web platform. The image quality issue then is not because we have started using different camera’s or taking images at a lower resolution it is due to the code that compresses the images from their full size to less than 600GB. Those images that were smaller in the first place have been less effected than the larger ones and hence the mixture of quality that we are seeing.
So as I said earlier we are trying hard to get this problem sorted and bring you back the kind of top rate images you are used to and hope to have things sorted with the next batch of images we upload. In the meantime please spare a thought for the team and remember that like you they are all volunteers, all be t with a slightly more vested interest in the research project. I hope that you will bear with us and keep up the much needed support you have always given us.
Symbiotic relationships are common in the Serengeti. They fall into two main types, mutualism, whereby both partners benefit from one another and commensalism, whereby one partner benefits from the actions of the other but the other partner is largely unaffected or unharmed. I wrote recently of oxpeckers and large herbivores, large herbivores provide food in the form of ticks for the oxpeckers and oxpeckers provide a cleaning service for the large herbivores, a good example of mutualism. Birds such as cattle egrets that follow buffalo around to catch the invertebrates the buffalo disturb as they graze is an example of commensalism. Of course it is not just animals that have symbiotic relationships; my blog last week on termites and mushrooms was another example of mutualism.
So what about zebras and wildebeests? We see them all the time on Snapshot Serengeti in mixed herds, grazing peaceably with one another. Is this just coincidence or is this a form of symbiosis?
It is actually hard to say and of course that is why labelling things, especially behaviour is often tricky.
Zebra and wildebeest are both grazers meaning they mostly eat grasses but that doesn’t mean they share the same diet. They preferentially eat different parts of the plants that they consume. Zebras are quite content chewing longer tougher grasses where as wildebeest prefer shorter, more tender shoots. This partition of resources means they can quite happily graze side by side with out exerting pressure on each other.
Another good reason to team up is the extra safety that numbers provide. Not only do more ears and eyes provide better early warning systems but the odds of the individual being targeted by a predator are reduced when there are greater numbers to choose from. Apparently zebra have better eyesight but wildebeest have better hearing so the two complement each other.
There could be another reason. Our very own Meredith Palmer just published a paper about interspecies reaction to each other’s alarm calls, you can read it here: https://www.sciencedirect.com/science/article/pii/S0003347217304207
She found that zebra, wildebeest and impala recognise each other’s alarm calls but that they did not always respond in the same manner. When zebra sounded the alarm all three herbivores reacted strongly but when impala gave the alarm zebra where likely to ignore it, or assess the relative danger themselves. It seems that this varied response is down to predator size. Impala are prey to a wide range of smaller predators that would not be able to handle a mammal the size of a zebra, so when impala give the call it doesn’t always signal danger for the zebra. However when a zebra, the largest of the three herbivores sounds the alarm, whatever it has seen will probably be able to take down the wildebeest or the impala too so it’s prudent that all three scarper.
It is an interesting reaction and maybe wildebeest hang out with zebra because they are more trustworthy alarmists. I am not sure that the companionship of zebra and wildebeest can be classed as symbiotic I think it is more of an interaction due to a shared habitat but it seems that on some level they can benefit each other.