How AI can be used to track pollinators in strawberry crops.

Deep learning algorithms referred to as AI are being applied to digital video to identify four different pollinator species working in a strawberry farm, and to plot the insects’ visits to flowers.

A study conducted during his PhD and being continued by Dr Malika Ratnayake in his capacity as a postdoctoral researcher at Monash University, forecasts a system that automatically reports which pollinators visit a crop, how often they visit, and from what direction – potentially superseding the expensive and time-consuming manual methods of pollinator assessment.

This is part of a broader project being led by Associate Professor Alan Dorin in the Faculty of Information Technology, Monash University, to explore how technology can be used to improve insect pollination security of crops and native ecosystems in the face of changing climate.

More than a third of the world’s food production from crops relies on animal pollination, Ratnayake says. Each pollination-dependent crop has different requirements around the optimal number of visits a flower needs from effective insect pollinators to maximise fruit yield and quality.

But at a point in human history when there has never been greater need for efficient pollination of crops, climate change is affecting the pollinators’ ability to do their job – partly through direct effects on the individual insects, and partly because shifting climate zones are bringing new insect species into conflict with pollinator species.

“Currently, there are no automated or efficient ways to monitor pollinator performance and use the information to manage pollinator populations in a timely manner,” Ratnayake says.

“Our system can record insect movement data in different parts of a farm, automatically count insects and insect-flower visits in each area, and compare the contribution of different insect types to crop pollination.”

“With the data in hand, growers can see whether they need to shift bee hives to better support areas of their crop, or raise the side walls of a greenhouse to allow better access to insects from a certain direction.”

And it is not only farmers who are interested in the system. Ratnayake is working with the German Centre for Integrative Biodiversity Research (iDiv) seeking to adopt the technology for insect biodiversity monitoring.

Ratnayake and his Monash colleagues published the findings of their experiments on a commercial strawberry farm at Boneo, Victoria, in the International Journal of Computer Vision. For this work, the team used nine camera trap modules built from low-cost Raspberry Pi cameras and Raspberry Pi computing boards to capture video at 1920 × 1080 resolution, at 30 frames per second. This was the upper limit of the platform’s processing power; lower quality would not have provided enough resolution for analysis.

As it was, Ratnayake says, material collected to video had to be downloaded from each unit for remote processing – a step he hopes to eliminate on the way to developing a more streamlined version of the system.

The video material was passed through an automatic algorithm that was largely adapted from two algorithms that Ratnayake developed during his PhD – one to track a single insect through occlusions by foliage and changing backgrounds of an outdoor environment, and an evolved version that can track multiple insects simultaneously.

The latest deep learning model distinguishes between insect species (four types at the time of publication), and forecasts likely flight trajectories for each species, to build a picture of how each species visits the crop’s flowers.

The flowers themselves also had to be tracked. Far from being static objects, flowers are shaken by wind, insects and in some crops follow the sun throughout the day. The deep learning model was used to map flower movements.

The deep learning-based object detector model starts by identifying the presence of an insect when it enters a video frame. After detection and identification, the insect is tracked through subsequent frames while its position is compared with the position of recorded flowers to identify flower visits.

Various thresholds are programmed in to guard against false positives and other precautions are also taken – for instance, the system snaps a still image of an insect on first detection to support human identification if it is needed.

Ratnayake and his colleagues are now dealing with the wider interest their findings have aroused and are working on expanding the developed system to an end-to-end fully-automated real-time “precision pollination” framework.

Also in Cosmos: People can learn to detect AI writing

Building a commercial version would require new capabilities with increased computing efficiency. So far, the team has provided proof-of-concept in greenhouse-grown strawberries that grow in flat clumps with flowers that are spread across what Ratnayake describes as a “2D structure”. Now they need to develop their algorithms to deal with “3D” plants like blueberry bushes, and they need to add more pollinator species to the deep learning model.

This work will likely require an increase in computational power. At the same time, commercial adoption of the system will likely be improved if pollinator activity can be directly reported from monitoring units – which means that every unit will need to be capable of its own on-board processing.

These challenges, and the need to build a system capable of providing meaningful results in open fields rather than just greenhouse environments, are now being worked on with a growing range of partners.

Ratnayake is optimistic about results.

“Currently there is no other practical system capable of building data on pollinator activity like this technology,” he says. “And there is nothing that can identify and classify insect pollination behaviour across large-scale industrial agricultural areas in a way that makes it possible to increase farm yield via improved pollination.

“Right now, the space is ours.”