Self driving cars, facial recognition software, mobile cheque deposits, the way facebook knows to tag your friend Andrew in all your pictures of him without asking; a huge variety of technologies, even those we take for granted, employ state of the art technology in computer vision. In agriculture, the applications of computer vision are also growing fast. In part one of our series on computer vision in agtech, we’ll answer some basic questions about the nature of computer vision and discuss its applications in autonomous weeding, harvest and quality control.
What is Computer Vision?
Computer vision may seem simple enough to understand, but below the surface it is a complex and multidisciplinary field, tied up with a variety of technologies, both old and emerging. Computer vision may use a variety of cameras to act as the machine’s “eyes” depending on the imaging requirements. In one popular fruit detection method, colour cameras are used to identify fruit from tree, and additional stereo cameras detect the relative position of the fruit for automatic harvesting. This is far from the only computer vision model though. Infrared, multispectral, thermal or even three dimensional cameras can be used. Computers can also use point or curve based identification, or even detect objects by shape or texture contrast. For agricultural applications, merely detecting a preset object just isn’t enough - produce is rarely uniform and may vary in size and colour, and more processing is needed to actually identify the fruit. In short, the “seeing” aspect of computer vision is only the beginning, identifying or understanding the image is the next step.
This “understanding” of an image is a significantly more complicated affair, and the processes are typically segmented into either machine learning, or deep learning. In layman’s terms, machine learning is an algorithm based system, that parses data, learns from it, and applies that data to future decisions or recommendations. Applications like Spotify use simple machine learning of this type to analyze your music tastes and make recommendations. Your liked songs act as structured data for the algorithm, which then finds similar songs to recommend to you.
Deep learning is essentially a subset of machine learning, using a layered artificial neural network to gather information and identify patterns and learn while doing. Meant to mimic the way the human brain learns, deep learning is more complex, and can be applied to complex problems like natural language processing, and image recognition.
Weeding with Machine Vision
Combating weeds and making sure crops have room to grow is a constant struggle for farmers. New Zealand based agrioptics’s WeedSeeker® is a commercially available weed sensor with agricultural applications that uses light emitting diode technology to identify weeds for spraying.
Xarvio image recognition is another advanced scouting technology that can even make use of your smartphone camera, with a 61-100% accuracy rate in identifying 123 types of weeds in their database. The Xarvio app can also detect and identify diseases affecting their 37 listed crops, in addition to their leaf damage analysis system that detects damage to foliage based on a green-colour ratio.
Harvesting with Machine Vision
Automatic harvesting brings additional challenges to the table, with various hardware requirements for grabbing delicate produce without damaging it. An automatic harvesting machine can be boiled down to three basic requirements: fruit detection, location detection, and the physical harvesting equipment to grasp and detach fruit from the plant.
Fruit detection remains one of the biggest challenges in agricultural AI, mainly due to the extremely varied harvesting conditions. Unpredictable lighting, inconsistent shapes, obstruction by foliage or other fruit, all these factors and more make in-field detection a complicated affair.
One New Zealand research paper, published in the the May 2019 issue of Biosystems Engineering, presents a design for an automatic kiwi harvester, that uses paired colour cameras and deep learning, combined with stereo cameras to identify and locate fruit for harvest. Four arms equipped with vertical grippers identify fruit, locate them, and then grip and twist the kiwis to snap them off the stem at a rate of one kiwi per second!
Innovations in so called “soft” harvesting, where machines are equipped with delicate suction cups or padded grabbers are just one way growers are protecting their fruit during autonomous harvests. In 2016 developers at Abundant Robotics created the first vacuum harvester, which uses computer vision to identify apples and then sucks them through a soft hose at a rate of an apple per second.
Different types of produce have very different hardware requirements for harvesting. Strawberries, like many berries and tender fruits, demand a very intensive harvesting program, with special consideration for handling the delicate berries. Agrobot™ is an ag-tech start-up developing a range of robots to help solve strawberry farmers’ harvesting challenges. One of Agrobot’s flagship projects is a 24-arm robotic harvester, designed to pick strawberries. Agrobot uses machine learning technology to identify and measure the ripeness of strawberries, then cuts the fruit just above the calyx and grips the stems to transfer them to harvest containers. In this way, the harvester never actually makes contact with the strawberries, minimizing the risk of damage or bruising.
QA/Grading with Machine Vision
Quality assurance and fruit grading may need even more detailed computer vision technology to match the effectiveness of a human operator. Beyond fruit detection, quality assurance programs may measure size, colour profile, ripeness, and detect defects, identify diseases, and classify scanned fruit based on these metrics.
Croptracker’s Harvest Quality Vision™ is an in-field apple grading technology using a camera attachment mounted on a standard iPad tablet that is scanned over a bin of apples to determine the size, colour profile, and quantity of fruit in visible range. The system can then classify the apples into custom harvest categories based on size and colour heuristics supplied by the grower.
This has been part one in our series on computer vision. Tune in next week, when we take a look at some different styles of machine vision grading and sorting technology, and the applications of machine vision in UAV monitoring and autonomous farm equipment.
Interested in learning more about Croptracker? Learn more about our Farm Management Software, or book a demonstration to schedule a meeting with our product experts.
And as always, if you're ever stuck, never hesitate to e-mail us at support@croptracker.com or Live Chat with us by clicking the green speech bubble 
in your bottom right-hand corner. We're always happy to help you make the most of Croptracker to make your farm more efficient, safer, and more profitable!
