Build "Plant Identifier" app with Google Lens API

There are a lot of use cases for Google Lens API, one of an interesting use case is identifying plant. Today, I am going to demonstrate building a plant identifier app from scratch using the Google Lens API. Generally, you would need to build your own custom Machine Learning model to accomplish the same thing. You might want to do so if you require finely detail and customization. But for most cases, relying on Google technology is a good decision.

Prerequisite

• Cloudinary - SerpApi's Google Lens API only support a hosted photo currently, meaning photo upload by the user cannot be used directly on the API. We use Cloudinary in this demo host the user uploaded photos, with that, we can generate a public URL to send to the Google Lens API. P.S. We are making an improvement to support photo upload, stay tuned 😉.
• Tech Stack - The app is build on NextJS, a popular react framework.

SerpApi - Sign up if you haven't done so. Upon signing up, you get good amount of free searches to explore the APIs. Visit the

Api Key page to retrieve the API key.

The App

Want straight dive into the code? Checkout the Github repository.

You can try it live here - https://plant-identification-demo-app.vercel.app/

Flow and design

The flow is very straightforward. It starts with allowing users to capture new photo or upload an existing one. Then the photo is first upload to Cloudinary and the generated URL is send to Google Lens API for further processing. The final results is display on screen in a simple and intuitive UI.

1. Take/upload a photo

2. Identify the photo

Behind the scene, photo is upload to Cloudinary and then the generated photo URL is send to SerpApi's Google Lens API for further processing.

3. Show the results

Wah-la!

Integrating Google Lens API

Here the standard response from the API:

{
  ...
  "visual_matches": [
    {
      "position": 1,
      "title": "Amazon.com: Costa Farms Higuera de hoja de violín + plantas tropicales de fácil cuidado de Dieffenbachia : Patio, Césped y Jardín",
      "link": "https://www.amazon.com/-/es/Higuera-plantas-tropicales-cuidado-Dieffenbachia/dp/B0CBCR79YQ",
      "source": "Amazon.com",
      "source_icon": "https://serpapi.com/searches/685b5631bb40c82c0f4e056a/images/fa57478952d91a9c621a55c27531d73242b4ec7e1446e82c1d38f4f39bf46cca.png",
      "thumbnail": "https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQyEoaU4Py76QO4-Us5OD4sEA9vE6Xml3-p3tBD1j3CVi_QXaoJ",
      "thumbnail_width": 224,
      "thumbnail_height": 224,
      "image": "https://m.media-amazon.com/images/I/81X0RENBWpL._UF894,1000_QL80_.jpg",
      "image_width": 894,
      "image_height": 894
    },
    {
      "position": 2,
      "title": "Tropic Snow Dieffenbachia | Plant Addicts",
      "link": "https://plantaddicts.com/tropic-snow-dieffenbachia",
      "source": "Plant Addicts",
      "source_icon": "https://serpapi.com/searches/685b5631bb40c82c0f4e056a/images/fa57478952d91a9c6275f81b81641d4deb24ee21f7b2e74ea49f0cc8f068e36d.png",
      "rating": 3,
      "reviews": 1,
      "price": {
        "value": "$21*",
        "extracted_value": 21,
        "currency": "$"
      },
      "in_stock": false,
      "thumbnail": "https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcREKAn6EHoI38fkHLmmZ-A4RMoy8J3xmBZCoBmQvgaVpO5OWL2N",
      "thumbnail_width": 225,
      "thumbnail_height": 225,
      "image": "https://cdn11.bigcommerce.com/s-jmzfi5zcr2/images/stencil/original/products/5868/18449/_Tropic_Snow_Dieffenbachia_Size__20332.1664304667.png?c=2",
      "image_width": 1000,
      "image_height": 1000
    },
    {
      "position": 3,
      "title": "Gardenera Premium Dumb Cane Potting Soil Mix - Ideal Blend for Growing Robust and Healthy Dumb Cane Plants - 2 Quart : Patio, Lawn & Garden - Amazon.com",
      "link": "https://www.amazon.com/Gardenera-Premium-Dumb-Cane-Potting/dp/B0D5W86VCF",
      "source": "Amazon.com",
      "source_icon": "https://serpapi.com/searches/685b5631bb40c82c0f4e056a/images/fa57478952d91a9c8668b2e158dbedbe049f0a469d5049f33ff7e15db6591562.png",
      "price": {
        "value": "$17*",
        "extracted_value": 17,
        "currency": "$"
      },
      "in_stock": true,
      "thumbnail": "https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQ9g2MsLNe9Kg5gCGKZsm3zGDihIkpAahRHIUwU8-P_0ad2C45y",
      "thumbnail_width": 212,
      "thumbnail_height": 237,
      "image": "https://m.media-amazon.com/images/I/61myoAGflbL._AC_UF350,350_QL80_.jpg",
      "image_width": 313,
      "image_height": 350
    },
    ...
  ],
  "related_content": [
    {
      "query": "Dieffenbachia seguine",
      "link": "https://www.google.com/search?sca_esv=505db82e361840e6&lns_surface=26&q=Dieffenbachia+seguine&kgmid=/m/04_1kjb&sa=X&ved=2ahUKEwjzqd2Au4uOAxXSg4QIHS4VGg0Q9_gLKAB6BQjyAhAB",
      "thumbnail": "https://serpapi.com/searches/685b5631bb40c82c0f4e056a/images/6136f8c648305ff4e247cc1a2ad591974447f7ac365e47739057781c129d8a53.jpeg",
      "serpapi_link": "https://serpapi.com/search.json?device=desktop&engine=google&google_domain=google.com&kgmid=%2Fm%2F04_1kjb&q=Dieffenbachia+seguine"
    },
    {
      "query": "Dieffenbachia amoena",
      "link": "https://www.google.com/search?sca_esv=505db82e361840e6&lns_surface=26&q=Dieffenbachia+amoena&kgmid=/m/03hkj_7&sa=X&ved=2ahUKEwjzqd2Au4uOAxXSg4QIHS4VGg0Q9_gLKAF6BQjyAhAD",
      "thumbnail": "https://serpapi.com/searches/685b5631bb40c82c0f4e056a/images/6136f8c648305ff4e247cc1a2ad591972f67e0dabaf65e84413f1ba433216509.jpeg",
      "serpapi_link": "https://serpapi.com/search.json?device=desktop&engine=google&google_domain=google.com&kgmid=%2Fm%2F03hkj_7&q=Dieffenbachia+amoena"
    },
    {
      "query": "Dumb canes",
      "link": "https://www.google.com/search?sca_esv=505db82e361840e6&lns_surface=26&q=Dumb+canes&kgmid=/m/01hcl4&sa=X&ved=2ahUKEwjzqd2Au4uOAxXSg4QIHS4VGg0Q9_gLKAJ6BQjyAhAF",
      "thumbnail": "https://serpapi.com/searches/685b5631bb40c82c0f4e056a/images/6136f8c648305ff4e247cc1a2ad591970b6375c0d40cb26c7a640fbb3e06703a.jpeg",
      "serpapi_link": "https://serpapi.com/search.json?device=desktop&engine=google&google_domain=google.com&kgmid=%2Fm%2F01hcl4&q=Dumb+canes"
    }
  ]
}

Google always return related searches for the given image, clicking on the link bring you to Google Search about the query. We can assume this is a good place where the names for the plant will show up, thus we use related_content from the API response for possible names. In addition, the content from other matches come from visual_matches.

Demo

If you would like to dive deeper into the Google Lens API, feel free to checkout our other comprehensive tutorial below.