Image Classification Android App with TensorFlow Lite for Beginner

Today, Machine Learning (ML) is all over the place. The ML reduces human works, suppose I'm trying to identify birds, as a human what we generally do we search for every bird that is present in this world and identify the particular one. But Machine Learning reduces that, in ML we will simply create one model which involves every bird. Then when I tried to search for any birds, that model will simply tell me which bird it is.

So in this blog, we will see how to connect a tflite model in android or how to use a tflite model in the android app.

In this tutorial, we’ll look at how to use TensorFlow Lite to integrate machine learning into an Android app by creating a basic app.

TensorFlow Lite

TensorFlow Lite is a set of tools that enables on-device machine learning by helping developers run their models on mobile, embedded, and IoT devices.

Key Features of TF-Lite:

• Multiple platform support, covering Android and iOS devices, embedded Linux, and microcontrollers.
• Multiple language support, which includes Java, Swift, Objective-C, C++, and Python.
• High performance, with hardware acceleration and model optimization.

In order to start, you will require a trained .tflite model that you can download from here.

Note: after downloading name this file as BirdsModel.tflite.

Add TensorFlow Lite to the Android app.

Step 1: Head over to android studio & Create a new android project.

Step 2: Add TensorFlow Lite to the Android App.

Right-click on the package name in my case it is com.yourpackagename or click on File, then New > Other > TensorFlow Lite Model. Select the model location where you have downloaded the custom trained BirdsModel.tflite earlier

Step 3: Click finish.

Note that the tooling will automatically configure the module’s dependencies for you using ML Model binding and all requirements will be added into your Android module’s build.gradle file.

Step 4: At the end, you’ll see the following. The BirdModel.tflite file has been successfully imported, and it displays high-level model information like as input and output, as well as some sample code to get you started.

Processing image and showing result

following are the simple steps to implement the bird classification model.

1. Create Tensor Flow Lite Model variable and initialize it.

val birdModel = BirdsModel.newInstance(this)

2. Converting input image into tensor flow image. as you can see input image is require to be in bitmap format.

val tfImage = TensorImage.fromBitmap(bitmap)

3. Processing the image and then sorting output results into descending order then picking top result (pick out highest probability one’s).

val outputs = birdModel.process(tfImage)

    .probabilityAsCategoryList.apply {

        sortByDescending { it.score }

    }

//getting result having high probability

val highProbabilityOutput = outputs[0]

4. Dispalying the result into TextView

tvOutput.text = highProbabilityOutput.label

Prepare layout for Your app

This layout contains following views,

• Image view, for displaying our captured bird photo.
• Button, for launching the camera to take the photo.
• Text view, to display output result.

<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context=".MainActivity">
    <Button
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:id="@+id/btn_load_image"
        android:text="Load Image"
        app:layout_constraintEnd_toStartOf="@id/guideline2"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toBottomOf="@+id/imageView"
        android:layout_marginTop="18dp"/>
    <ImageView
        android:layout_width="300dp"
        android:layout_height="450dp"
        android:id="@+id/imageView"
        android:layout_marginTop="16dp"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toTopOf="parent"
        app:layout_constraintHorizontal_bias="0.5"
        android:src="@drawable/place_holder"/>
    <Button
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:id="@+id/btn_capture_image"
        android:text="Take Image"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="@id/guideline2"
        app:layout_constraintTop_toBottomOf="@id/imageView"
        android:layout_marginTop="18dp"/>
    <TextView
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:id="@+id/textview"
        android:text="Output: "
        android:textSize="21sp"
        android:layout_marginTop="24dp"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toBottomOf="@id/btn_capture_image"
        android:layout_marginStart="36dp"
        android:textColor="@android:color/black"/>
    <TextView
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:id="@+id/tv_output"
        android:text="Result here"
        android:textSize="21sp"
        android:textColor="@android:color/holo_red_dark"
        app:layout_constraintStart_toEndOf="@id/textview"
        app:layout_constraintTop_toBottomOf="@id/btn_capture_image"
        android:layout_marginTop="24dp" />
    <TextView
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="Click on the result text to search on Google"
        app:layout_constraintTop_toBottomOf="@id/textview"
        android:layout_marginTop="8dp"
        android:layout_marginStart="36dp"
        app:layout_constraintStart_toStartOf="parent"
        android:textSize="13sp"/>

    <androidx.constraintlayout.widget.Guideline
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:id="@+id/guideline2"
        app:layout_constraintGuide_percent="0.5"
        android:orientation="vertical"/>
</androidx.constraintlayout.widget.ConstraintLayout>

Complete MainActivity.Kt

package com.codewithgolap.birdclassificationkotlin

import android.app.Activity

import android.app.Instrumentation

import android.content.ContentValues

import android.content.Intent

import android.content.pm.PackageManager

import android.graphics.Bitmap

import android.graphics.BitmapFactory

import android.graphics.drawable.BitmapDrawable

import android.net.Uri

import androidx.appcompat.app.AppCompatActivity

import android.os.Bundle

import android.provider.MediaStore

import android.util.Log

import android.widget.Button

import android.widget.ImageView

import android.widget.TextView

import android.widget.Toast

import androidx.activity.result.ActivityResult

import androidx.activity.result.contract.ActivityResultContracts

import androidx.appcompat.app.AlertDialog

import androidx.core.content.ContextCompat

import com.codewithgolap.birdclassificationkotlin.databinding.ActivityMainBinding

import com.codewithgolap.birdclassificationkotlin.ml.BirdsModel

import org.tensorflow.lite.support.image.TensorImage

import java.io.IOException

class MainActivity : AppCompatActivity() {

    private lateinit var binding: ActivityMainBinding

    private lateinit var imageView: ImageView

    private lateinit var button: Button

    private lateinit var tvOutput: TextView

    private val GALLERY_REQUEST_CODE = 123

    override fun onCreate(savedInstanceState: Bundle?) {

        super.onCreate(savedInstanceState)

        binding = ActivityMainBinding.inflate(layoutInflater)

        val view = binding.root

        setContentView(view)

        imageView = binding.imageView

        button = binding.btnCaptureImage

        tvOutput = binding.tvOutput

        val buttonLoad = binding.btnLoadImage

        button.setOnClickListener {

            if (ContextCompat.checkSelfPermission(this, android.Manifest.permission.CAMERA)

                == PackageManager.PERMISSION_GRANTED

            ) {

                takePicturePreview.launch(null)

            }

            else {

                requestPermission.launch(android.Manifest.permission.CAMERA)

            }

        }

        buttonLoad.setOnClickListener {

            if (ContextCompat.checkSelfPermission(this, android.Manifest.permission.READ_EXTERNAL_STORAGE)

            == PackageManager.PERMISSION_GRANTED){

                val intent = Intent(Intent.ACTION_PICK, MediaStore.Images.Media.EXTERNAL_CONTENT_URI)

                intent.type = "image/*"

                val mimeTypes = arrayOf("image/jpeg","image/png","image/jpg")

                intent.putExtra(Intent.EXTRA_MIME_TYPES, mimeTypes)

                intent.flags = Intent.FLAG_GRANT_READ_URI_PERMISSION

                onresult.launch(intent)

            }else {

                requestPermission.launch(android.Manifest.permission.READ_EXTERNAL_STORAGE)

            }

        }

        //to redirct user to google search for the scientific name

        tvOutput.setOnClickListener {

            val intent = Intent(Intent.ACTION_VIEW, Uri.parse("https://www.google.com/search?q=${tvOutput.text}"))

            startActivity(intent)

        }

        // to download image when longPress on ImageView

        imageView.setOnLongClickListener {

            requestPermissionLauncher.launch(android.Manifest.permission.WRITE_EXTERNAL_STORAGE)

            return@setOnLongClickListener true

        }

    }

    //request camera permission

    private val requestPermission = registerForActivityResult(ActivityResultContracts.RequestPermission()){granted->

        if (granted){

            takePicturePreview.launch(null)

        }else {

            Toast.makeText(this, "Permission Denied !! Try again", Toast.LENGTH_SHORT).show()

        }

    }

    //launch camera and take picture

    private val takePicturePreview = registerForActivityResult(ActivityResultContracts.TakePicturePreview()){bitmap->

        if(bitmap != null){

            imageView.setImageBitmap(bitmap)

            outputGenerator(bitmap)

        }

    }

    //to get image from gallery

    private val onresult = registerForActivityResult(ActivityResultContracts.StartActivityForResult()){result->

        Log.i("TAG", "This is the result: ${result.data} ${result.resultCode}")

        onResultReceived(GALLERY_REQUEST_CODE,result)

    }

    private fun  onResultReceived(requestCode: Int, result: ActivityResult?){

        when(requestCode){

            GALLERY_REQUEST_CODE ->{

                if (result?.resultCode == Activity.RESULT_OK){

                    result.data?.data?.let{uri ->

                        Log.i("TAG", "onResultReceived: $uri")

                        val bitmap = BitmapFactory.decodeStream(contentResolver.openInputStream(uri))

                        imageView.setImageBitmap(bitmap)

                        outputGenerator(bitmap)

                    }

                }else {

                    Log.e("TAG", "onActivityResult: error in selecting image")

                }

            }

        }

    }

    private fun outputGenerator(bitmap: Bitmap){

        //declearing tensor flow lite model variable

        val birdsModel = BirdsModel.newInstance(this)

        // converting bitmap into tensor flow image

        val newBitmap = bitmap.copy(Bitmap.Config.ARGB_8888, true)

        val tfimage = TensorImage.fromBitmap(newBitmap)

        //process the image using trained model and sort it in descending order

        val outputs = birdsModel.process(tfimage)

            .probabilityAsCategoryList.apply {

                sortByDescending { it.score }

            }

        //getting result having high probability

        val highProbabilityOutput = outputs[0]

        //setting ouput text

        tvOutput.text = highProbabilityOutput.label

        Log.i("TAG", "outputGenerator: $highProbabilityOutput")

    }

    // to download image to device

    private val requestPermissionLauncher = registerForActivityResult(ActivityResultContracts.RequestPermission()){

        isGranted: Boolean ->

        if (isGranted){

            AlertDialog.Builder(this).setTitle("Download Image?")

                .setMessage("Do you want to download this image to your device?")

                .setPositiveButton("Yes"){_, _ ->

                    val drawable:BitmapDrawable = imageView.drawable as BitmapDrawable

                    val bitmap = drawable.bitmap

                    downloadImage(bitmap)

                }

                .setNegativeButton("No") {dialog, _ ->

                    dialog.dismiss()

                }

                .show()

        }else {

            Toast.makeText(this, "Please allow permission to download image", Toast.LENGTH_LONG).show()

        }

    }

    //fun that takes a bitmap and store to user's device

    private fun downloadImage(mBitmap: Bitmap):Uri? {

        val contentValues = ContentValues().apply {

            put(MediaStore.Images.Media.DISPLAY_NAME,"Birds_Images"+ System.currentTimeMillis()/1000)

            put(MediaStore.Images.Media.MIME_TYPE,"image/png")

        }

        val uri = MediaStore.Images.Media.EXTERNAL_CONTENT_URI

        if (uri != null){

            contentResolver.insert(uri, contentValues)?.also {

                contentResolver.openOutputStream(it).use { outputStream ->

                    if (!mBitmap.compress(Bitmap.CompressFormat.PNG, 100, outputStream)){

                        throw IOException("Couldn't save the bitmap")

                    }

                    else{

                        Toast.makeText(applicationContext, "Image Saved", Toast.LENGTH_LONG).show()

                    }

                }

                return it

            }

        }

        return null

    }

}

Final Result:

Watch the full video on youtube: