Ticker

6/recent/ticker-posts

Image Classification Android App with TensorFlow Lite for Beginner

Today, Machine Learning (ML) is all over the place. The ML reduces human works, suppose I'm trying to identify birds, as a human what we generally do we search for every bird that is present in this world and identify the particular one. But Machine Learning reduces that, in ML we will simply create one model which involves every bird. Then when I tried to search for any birds, that model will simply tell me which bird it is.



So in this blog, we will see how to connect a tflite model in android or how to use a tflite model in the android app.

In this tutorial, we’ll look at how to use TensorFlow Lite to integrate machine learning into an Android app by creating a basic app.


TensorFlow Lite

TensorFlow Lite is a set of tools that enables on-device machine learning by helping developers run their models on mobile, embedded, and IoT devices.

Key Features of TF-Lite:


In order to start, you will require a trained .tflite model that you can download from here.


Note: after downloading name this file as BirdsModel.tflite.




Add TensorFlow Lite to the Android app.


Step 1Head over to android studio & Create a new android project.



Step 2: Add TensorFlow Lite to the Android App.

Right-click on the package name in my case it is com.yourpackagename or click on File, then New > Other > TensorFlow Lite Model. Select the model location where you have downloaded the custom trained BirdsModel.tflite earlier





Step 3: Click finish.




Note that the tooling will automatically configure the module’s dependencies for you using ML Model binding and all requirements will be added into your Android module’s build.gradle file.


Step 4: At the end, you’ll see the following. The BirdModel.tflite file has been successfully imported, and it displays high-level model information like as input and output, as well as some sample code to get you started.




Processing image and showing result


following are the simple steps to implement the bird classification model.


1. Create Tensor Flow Lite Model variable and initialize it.


val birdModel = BirdsModel.newInstance(this)


2. Converting input image into tensor flow image. as you can see input image is require to be in bitmap format.


val tfImage = TensorImage.fromBitmap(bitmap)


3. Processing the image and then sorting output results into descending order then picking top result (pick out highest probability one’s).


val outputs = birdModel.process(tfImage)

    .probabilityAsCategoryList.apply {

        sortByDescending { it.score }

    }


//getting result having high probability

val highProbabilityOutput = outputs[0]


4. Dispalying the result into TextView


tvOutput.text = highProbabilityOutput.label


Prepare layout for Your app


This layout contains following views,

  • Image view, for displaying our captured bird photo.
  • Button, for launching the camera to take the photo.
  • Text view, to display output result.


<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context=".MainActivity">
    <Button
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:id="@+id/btn_load_image"
        android:text="Load Image"
        app:layout_constraintEnd_toStartOf="@id/guideline2"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toBottomOf="@+id/imageView"
        android:layout_marginTop="18dp"/>
    <ImageView
        android:layout_width="300dp"
        android:layout_height="450dp"
        android:id="@+id/imageView"
        android:layout_marginTop="16dp"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toTopOf="parent"
        app:layout_constraintHorizontal_bias="0.5"
        android:src="@drawable/place_holder"/>
    <Button
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:id="@+id/btn_capture_image"
        android:text="Take Image"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="@id/guideline2"
        app:layout_constraintTop_toBottomOf="@id/imageView"
        android:layout_marginTop="18dp"/>
    <TextView
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:id="@+id/textview"
        android:text="Output: "
        android:textSize="21sp"
        android:layout_marginTop="24dp"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toBottomOf="@id/btn_capture_image"
        android:layout_marginStart="36dp"
        android:textColor="@android:color/black"/>
    <TextView
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:id="@+id/tv_output"
        android:text="Result here"
        android:textSize="21sp"
        android:textColor="@android:color/holo_red_dark"
        app:layout_constraintStart_toEndOf="@id/textview"
        app:layout_constraintTop_toBottomOf="@id/btn_capture_image"
        android:layout_marginTop="24dp" />
    <TextView
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="Click on the result text to search on Google"
        app:layout_constraintTop_toBottomOf="@id/textview"
        android:layout_marginTop="8dp"
        android:layout_marginStart="36dp"
        app:layout_constraintStart_toStartOf="parent"
        android:textSize="13sp"/>

    <androidx.constraintlayout.widget.Guideline
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:id="@+id/guideline2"
        app:layout_constraintGuide_percent="0.5"
        android:orientation="vertical"/>
</androidx.constraintlayout.widget.ConstraintLayout>


Complete MainActivity.Kt

package com.codewithgolap.birdclassificationkotlin

import android.app.Activity
import android.app.Instrumentation
import android.content.ContentValues
import android.content.Intent
import android.content.pm.PackageManager
import android.graphics.Bitmap
import android.graphics.BitmapFactory
import android.graphics.drawable.BitmapDrawable
import android.net.Uri
import androidx.appcompat.app.AppCompatActivity
import android.os.Bundle
import android.provider.MediaStore
import android.util.Log
import android.widget.Button
import android.widget.ImageView
import android.widget.TextView
import android.widget.Toast
import androidx.activity.result.ActivityResult
import androidx.activity.result.contract.ActivityResultContracts
import androidx.appcompat.app.AlertDialog
import androidx.core.content.ContextCompat
import com.codewithgolap.birdclassificationkotlin.databinding.ActivityMainBinding
import com.codewithgolap.birdclassificationkotlin.ml.BirdsModel
import org.tensorflow.lite.support.image.TensorImage
import java.io.IOException

class MainActivity : AppCompatActivity() {

    private lateinit var binding: ActivityMainBinding
    private lateinit var imageView: ImageView
    private lateinit var button: Button
    private lateinit var tvOutput: TextView
    private val GALLERY_REQUEST_CODE = 123


    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        binding = ActivityMainBinding.inflate(layoutInflater)
        val view = binding.root
        setContentView(view)

        imageView = binding.imageView
        button = binding.btnCaptureImage
        tvOutput = binding.tvOutput
        val buttonLoad = binding.btnLoadImage

        button.setOnClickListener {
            if (ContextCompat.checkSelfPermission(this, android.Manifest.permission.CAMERA)
                == PackageManager.PERMISSION_GRANTED
            ) {
                takePicturePreview.launch(null)
            }
            else {
                requestPermission.launch(android.Manifest.permission.CAMERA)
            }
        }
        buttonLoad.setOnClickListener {
            if (ContextCompat.checkSelfPermission(this, android.Manifest.permission.READ_EXTERNAL_STORAGE)
            == PackageManager.PERMISSION_GRANTED){
                val intent = Intent(Intent.ACTION_PICK, MediaStore.Images.Media.EXTERNAL_CONTENT_URI)
                intent.type = "image/*"
                val mimeTypes = arrayOf("image/jpeg","image/png","image/jpg")
                intent.putExtra(Intent.EXTRA_MIME_TYPES, mimeTypes)
                intent.flags = Intent.FLAG_GRANT_READ_URI_PERMISSION
                onresult.launch(intent)
            }else {
                requestPermission.launch(android.Manifest.permission.READ_EXTERNAL_STORAGE)
            }
        }

        //to redirct user to google search for the scientific name
        tvOutput.setOnClickListener {
            val intent = Intent(Intent.ACTION_VIEW, Uri.parse("https://www.google.com/search?q=${tvOutput.text}"))
            startActivity(intent)
        }

        // to download image when longPress on ImageView
        imageView.setOnLongClickListener {
            requestPermissionLauncher.launch(android.Manifest.permission.WRITE_EXTERNAL_STORAGE)
            return@setOnLongClickListener true
        }

    }

    //request camera permission
    private val requestPermission = registerForActivityResult(ActivityResultContracts.RequestPermission()){granted->
        if (granted){
            takePicturePreview.launch(null)
        }else {
            Toast.makeText(this, "Permission Denied !! Try again", Toast.LENGTH_SHORT).show()
        }
    }

    //launch camera and take picture
    private val takePicturePreview = registerForActivityResult(ActivityResultContracts.TakePicturePreview()){bitmap->
        if(bitmap != null){
            imageView.setImageBitmap(bitmap)
            outputGenerator(bitmap)
        }
    }

    //to get image from gallery
    private val onresult = registerForActivityResult(ActivityResultContracts.StartActivityForResult()){result->
        Log.i("TAG", "This is the result: ${result.data} ${result.resultCode}")
        onResultReceived(GALLERY_REQUEST_CODE,result)
    }

    private fun  onResultReceived(requestCode: Int, result: ActivityResult?){
        when(requestCode){
            GALLERY_REQUEST_CODE ->{
                if (result?.resultCode == Activity.RESULT_OK){
                    result.data?.data?.let{uri ->
                        Log.i("TAG", "onResultReceived: $uri")
                        val bitmap = BitmapFactory.decodeStream(contentResolver.openInputStream(uri))
                        imageView.setImageBitmap(bitmap)
                        outputGenerator(bitmap)
                    }
                }else {
                    Log.e("TAG", "onActivityResult: error in selecting image")
                }
            }
        }
    }

    private fun outputGenerator(bitmap: Bitmap){
        //declearing tensor flow lite model variable

        val birdsModel = BirdsModel.newInstance(this)

        // converting bitmap into tensor flow image
        val newBitmap = bitmap.copy(Bitmap.Config.ARGB_8888, true)
        val tfimage = TensorImage.fromBitmap(newBitmap)

        //process the image using trained model and sort it in descending order
        val outputs = birdsModel.process(tfimage)
            .probabilityAsCategoryList.apply {
                sortByDescending { it.score }
            }

        //getting result having high probability
        val highProbabilityOutput = outputs[0]

        //setting ouput text
        tvOutput.text = highProbabilityOutput.label
        Log.i("TAG", "outputGenerator: $highProbabilityOutput")

    }

    // to download image to device
    private val requestPermissionLauncher = registerForActivityResult(ActivityResultContracts.RequestPermission()){
        isGranted: Boolean ->
        if (isGranted){
            AlertDialog.Builder(this).setTitle("Download Image?")
                .setMessage("Do you want to download this image to your device?")
                .setPositiveButton("Yes"){_, _ ->
                    val drawable:BitmapDrawable = imageView.drawable as BitmapDrawable
                    val bitmap = drawable.bitmap
                    downloadImage(bitmap)
                }
                .setNegativeButton("No") {dialog, _ ->
                    dialog.dismiss()
                }
                .show()
        }else {
            Toast.makeText(this, "Please allow permission to download image", Toast.LENGTH_LONG).show()
        }
    }

    //fun that takes a bitmap and store to user's device
    private fun downloadImage(mBitmap: Bitmap):Uri? {
        val contentValues = ContentValues().apply {
            put(MediaStore.Images.Media.DISPLAY_NAME,"Birds_Images"+ System.currentTimeMillis()/1000)
            put(MediaStore.Images.Media.MIME_TYPE,"image/png")
        }
        val uri = MediaStore.Images.Media.EXTERNAL_CONTENT_URI
        if (uri != null){
            contentResolver.insert(uri, contentValues)?.also {
                contentResolver.openOutputStream(it).use { outputStream ->
                    if (!mBitmap.compress(Bitmap.CompressFormat.PNG, 100, outputStream)){
                        throw IOException("Couldn't save the bitmap")
                    }
                    else{
                        Toast.makeText(applicationContext, "Image Saved", Toast.LENGTH_LONG).show()
                    }
                }
                return it
            }
        }
        return null
    }
}


Final Result:


Watch the full video on youtube:




Post a Comment

0 Comments