Skip to main content

Integrate into your Android app

Register an SDK Key​

Get your free SDK key on https://dev.quickpose.ai, usage limits may apply. SDK Keys are linked to your bundle ID, please check Key before distributing to the App Store.

Installing the SDK​

Maven​

Step 1: Open your app's build.gradle:

Step 2: Add the Maven package and it's dependencies:

dependencies {
    // recommended for starting quickpose
    implementation ("androidx.lifecycle:lifecycle-runtime-ktx:2.6.1'")
    
    // CameraX core library
    val camerax_version = ("1.4.1")
    
    implementation("androidx.camera:camera-core:$camerax_version")
    implementation("androidx.camera:camera-camera2:$camerax_version")
    implementation("androidx.camera:camera-lifecycle:$camerax_version")
    implementation("androidx.camera:camera-view:$camerax_version")

    implementation("com.google.flogger:flogger:latest.release")
    implementation("com.google.flogger:flogger-system-backend:latest.release")
    implementation("com.google.guava:guava:27.0.1-android")
    implementation("com.google.protobuf:protobuf-javalite:3.19.1")

    implementation("com.microsoft.onnxruntime:onnxruntime-android:latest.release")
    implementation("ai.quickpose:quickpose-mp:0.1")
    implementation("ai.quickpose:quickpose-core:0.5")
}

Add Camera Permissions Check To Main App​

Next, you have to explicitly request access to the camera, which will provide the Android standard camera permission prompt. This is only a demo implementation, as you'd typically want to give the user an idea of what your app does first and why camera permissions help them.

class MainActivity : ComponentActivity() {
    private var hasPermissions = mutableStateOf(false)

    override fun onCreate(savedInstanceState: Bundle?) {
        hasPermissions.value = PermissionsHelper.checkAndRequestCameraPermissions(this)
    }

    override fun onResume() {
        super.onResume()

        hasPermissions.value = PermissionsHelper.checkAndRequestCameraPermissions(this)
        if (!hasPermissions.value) {
            return
        }
    }

We use this standard permission helper:

public class PermissionsHelper {
    companion object {
        fun checkAndRequestCameraPermissions(context: Activity): Boolean {
            val hasPermission = cameraPermissionsGranted(context)
            if (!hasPermission) {
                ActivityCompat.requestPermissions(context, arrayOf(Manifest.permission.CAMERA), 0);
            }
            return hasPermission
        }

        fun cameraPermissionsGranted(context: Activity): Boolean {
            return ContextCompat.checkSelfPermission(
                context,
                Manifest.permission.CAMERA
            ) == PackageManager.PERMISSION_GRANTED
        }
    }
}

Initialize the SDK​

private var quickPose: QuickPose =
        QuickPose(this, sdkKey = "YOUR SDK KEY HERE"
        ) // register for your free key at https://dev.quickpose.ai

Attach SDK to Views​

This is our standard boilerplate implentation providing:

  1. Camera Permission Checking
  2. A fullscreen camera display.
  3. A canvas overlay showing the AI user's landmarks.
  4. Sensible memory releasing when the view is no longer visible.

Declare an xml definition the quickpose camera and overlay stack.

<FrameLayout
        android:id="@+id/quickpose_camera_and_overlay_view"
        android:layout_width="match_parent"
        android:layout_height="0dp"
        app:layout_constraintTop_toTopOf="parent"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintEnd_toEndOf="parent">
</FrameLayout>

Declare a reference to the camera and overlay view group.

private var cameraAndOverlay: ViewGroup? = null
Plus a reference to the camera view explicitly.
private var cameraSwitchView: QuickPoseCameraSwitchView? = null

Connect up the views

override fun onCreate(savedInstanceState: Bundle?) {
        cameraAndOverlay = findViewById(R.id.quickpose_camera_and_overlay_view)
        cameraSwitchView = QuickPoseCameraSwitchView(this, quickPose)
        cameraAndOverlay?.addView(cameraSwitchView)

And on resume, start the camera and overlay stack and quickpose. This is a bit simplified as you need to request camera permissions.

override fun onResume() {
        super.onResume()
        lifecycleScope.launch {
            cameraSwitchView?.start(useFrontCamera)!!
            quickPose.start(
                    arrayOf(Feature.RangeOfMotion(RangeOfMotion.Shoulder(Side.LEFT, false))),
                    onFrame = { status, overlay, features, feedback, landmarks ->
                        landmarks?.let { println(it.allLandmarksForBody()[0].x) }
                    }
            )
        }
}

View the raw landmarks​

landmarks?.let { println(it.allLandmarksForBody()[0]) }

Access the overlay image.​

For performance reasons, the overlay is returned as a canvas. This differs from iOS, use PixelCopy to efficiently convert the canvas to a bitmap.

quickPose.start(
    arrayOf(Feature.Overlay(Landmarks.Group.WholeBody())),
    onFrame = { status, overlay, features, feedback, landmarks ->         
        overlay?.let { outputOverlay ->
            val bitmap =
                    Bitmap.createBitmap(
                            outputOverlay.width,
                            outputOverlay.height,
                            Bitmap.Config.ARGB_8888
                    )

            PixelCopy.request(
                    outputOverlay,
                    bitmap,
                    { copyResult ->
                        if (copyResult == PixelCopy.SUCCESS) {
                            println(
                                    "Saved Bitmap ${bitmap.height}, ${bitmap.width}"
                            )
                        }
                    },
                    Handler(Looper.getMainLooper())
            )
        }
    }
)