Simplify IAMF extension build process

Removed `Android.mk` and `Application.mk`, allowing `CMake` to run directly from the `build.gradle` file. Users no longer need to check out `NDK` or depend on it, simplifying the usage of the IAMF extension.

PiperOrigin-RevId: 684471874

RELEASENOTES.md

@@ -112,6 +112,10 @@
     *   Fix bug where a Media3 controller was sometimes unable to let a session
         app start a foreground service after requesting `play()`.
     *   Restrict `CommandButton.Builder.setIconUri` to only accept content Uris.
+    *   Pass connection hints of a Media3 browser to the initial
+        `MediaBrowserCompat` when connecting to a legacy `MediaBrowserCompat`.
+        The service can receive the connection hints passed in as root hints
+        with the first call to `onGetRoot()`.
 *   UI:
     *   Make the stretched/cropped video in
         `PlayerView`-in-Compose-`AndroidView` workaround opt-in, due to issues

libraries/common/src/main/java/androidx/media3/common/audio/SpeedChangingAudioProcessor.java

@@ -115,23 +115,23 @@ public final class SpeedChangingAudioProcessor extends BaseAudioProcessor {
 
   @Override
   public void queueInput(ByteBuffer inputBuffer) {
-    long timeUs =
+    long currentTimeUs =
         Util.scaleLargeTimestamp(
             /* timestamp= */ bytesRead,
             /* multiplier= */ C.MICROS_PER_SECOND,
             /* divisor= */ (long) inputAudioFormat.sampleRate * inputAudioFormat.bytesPerFrame);
-    float newSpeed = speedProvider.getSpeed(timeUs);
+    float newSpeed = speedProvider.getSpeed(currentTimeUs);
 
-    updateSpeed(newSpeed, timeUs);
+    updateSpeed(newSpeed, currentTimeUs);
 
     int inputBufferLimit = inputBuffer.limit();
-    long nextSpeedChangeTimeUs = speedProvider.getNextSpeedChangeTimeUs(timeUs);
+    long nextSpeedChangeTimeUs = speedProvider.getNextSpeedChangeTimeUs(currentTimeUs);
     int bytesToNextSpeedChange;
     if (nextSpeedChangeTimeUs != C.TIME_UNSET) {
       bytesToNextSpeedChange =
           (int)
               Util.scaleLargeValue(
-                  /* timestamp= */ nextSpeedChangeTimeUs - timeUs,
+                  /* timestamp */ nextSpeedChangeTimeUs - currentTimeUs,
                   /* multiplier= */ (long) inputAudioFormat.sampleRate
                       * inputAudioFormat.bytesPerFrame,
                   /* divisor= */ C.MICROS_PER_SECOND,

libraries/decoder_iamf/README.md

@@ -1,7 +1,7 @@
 # IAMF decoder module
 
-The IAMF module provides `LibiamfAudioRenderer`, which uses libiamf (the IAMF
+The IAMF module provides `LibiamfAudioRenderer`, which uses the libiamf native
-decoding library) to decode IAMF audio.
+library to decode IAMF audio.
 
 ## License note
 
@@ -17,58 +17,33 @@ To use the module you need to clone this GitHub project and depend on its
 modules locally. Instructions for doing this can be found in the
 [top level README][].
 
-In addition, it's necessary to build the module's native components as follows:
+In addition, it's necessary to fetch libiamf as follows:
 
 * Set the following environment variables:
 
 ```
 cd "<path to project checkout>"
-
 IAMF_MODULE_PATH="$(pwd)/libraries/decoder_iamf/src/main"
 ```
 
-*   Download the [Android NDK][] and set its location in an environment
+* Fetch libiamf:
-    variable. This build configuration has been tested on NDK r27.
 
 ```
-NDK_PATH="<path to Android NDK>"
+cd "${IAMF_MODULE_PATH}/jni" && \
+git clone https://github.com/AOMediaCodec/libiamf.git
 ```
 
-*   Fetch libiamf:
+* [Install CMake][]
 
-Clone the repository containing libiamf to a local folder of choice - preferably
+Having followed these steps, gradle will build the module automatically when run
-outside of the project checkout. Link it to the project's jni folder through
+on the command line or via Android Studio, using [CMake][] and [Ninja][] to
-symlink.
+configure and build libiamf and the module's [JNI wrapper library][].
-
-```
-cd <preferred location for libiamf>
-
-
-git clone https://github.com/AOMediaCodec/libiamf.git libiamf && \
-cd libiamf && \
-LIBIAMF_PATH=$(pwd)
-```
-
-*   Symlink the folder containing libiamf to the project's JNI folder and run
-    the script to convert libiamf code to NDK compatible format:
-
-```
-cd "${IAMF_MODULE_PATH}"/jni && \
-ln -s $LIBIAMF_PATH libiamf && \
-cd libiamf/code &&\
-cmake . && \
-make
-```
-
-* Build the JNI native libraries from the command line:
-
-```
-cd "${IAMF_MODULE_PATH}"/jni && \
-${NDK_PATH}/ndk-build APP_ABI=all -j4
-```
 
 [top level README]: ../../README.md
-[Android NDK]: https://developer.android.com/tools/sdk/ndk/index.html
+[Install CMake]: https://developer.android.com/studio/projects/install-ndk
+[CMake]: https://cmake.org/
+[Ninja]: https://ninja-build.org
+[JNI wrapper library]: src/main/jni/iamf_jni.cc
 
 ## Build instructions (Windows)
 
@@ -77,13 +52,6 @@ be possible to follow the Linux instructions in [Windows PowerShell][].
 
 [Windows PowerShell]: https://docs.microsoft.com/en-us/powershell/scripting/getting-started/getting-started-with-windows-powershell
 
-## Notes
-
-*   Every time there is a change to the libiamf checkout clean and re-build the
-    project.
-*   If you want to use your own version of libiamf, place it in
-    `${IAMF_MODULE_PATH}/jni/libiamf`.
-
 ## Using the module with ExoPlayer
 
 Once you've followed the instructions above to check out, build and depend on

libraries/decoder_iamf/build.gradle

@@ -1,4 +1,4 @@
-// Copyright (C) 2024 The Android Open Source Project
+// Copyright 2024 The Android Open Source Project
 //
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
@@ -17,12 +17,32 @@ android {
     namespace 'androidx.media3.decoder.iamf'
 
     sourceSets {
-        main {
-            jniLibs.srcDir 'src/main/libs'
-            jni.srcDirs = [] // Disable the automatic ndk-build call by Android Studio.
-        }
         androidTest.assets.srcDir '../test_data/src/test/assets'
     }
+
+    defaultConfig {
+        externalNativeBuild {
+            cmake {
+                targets "iamfJNI"
+            }
+        }
+    }
+}
+
+// Configure the native build only if libiamf is present to avoid gradle sync
+// failures if libiamf hasn't been built according to the README instructions.
+if (project.file('src/main/jni/libiamf').exists()) {
+    android.externalNativeBuild.cmake {
+        path = 'src/main/jni/CMakeLists.txt'
+        version = '3.21.0+'
+        if (project.hasProperty('externalNativeBuildDir')) {
+            if (!new File(externalNativeBuildDir).isAbsolute()) {
+                ext.externalNativeBuildDir =
+                        new File(rootDir, it.externalNativeBuildDir)
+            }
+            buildStagingDirectory = "${externalNativeBuildDir}/${project.name}"
+        }
+    }
 }
 
 dependencies {

libraries/decoder_iamf/src/main/jni/Android.mk

@@ -1,32 +0,0 @@
-#
-# Copyright (C) 2024 The Android Open Source Project
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      https://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-WORKING_DIR := $(call my-dir)
-include $(CLEAR_VARS)
-
-# build libiamf.a
-LOCAL_PATH := $(WORKING_DIR)
-include libiamf.mk
-
-# build libiamfJNI.so
-include $(CLEAR_VARS)
-LOCAL_PATH := $(WORKING_DIR)
-LOCAL_MODULE := libiamfJNI
-LOCAL_ARM_MODE := arm
-LOCAL_CPP_EXTENSION := .cc
-LOCAL_SRC_FILES := iamf_jni.cc
-LOCAL_LDLIBS := -llog -lz -lm
-LOCAL_STATIC_LIBRARIES := libiamf
-include $(BUILD_SHARED_LIBRARY)

libraries/decoder_iamf/src/main/jni/Application.mk

@@ -1,18 +0,0 @@
-#
-# Copyright (C) 2024 The Android Open Source Project
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      https://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-APP_ABI := all
-APP_PLATFORM := android-21

libraries/decoder_iamf/src/main/jni/CMakeLists.txt

@@ -0,0 +1,46 @@
+#
+# Copyright 2024 The Android Open Source Project
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+cmake_minimum_required(VERSION 3.21.0 FATAL_ERROR)
+
+# Enable C++11 features.
+set(CMAKE_CXX_STANDARD 11)
+
+# Define project name for your JNI module
+project(libiamfJNI C CXX)
+
+set(libiamf_jni_root "${CMAKE_CURRENT_SOURCE_DIR}")
+
+# Build libiamf.
+add_subdirectory("${libiamf_jni_root}/libiamf/code"
+                 EXCLUDE_FROM_ALL)
+
+# Add the include directory from libiamf.
+include_directories ("${libiamf_jni_root}/libiamf/code/include")
+
+# Build libiamfJNI.
+add_library(iamfJNI
+            SHARED
+            iamf_jni.cc)
+
+# Locate NDK log library.
+find_library(android_log_lib log)
+
+# Link libgav1JNI against used libraries.
+target_link_libraries(iamfJNI
+                      PRIVATE android
+                      PRIVATE iamf
+                      PRIVATE ${android_log_lib})

libraries/decoder_iamf/src/main/jni/libiamf.mk

@@ -1,35 +0,0 @@
-#
-# Copyright (C) 2024 The Android Open Source Project
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      https://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-LOCAL_PATH := $(WORKING_DIR)/libiamf
-
-include $(CLEAR_VARS)
-
-LOCAL_MODULE := libiamf
-LOCAL_ARM_MODE := arm
-LOCAL_C_INCLUDES := $(LOCAL_PATH)/code/include \
-                    $(LOCAL_PATH)/code/src/iamf_dec \
-                    $(LOCAL_PATH)/code/src/common \
-                    $(LOCAL_PATH)/code/dep_codecs/include \
-                    $(LOCAL_PATH)/code/dep_external/include
-LOCAL_SRC_FILES := $(shell find $(LOCAL_PATH)/code/src -name "*.c")
-LOCAL_EXPORT_C_INCLUDES := $(LOCAL_PATH)/code/include \
-                    $(LOCAL_PATH)/code/src/iamf_dec \
-                    $(LOCAL_PATH)/code/src/common \
-                    $(LOCAL_PATH)/code/dep_codecs/include \
-                    $(LOCAL_PATH)/code/dep_external/include
-
-include $(BUILD_STATIC_LIBRARY)

libraries/muxer/src/main/java/androidx/media3/muxer/Boxes.java

@@ -898,7 +898,8 @@ import org.checkerframework.checker.nullness.qual.PolyNull;
         ByteBuffer.allocate(
             2 * BYTES_PER_INTEGER + 2 * compositionOffsets.size() * BYTES_PER_INTEGER);
 
-    contents.putInt(1); // version and flags.
+    int versionAndFlags = 1 << 24; // version (value 1, 8 bits) + flag (value 0, 24 bits)
+    contents.putInt(versionAndFlags);
 
     // Total entry count is known only after processing all the composition offsets, so put in
     // a placeholder for total entry count and store its index.

libraries/session/src/main/java/androidx/media3/session/MediaBrowser.java

@@ -292,7 +292,7 @@ public final class MediaBrowser extends MediaController {
     if (token.isLegacySession()) {
       impl =
           new MediaBrowserImplLegacy(
-              context, this, token, applicationLooper, checkNotNull(bitmapLoader));
+              context, this, token, connectionHints, applicationLooper, checkNotNull(bitmapLoader));
     } else {
       impl = new MediaBrowserImplBase(context, this, token, connectionHints, applicationLooper);
     }

libraries/session/src/main/java/androidx/media3/session/MediaBrowserImplLegacy.java

@@ -62,9 +62,10 @@ import org.checkerframework.checker.initialization.qual.UnderInitialization;
       Context context,
       @UnderInitialization MediaBrowser instance,
       SessionToken token,
+      Bundle connectionHints,
       Looper applicationLooper,
       BitmapLoader bitmapLoader) {
-    super(context, instance, token, applicationLooper, bitmapLoader);
+    super(context, instance, token, connectionHints, applicationLooper, bitmapLoader);
     this.instance = instance;
     commandButtonsForMediaItems = ImmutableMap.of();
   }

libraries/session/src/main/java/androidx/media3/session/MediaController.java

@@ -568,7 +568,7 @@ public class MediaController implements Player {
       @Nullable BitmapLoader bitmapLoader) {
     if (token.isLegacySession()) {
       return new MediaControllerImplLegacy(
-          context, this, token, applicationLooper, checkNotNull(bitmapLoader));
+          context, this, token, connectionHints, applicationLooper, checkNotNull(bitmapLoader));
     } else {
       return new MediaControllerImplBase(context, this, token, connectionHints, applicationLooper);
     }
@@ -2073,6 +2073,10 @@ public class MediaController implements Player {
     return impl.getBinder();
   }
 
+  /* package */ Bundle getConnectionHints() {
+    return impl.getConnectionHints();
+  }
+
   private void verifyApplicationThread() {
     checkState(Looper.myLooper() == getApplicationLooper(), WRONG_THREAD_ERROR_MESSAGE);
   }
@@ -2081,6 +2085,8 @@ public class MediaController implements Player {
 
     void connect(@UnderInitialization MediaControllerImpl this);
 
+    Bundle getConnectionHints();
+
     void addListener(Player.Listener listener);
 
     void removeListener(Player.Listener listener);

libraries/session/src/main/java/androidx/media3/session/MediaControllerImplBase.java

@@ -215,6 +215,11 @@ import org.checkerframework.checker.nullness.qual.NonNull;
     }
   }
 
+  @Override
+  public Bundle getConnectionHints() {
+    return connectionHints;
+  }
+
   @Override
   public void addListener(Listener listener) {
     listeners.add(listener);

libraries/session/src/main/java/androidx/media3/session/MediaControllerImplLegacy.java

@@ -102,6 +102,7 @@ import org.checkerframework.checker.initialization.qual.UnderInitialization;
   private final ControllerCompatCallback controllerCompatCallback;
   private final BitmapLoader bitmapLoader;
   private final ImmutableList<CommandButton> commandButtonsForMediaItems;
+  private final Bundle connectionHints;
 
   @Nullable private MediaControllerCompat controllerCompat;
   @Nullable private MediaBrowserCompat browserCompat;
@@ -117,6 +118,7 @@ import org.checkerframework.checker.initialization.qual.UnderInitialization;
       Context context,
       @UnderInitialization MediaController instance,
       SessionToken token,
+      Bundle connectionHints,
       Looper applicationLooper,
       BitmapLoader bitmapLoader) {
     // Initialize default values.
@@ -134,6 +136,7 @@ import org.checkerframework.checker.initialization.qual.UnderInitialization;
     this.instance = instance;
     controllerCompatCallback = new ControllerCompatCallback(applicationLooper);
     this.token = token;
+    this.connectionHints = connectionHints;
     this.bitmapLoader = bitmapLoader;
     currentPositionMs = C.TIME_UNSET;
     lastSetPlayWhenReadyCalledTimeMs = C.TIME_UNSET;
@@ -154,6 +157,11 @@ import org.checkerframework.checker.initialization.qual.UnderInitialization;
     }
   }
 
+  @Override
+  public Bundle getConnectionHints() {
+    return connectionHints;
+  }
+
   @Override
   public void addListener(Listener listener) {
     listeners.add(listener);
@@ -1410,7 +1418,10 @@ import org.checkerframework.checker.initialization.qual.UnderInitialization;
               // Create it on the application looper to respect that.
               browserCompat =
                   new MediaBrowserCompat(
-                      context, token.getComponentName(), new ConnectionCallback(), null);
+                      context,
+                      token.getComponentName(),
+                      new ConnectionCallback(),
+                      instance.getConnectionHints());
               browserCompat.connect();
             });
   }

libraries/test_data/src/test/assets/muxerdumps/ctts_box.dump

@@ -1,2 +1,2 @@
 ctts (88 bytes):
-  Data = length 80, hash FC850C18
+  Data = length 80, hash D9A9F376

libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaBrowserListenerWithMediaBrowserServiceCompatTest.java

@@ -19,6 +19,7 @@ import static androidx.media3.session.LibraryResult.RESULT_SUCCESS;
 import static androidx.media3.session.MediaConstants.EXTRAS_KEY_COMPLETION_STATUS;
 import static androidx.media3.session.MediaConstants.EXTRAS_VALUE_COMPLETION_STATUS_PARTIALLY_PLAYED;
 import static androidx.media3.session.MediaConstants.EXTRA_KEY_ROOT_CHILDREN_BROWSABLE_ONLY;
+import static androidx.media3.session.MockMediaBrowserServiceCompat.EXTRAS_KEY_SEND_ROOT_HINTS_AS_SESSION_EXTRAS;
 import static androidx.media3.test.session.common.CommonConstants.MOCK_MEDIA_BROWSER_SERVICE_COMPAT;
 import static androidx.media3.test.session.common.MediaBrowserConstants.PARENT_ID;
 import static androidx.media3.test.session.common.MediaBrowserConstants.ROOT_EXTRAS_KEY;
@@ -136,6 +137,33 @@ public class MediaBrowserListenerWithMediaBrowserServiceCompatTest {
     assertThat(thrown).hasCauseThat().isInstanceOf(SecurityException.class);
   }
 
+  @Test
+  public void connect_useConnectionHints_connectionHintsPassedToLegacyServerOnGetRootAsRootHints()
+      throws Exception {
+    Bundle connectionHints = new Bundle();
+    connectionHints.putBoolean(EXTRAS_KEY_SEND_ROOT_HINTS_AS_SESSION_EXTRAS, true);
+    CountDownLatch latch = new CountDownLatch(/* count= */ 1);
+    AtomicReference<Bundle> extrasRef = new AtomicReference<>();
+    createBrowser(
+        connectionHints,
+        /* maxCommandsForMediaItems= */ 0,
+        /* listener= */ new MediaBrowser.Listener() {
+          @Override
+          public void onExtrasChanged(MediaController controller, Bundle extras) {
+            extrasRef.set(extras);
+            latch.countDown();
+          }
+        });
+
+    assertThat(latch.await(TIMEOUT_MS, MILLISECONDS)).isTrue();
+    assertThat(
+            extrasRef
+                .get()
+                .getBoolean(
+                    EXTRAS_KEY_SEND_ROOT_HINTS_AS_SESSION_EXTRAS, /* defaultValue= */ false))
+        .isTrue();
+  }
+
   @Test
   public void getLibraryRoot_browseActionsAvailable() throws Exception {
     remoteService.setProxyForTest(TEST_MEDIA_ITEMS_WITH_BROWSE_ACTIONS);

libraries/test_session_current/src/main/java/androidx/media3/session/MockMediaBrowserServiceCompat.java

@@ -40,7 +40,9 @@ import static java.lang.Math.min;
 import android.content.Intent;
 import android.net.Uri;
 import android.os.Bundle;
+import android.os.Handler;
 import android.os.IBinder;
+import android.os.Looper;
 import android.os.RemoteException;
 import android.support.v4.media.MediaBrowserCompat;
 import android.support.v4.media.MediaBrowserCompat.MediaItem;
@@ -68,6 +70,13 @@ public class MockMediaBrowserServiceCompat extends MediaBrowserServiceCompat {
    */
   public static final ImmutableList<MediaItem> MEDIA_ITEMS = createMediaItems();
 
+  /**
+   * Key in the browser root hints to request a confirmation of the call to {@link
+   * #onGetRoot(String, int, Bundle)}.
+   */
+  public static final String EXTRAS_KEY_SEND_ROOT_HINTS_AS_SESSION_EXTRAS =
+      "confirm_on_get_root_with_custom_action";
+
   private static final String TAG = "MockMBSCompat";
   private static final Object lock = new Object();
 
@@ -166,12 +175,18 @@ public class MockMediaBrowserServiceCompat extends MediaBrowserServiceCompat {
       // Test only -- reject any other request.
       return null;
     }
+    if (rootHints.getBoolean(EXTRAS_KEY_SEND_ROOT_HINTS_AS_SESSION_EXTRAS, false)) {
+      // Send delayed because the Media3 browser is in the process of connecting at this point and
+      // won't receive listener callbacks before being connected.
+      new Handler(Looper.myLooper())
+          .postDelayed(() -> sessionCompat.setExtras(rootHints), /* delayMillis= */ 100L);
+    }
     synchronized (lock) {
       if (isProxyOverridesMethod("onGetRoot")) {
         return serviceProxy.onGetRoot(clientPackageName, clientUid, rootHints);
       }
     }
-    return new BrowserRoot("stub", null);
+    return new BrowserRoot("stub", /* extras= */ rootHints);
   }
 
   @Override