History

hschlueter a105d033a7 Check targetSdkVersion for frame dropping workaround.

Based on
https://developer.android.com/reference/android/media/MediaCodec#using-an-output-surface,
frame dropping behaviour depends on the target SDK version.
After this change transformer will only use
MediaFormat#KEY_ALLOW_FRAME_DROP if both the target and system SDK
version are at least 29 and default to its pre 29 behaviour where each
decoder output frame must be processed before a new one is rendered
to prevent frame dropping otherwise.

Also remove deprecated Transformer.Builder constructor without a
context and the context setter.

PiperOrigin-RevId: 453971097

2022-06-09 18:48:45 +00:00

src

Check targetSdkVersion for frame dropping workaround.

2022-06-09 18:48:45 +00:00

BUILD.bazel

Add frame processor based on MediaPipe to demo

2022-05-09 10:56:08 +01:00

build.gradle

Create withMediaPipe variant only if AAR is present

2022-05-31 13:55:58 +00:00

README.md

Create withMediaPipe variant only if AAR is present

2022-05-31 13:55:58 +00:00

README.md

Transformer demo

This app demonstrates how to use the Transformer API to modify videos, for example by removing audio or video.

See the demos README for instructions on how to build and run this demo.

MediaPipe frame processing demo

Building the demo app with MediaPipe integration enabled requires some extra manual steps.

Follow the instructions to install MediaPipe.

Copy the Transformer demo's build configuration and MediaPipe graph text protocol buffer under the MediaPipe source tree. This makes it easy to build an AAR with bazel by reusing MediaPipe's workspace.

cd "<path to MediaPipe checkout>"
MEDIAPIPE_ROOT="$(pwd)"
MEDIAPIPE_TRANSFORMER_ROOT="${MEDIAPIPE_ROOT}/mediapipe/java/com/google/mediapipe/transformer"
cd "<path to the transformer demo (containing this readme)>"
TRANSFORMER_DEMO_ROOT="$(pwd)"
mkdir -p "${MEDIAPIPE_TRANSFORMER_ROOT}"
mkdir -p "${TRANSFORMER_DEMO_ROOT}/libs"
cp ${TRANSFORMER_DEMO_ROOT}/BUILD.bazel ${MEDIAPIPE_TRANSFORMER_ROOT}/BUILD
cp ${TRANSFORMER_DEMO_ROOT}/src/withMediaPipe/assets/edge_detector_mediapipe_graph.pbtxt \
  ${MEDIAPIPE_TRANSFORMER_ROOT}

Build the AAR and the binary proto for the demo's MediaPipe graph, then copy them to Transformer.

cd ${MEDIAPIPE_ROOT}
bazel build -c opt --strip=ALWAYS \
  --host_crosstool_top=@bazel_tools//tools/cpp:toolchain \
  --fat_apk_cpu=arm64-v8a,armeabi-v7a \
  --legacy_whole_archive=0 \
  --features=-legacy_whole_archive \
  --copt=-fvisibility=hidden \
  --copt=-ffunction-sections \
  --copt=-fdata-sections \
  --copt=-fstack-protector \
  --copt=-Oz \
  --copt=-fomit-frame-pointer \
  --copt=-DABSL_MIN_LOG_LEVEL=2 \
  --linkopt=-Wl,--gc-sections,--strip-all \
  mediapipe/java/com/google/mediapipe/transformer:edge_detector_mediapipe_aar.aar
cp bazel-bin/mediapipe/java/com/google/mediapipe/transformer/edge_detector_mediapipe_aar.aar \
  ${TRANSFORMER_DEMO_ROOT}/libs
bazel build mediapipe/java/com/google/mediapipe/transformer:edge_detector_binary_graph
cp bazel-bin/mediapipe/java/com/google/mediapipe/transformer/edge_detector_mediapipe_graph.binarypb \
  ${TRANSFORMER_DEMO_ROOT}/src/withMediaPipe/assets

In Android Studio, gradle sync and select the withMediaPipe build variant (this will only appear if the AAR is present), then build and run the demo app and select a MediaPipe-based effect.