514 Commits

Author SHA1 Message Date
tonihei
2322462404 Do not trim audio samples by changing their timestamp
MP4 edit lists sometimes ask to start playback between two samples.
If this happens, we currently change the timestamp of the first
sample to zero to trim it (e.g. to display the first frame for a
slightly shorter period of time). However, we can't do this to audio
samples are they have an inherent duration and trimming them this
way is not possible.

#minor-release

PiperOrigin-RevId: 543420218
2023-06-29 22:50:04 +00:00
andrewlewis
90c8f642af Pass GlObjectsProvider to methods
By passing this class where it's needed, implementations don't need to store it
in a field (reducing boilerplate) and it's clearer that it can't be unset when
needed.

PiperOrigin-RevId: 542823522
2023-06-23 16:49:12 +00:00
claincly
1831220a53 Fix the last shader program doesn't get GlObjectsProvider
PiperOrigin-RevId: 542299317
2023-06-22 15:40:33 +00:00
tofunmi
09fe0d7390 Correct the DefaultFrameDroppingShaderProgram logic
copies the previous texture to a temp texture to ensure the correct frame is queued.

#minor-release

PiperOrigin-RevId: 541972349
2023-06-22 15:29:41 +00:00
huangdarwin
949e9cbd96 GL: Rename GL methods.
To more accurately describe what they do, especially as Compositor will
starts to use more contexts or threads, and it's important to know what
needs to be reset/recreated/focused before what methods.

PiperOrigin-RevId: 541010135
2023-06-19 16:29:03 +01:00
huangdarwin
daa42322d7 GL: Remove configAttributes from createFocusedPlaceholderEglSurface.
PiperOrigin-RevId: 540901189
2023-06-19 16:27:02 +01:00
claincly
fe33f0e390 Simplify EOS signaling in DefaultVideoFrameProcessor
Previously, TextureMangers have a method to signal ending of a current input
stream, and a method to end the **entire input**. The responsibility of both
methods are not easy to document, understand and read.

With the new design,

- Only `TextureManager.signalEndOfCurrentInputStream()` is kept
  - It's called for every MediaItem in the sequence, include the final one
- FinalWrapper now takes explicit signal that frame processing is ending,
  rather than relying on the return value of `onCurrentInputStreamProcessed()`
- On DVFP receiving EOS from the pipeline, it signals FinalWrapper the stream
  is ending, **before** signaling the input switcher, so that FinalWrapper is
  able to end the stream when the onCurrentInputStreamEnded signal eventually
  reaches FinalWrapper

PiperOrigin-RevId: 540856680
2023-06-19 16:24:33 +01:00
huangdarwin
4178d61ccd Effect: Move VideoFrameProcessingTask to VFPTaskexecutor
Group things that are closely related. VFPTask is only ever
used alongside VFPTaskExecutor.

PiperOrigin-RevId: 540850948
2023-06-19 16:22:09 +01:00
tofunmi
251fb013c3 Reset the simple frame dropping logic on flush or EOS.
#minor-release

PiperOrigin-RevId: 540633875
2023-06-19 16:20:09 +01:00
huangdarwin
5fc5105f7a Test: Log saved bitmap file path.
This helps debug bitmap tests. Without this CL, I typically
need to go into the file to find the test name and absolute
path. With this CL, I can just adb pull the printed file path,
which is much simpler.

PiperOrigin-RevId: 540539042
2023-06-19 16:10:30 +01:00
tonihei
51fb72b00d Replace deprecated NullableType from checkerframework with our own one
The existing NullableType has been deprecated 5 years ago and causes
crashes in Kotlin apps because Kotlin doesn't recognize this annotation
as a nullable type annotation.

While we can't align on a single @Nullable annotation yet, we can at
least replace this one by JSR305's @Nonnull(MAYBE) as it fulfils all
requirements, including full Kotlin compatiblity. To avoid the
cumbersome name, we can redefine it as our own @NullableType
annotation. (We can't use @Nullable to avoid name clashes with the main
@Nullable annotation from AndroidX)

Issue: google/ExoPlayer#6792
PiperOrigin-RevId: 540497469
2023-06-19 16:08:20 +01:00
claincly
aa94f4e128 Add a blank line to break long paragraph
PiperOrigin-RevId: 540222936
2023-06-14 20:38:24 +01:00
claincly
abd96598d9 Add a timer to end a video stream prematurely in ExtTexMgr
Partially roll forward of 5c29abbbf4, and adds some extra logic

Changes to the original CL

The original logic (a66f08ba97) fails in the following case:

> This is only seem on emulators.

- EOS is sent to ExtTexMgr
- The timer starts
- One frame arrives on SurfaceTexture, reset the timer
  - The frame is sent for processing, now `availablFrames == 0`
- One frame arrives on Surface, reset the timer
  - The frame is kept on SurfaceTexture for the downstream shader doesn't have
    capacity, `availablFrames == 1`
- Timer times out as the downstream processor doesn't report being able to take
  another frame.
  - Although there's a frame available on the SurfaceTexture

This is solved by having the force EOS logic clear all the frames that the
SurfaceTexture holds. This also ensures the first frame dequeued from the next
stream isn't from the previous stream.

PiperOrigin-RevId: 540023359
2023-06-14 20:34:17 +01:00
tofunmi
219212d54f rename videoFrameProcessorTestRunner.queueInputBitmap
This methods currently registers and queues the bitmaps Will need to have a method that just queues bitmaps in future CLs.

#minor-release

PiperOrigin-RevId: 539605628
2023-06-12 14:06:22 +00:00
claincly
5f43180a68 Allow using different video effects per MediaItem
PiperOrigin-RevId: 539596345
2023-06-12 14:04:42 +00:00
kimvde
1c18503ad0 Transmux video with negative rotation if possible
PiperOrigin-RevId: 539593035
2023-06-12 14:03:06 +00:00
Googler
5c29abbbf4 Rollback of a66f08ba97
*** Original commit ***

Add a timer to end a video stream prematurely in ExtTexMgr

***

This has been submitting for more than 1.5hrs. "This presubmit is running slowly because you have been throttled by Build Queue due to using too much of your Product Area's quota."

adding NO_SQ as this is a pure rollback

PiperOrigin-RevId: 539135970
2023-06-09 18:36:17 +00:00
claincly
edf30433b6 Add missing empty lines
PiperOrigin-RevId: 539100987
2023-06-10 20:22:38 +00:00
claincly
a66f08ba97 Add a timer to end a video stream prematurely in ExtTexMgr
PiperOrigin-RevId: 539036285
2023-06-09 14:04:49 +00:00
tofunmi
276f2f1fe6 Add 'Keep every nth frame' frame dropping strategy.
#minor-release

PiperOrigin-RevId: 538804347
2023-06-09 13:55:39 +00:00
claincly
a7cff4e0e3 Fix calling extra registerInputStream and not handling EOS
PiperOrigin-RevId: 538796466
2023-06-09 13:52:44 +00:00
huangdarwin
0c924fcb40 Test: Add no-op effect test for GL tone mapping.
To ensure no regressions for the potentially confusing pipeline of:
* HDR electrical -> SDR linear EOTF+OOTF, and
* SDR linear -> SDR electrical OETF

PiperOrigin-RevId: 538741079
2023-06-08 12:46:34 +00:00
tofunmi
c3b9328d74 Tests for disabled color transfers
#minor-release

PiperOrigin-RevId: 538466615
2023-06-08 12:38:36 +00:00
claincly
133943a635 Fix recreation of DefaultShaderProgram in FinalWrapper
`outputSurfaceInfoChanged` is not reset when `defaultShaderProgram` is null.
That is, on the first time `ensureConfigured()` is called with output size
changed, `outputSurfaceInfoChanged` is not set to false after creating the
`defaultShaderProgram`, and `defaultShaderProgram` will be created again on the
second time `ensureConfigured()` is called.

PiperOrigin-RevId: 537870404
2023-06-06 18:01:20 +00:00
ibaker
7e6dae7b31 Remove exoplayer2-only javadoc machinery from media3
This introduces whitespace-only changes in exoplayer2 due to the way
copybara stripping works.

PiperOrigin-RevId: 537296933
2023-06-05 09:47:58 +00:00
claincly
f6dbe99c79 Use the designated UNSET value for aspect ratio
PiperOrigin-RevId: 536770380
2023-06-02 09:12:38 +00:00
huangdarwin
1c172e0bed Effect: Remove extra wait on taskExecutor release()
This future.get() duplicates the wait done in
singleThreadExecutorService.awaitTermination(). If awaitTermination times out, this future.get() would also result in unnecessary blocking.

PiperOrigin-RevId: 536442153
2023-05-30 18:46:43 +00:00
kimvde
f4d1a6c453 Transmux video if rotation is only effect applied
PiperOrigin-RevId: 535554628
2023-05-26 15:14:52 +00:00
tofunmi
a19e07c4d2 Move OnInputFrameProcessedListener into it's own file
PiperOrigin-RevId: 535183521
2023-05-25 15:09:36 +00:00
huangdarwin
abf649cdfa Effects: Have VFP texture output disable surface output.
Also, document that texture output disables manual frame release.

In the past, texture output would lead to surface output methods throwing. Now,
they're simply no-ops instead.

PiperOrigin-RevId: 534894168
2023-05-25 09:57:16 +01:00
andrewlewis
b988cce62c Clarify color transfers for internal textures
PiperOrigin-RevId: 534869452
2023-05-25 09:56:03 +01:00
andrewlewis
749c64c74f Tidy TextureManager javadoc
Fix `@param`s and clarify wording.

PiperOrigin-RevId: 534857204
2023-05-25 09:54:02 +01:00
claincly
d584a772e3 Add logging for ExtTexMgr
- Number of frames from SurfaceTexture that is sent downstream
- Times ExtTexMgr signaled end of current input stream

PiperOrigin-RevId: 534487842
2023-05-24 16:03:50 +01:00
huangdarwin
a6897aedaa Effect: Add GlTextureInfo release() and accessor methods.
This allows us to disallow access after release.

PiperOrigin-RevId: 534046475
2023-05-24 15:42:47 +01:00
andrewlewis
ea0b94c0eb Remove unnecessary volatile
This field is always accessed with the mutex held.

PiperOrigin-RevId: 534027096
2023-05-24 15:35:40 +01:00
andrewlewis
2cbc2c6176 Run clang-format on GLSL
PiperOrigin-RevId: 534015933
2023-05-24 15:33:25 +01:00
ibaker
d5f9cf4f19 Add singleVariant publishing config to all published gradle files
Docs:
* https://developer.android.com/build/publish-library/configure-pub-variants#single-pub-var
* https://developer.android.com/reference/tools/gradle-api/8.0/com/android/build/api/dsl/PublishingOptions

This resolves the following warning from Android Gradle Plugin 7.2.2 and
helps unblock the upgrade to AGP 8.0.1:

> Software Components will not be created automatically for Maven
> publishing from Android Gradle Plugin 8.0. To opt-in to the future
> behavior, set the Gradle property
> `android.disableAutomaticComponentCreation=true` in the
> `gradle.properties` file or use the new publishing DSL.
>
> Affected Modules: `lib-cast`, `lib-common`, `lib-container`,
> `lib-database`, `lib-datasource`, `lib-datasource-cronet`,
> `lib-datasource-okhttp`, `lib-datasource-rtmp`, `lib-decoder`,
> `lib-effect`, `lib-exoplayer`, `lib-exoplayer-all (legacy)`,
> `lib-exoplayer-dash`, `lib-exoplayer-hls`, `lib-exoplayer-ima`,
> `lib-exoplayer-rtsp`, `lib-exoplayer-smoothstreaming`,
> `lib-exoplayer-workmanager`, `lib-extractor`, `lib-media2 (legacy)`,
> `lib-mediasession (legacy)`, `lib-muxer`, `lib-session`,
> `lib-transformer`, `lib-ui`, `lib-ui-leanback`, `test-utils`,
> `test-utils-robolectric`

Issue: androidx/media#409
PiperOrigin-RevId: 533464133
2023-05-19 17:22:21 +01:00
ibaker
b3db85133a Add android.namespace to all build.gradle files
This is a pre-requisite for the Android Studio upgrade assistant to
upgrade from AGP 7.2.2 to 8.0.1, otherwise it fails and complains
this is missing.

Issue: androidx/media#409
PiperOrigin-RevId: 533463246
2023-05-19 17:19:15 +01:00
ibaker
0f8ddd5d1a Suffix package in test AndroidManifest.xml files with .test
This is blocking the upgrade to AGP 8.0.1. The Android Studio Upgrade
Assistant refuses to run with the error:
> Modules have the same package for their `main` and `androidTest`
> artifacts
>
> The package specifications in `AndroidManifest.xml` files define the
> same package for the `main` and `androidTest` artifacts, in the
> following modules: `effect`, `muxer`, `transformer`,
`test_exoplayer_abr`, `test_exoplayer_performance`, and `container`.

This change also adds the `.test` suffix to `test/AndroidManifest.xml`
files where it's missing, because although it's not mentioned in the
Upgrade Assistant error it does cause problems later.

```
> Incorrect package="androidx.media3.database" found in source AndroidManifest.xml: libraries/database/src/test/AndroidManifest.xml.
  Setting the namespace via the package attribute in the source AndroidManifest.xml is no longer supported.
  Recommendation: remove package="androidx.media3.database" from the source AndroidManifest.xml: libraries/database/src/test/AndroidManifest.xml.
```

Issue: androidx/media#409
PiperOrigin-RevId: 533460180
2023-05-19 17:18:22 +01:00
claincly
d7ad431cfc Add a getter method for texture manager
PiperOrigin-RevId: 533402277
2023-05-19 17:14:12 +01:00
huangdarwin
25fa2df2de Effect: Use callback to release texture
This allows us to avoid needing a reference to the VideoFrameProcessor, which
can be especially difficult if an App only has a reference to the
VideoFrameProcessor.Factory it passes into Transformer/ExoPlayer.

PiperOrigin-RevId: 533205983
2023-05-19 17:10:53 +01:00
claincly
24a4396990 Always use sRGB/BT.709 for bitmap inputs
PiperOrigin-RevId: 533117700
2023-05-19 17:05:16 +01:00
tofunmi
5fe10d7652 HDR texture asset loading
PiperOrigin-RevId: 532846248
2023-05-18 12:21:38 +00:00
huangdarwin
94efcd7917 Effect: Make TexturePool and use in FinalWrapper.
Have the FinalShaderProgramWrapper / VideoFrameProcessor texture
output access textures provided through a texture pool, that
recycles used textures.

Also, add the TexturePool interface to generally re-use textures.

PiperOrigin-RevId: 532754377
2023-05-17 16:33:57 +01:00
kimvde
028b3a7312 Simplify FrameConsumptionManager onReadyToAcceptInputFrame logic
Propagate the "end of current stream" signal directly after queueing the
last frame, instead of waiting for the next onReadyToAcceptInputFrame()
call.

PiperOrigin-RevId: 532739462
2023-05-17 16:30:29 +01:00
tofunmi
63ee5ccb28 Adjust image input ForPixelWidthHeightRatio
PiperOrigin-RevId: 532463400
2023-05-17 08:45:10 +01:00
huangdarwin
07ec1eaa48 Effect: Multiple Texture output
Allow the VideoFrameProcessor to output multiple textures at a time, so that
lifetime of textures is up to the consumer calling VFP.releaseOutputFrame.

The FinalShaderProgramWrapper also has a new maxCapacity limit added, to ensure
the a reasonable amount of textures is used and avoid using up memory.

PiperOrigin-RevId: 532094256
2023-05-16 09:49:52 +01:00
andrewlewis
6850391e45 Fix lost shader input capacity after end-of-stream
When exporting compositions with multiple images in a row, transformation could
get stuck if a shader was ready to accept input when end-of-stream was already
signaled and queued from upstream. Fix accounting for the downstream capacity.

Manually tested on concatenations with several images and several videos in a
row, by adding logging and verifying the capacity updates as expected across
edited media item transitions.

PiperOrigin-RevId: 532088793
2023-05-16 09:47:45 +01:00
andrewlewis
857e6ebee8 Signal end-of-stream after creating latch
The output end-of-stream notification from the last shader could theoretically
arrive before the latch for detecting it is created, which might cause waiting
on the latch indefinitely. Create the latch before signaling end of stream so
that it's guaranteed to be set before the end-of-stream signal arrives.

PiperOrigin-RevId: 532056472
2023-05-16 09:42:25 +01:00
huangdarwin
2736b11845 Transformer: Add latest input format to DebugTraceUtil
This can provide more information to help debug muxer errors

PiperOrigin-RevId: 531521974
2023-05-15 10:39:57 +01:00