235 Commits

Author SHA1 Message Date
huangdarwin
abf649cdfa Effects: Have VFP texture output disable surface output.
Also, document that texture output disables manual frame release.

In the past, texture output would lead to surface output methods throwing. Now,
they're simply no-ops instead.

PiperOrigin-RevId: 534894168
2023-05-25 09:57:16 +01:00
andrewlewis
b988cce62c Clarify color transfers for internal textures
PiperOrigin-RevId: 534869452
2023-05-25 09:56:03 +01:00
andrewlewis
749c64c74f Tidy TextureManager javadoc
Fix `@param`s and clarify wording.

PiperOrigin-RevId: 534857204
2023-05-25 09:54:02 +01:00
claincly
d584a772e3 Add logging for ExtTexMgr
- Number of frames from SurfaceTexture that is sent downstream
- Times ExtTexMgr signaled end of current input stream

PiperOrigin-RevId: 534487842
2023-05-24 16:03:50 +01:00
huangdarwin
a6897aedaa Effect: Add GlTextureInfo release() and accessor methods.
This allows us to disallow access after release.

PiperOrigin-RevId: 534046475
2023-05-24 15:42:47 +01:00
andrewlewis
ea0b94c0eb Remove unnecessary volatile
This field is always accessed with the mutex held.

PiperOrigin-RevId: 534027096
2023-05-24 15:35:40 +01:00
andrewlewis
2cbc2c6176 Run clang-format on GLSL
PiperOrigin-RevId: 534015933
2023-05-24 15:33:25 +01:00
ibaker
d5f9cf4f19 Add singleVariant publishing config to all published gradle files
Docs:
* https://developer.android.com/build/publish-library/configure-pub-variants#single-pub-var
* https://developer.android.com/reference/tools/gradle-api/8.0/com/android/build/api/dsl/PublishingOptions

This resolves the following warning from Android Gradle Plugin 7.2.2 and
helps unblock the upgrade to AGP 8.0.1:

> Software Components will not be created automatically for Maven
> publishing from Android Gradle Plugin 8.0. To opt-in to the future
> behavior, set the Gradle property
> `android.disableAutomaticComponentCreation=true` in the
> `gradle.properties` file or use the new publishing DSL.
>
> Affected Modules: `lib-cast`, `lib-common`, `lib-container`,
> `lib-database`, `lib-datasource`, `lib-datasource-cronet`,
> `lib-datasource-okhttp`, `lib-datasource-rtmp`, `lib-decoder`,
> `lib-effect`, `lib-exoplayer`, `lib-exoplayer-all (legacy)`,
> `lib-exoplayer-dash`, `lib-exoplayer-hls`, `lib-exoplayer-ima`,
> `lib-exoplayer-rtsp`, `lib-exoplayer-smoothstreaming`,
> `lib-exoplayer-workmanager`, `lib-extractor`, `lib-media2 (legacy)`,
> `lib-mediasession (legacy)`, `lib-muxer`, `lib-session`,
> `lib-transformer`, `lib-ui`, `lib-ui-leanback`, `test-utils`,
> `test-utils-robolectric`

Issue: androidx/media#409
PiperOrigin-RevId: 533464133
2023-05-19 17:22:21 +01:00
ibaker
b3db85133a Add android.namespace to all build.gradle files
This is a pre-requisite for the Android Studio upgrade assistant to
upgrade from AGP 7.2.2 to 8.0.1, otherwise it fails and complains
this is missing.

Issue: androidx/media#409
PiperOrigin-RevId: 533463246
2023-05-19 17:19:15 +01:00
ibaker
0f8ddd5d1a Suffix package in test AndroidManifest.xml files with .test
This is blocking the upgrade to AGP 8.0.1. The Android Studio Upgrade
Assistant refuses to run with the error:
> Modules have the same package for their `main` and `androidTest`
> artifacts
>
> The package specifications in `AndroidManifest.xml` files define the
> same package for the `main` and `androidTest` artifacts, in the
> following modules: `effect`, `muxer`, `transformer`,
`test_exoplayer_abr`, `test_exoplayer_performance`, and `container`.

This change also adds the `.test` suffix to `test/AndroidManifest.xml`
files where it's missing, because although it's not mentioned in the
Upgrade Assistant error it does cause problems later.

```
> Incorrect package="androidx.media3.database" found in source AndroidManifest.xml: libraries/database/src/test/AndroidManifest.xml.
  Setting the namespace via the package attribute in the source AndroidManifest.xml is no longer supported.
  Recommendation: remove package="androidx.media3.database" from the source AndroidManifest.xml: libraries/database/src/test/AndroidManifest.xml.
```

Issue: androidx/media#409
PiperOrigin-RevId: 533460180
2023-05-19 17:18:22 +01:00
claincly
d7ad431cfc Add a getter method for texture manager
PiperOrigin-RevId: 533402277
2023-05-19 17:14:12 +01:00
huangdarwin
25fa2df2de Effect: Use callback to release texture
This allows us to avoid needing a reference to the VideoFrameProcessor, which
can be especially difficult if an App only has a reference to the
VideoFrameProcessor.Factory it passes into Transformer/ExoPlayer.

PiperOrigin-RevId: 533205983
2023-05-19 17:10:53 +01:00
claincly
24a4396990 Always use sRGB/BT.709 for bitmap inputs
PiperOrigin-RevId: 533117700
2023-05-19 17:05:16 +01:00
tofunmi
5fe10d7652 HDR texture asset loading
PiperOrigin-RevId: 532846248
2023-05-18 12:21:38 +00:00
huangdarwin
94efcd7917 Effect: Make TexturePool and use in FinalWrapper.
Have the FinalShaderProgramWrapper / VideoFrameProcessor texture
output access textures provided through a texture pool, that
recycles used textures.

Also, add the TexturePool interface to generally re-use textures.

PiperOrigin-RevId: 532754377
2023-05-17 16:33:57 +01:00
kimvde
028b3a7312 Simplify FrameConsumptionManager onReadyToAcceptInputFrame logic
Propagate the "end of current stream" signal directly after queueing the
last frame, instead of waiting for the next onReadyToAcceptInputFrame()
call.

PiperOrigin-RevId: 532739462
2023-05-17 16:30:29 +01:00
tofunmi
63ee5ccb28 Adjust image input ForPixelWidthHeightRatio
PiperOrigin-RevId: 532463400
2023-05-17 08:45:10 +01:00
huangdarwin
07ec1eaa48 Effect: Multiple Texture output
Allow the VideoFrameProcessor to output multiple textures at a time, so that
lifetime of textures is up to the consumer calling VFP.releaseOutputFrame.

The FinalShaderProgramWrapper also has a new maxCapacity limit added, to ensure
the a reasonable amount of textures is used and avoid using up memory.

PiperOrigin-RevId: 532094256
2023-05-16 09:49:52 +01:00
andrewlewis
6850391e45 Fix lost shader input capacity after end-of-stream
When exporting compositions with multiple images in a row, transformation could
get stuck if a shader was ready to accept input when end-of-stream was already
signaled and queued from upstream. Fix accounting for the downstream capacity.

Manually tested on concatenations with several images and several videos in a
row, by adding logging and verifying the capacity updates as expected across
edited media item transitions.

PiperOrigin-RevId: 532088793
2023-05-16 09:47:45 +01:00
andrewlewis
857e6ebee8 Signal end-of-stream after creating latch
The output end-of-stream notification from the last shader could theoretically
arrive before the latch for detecting it is created, which might cause waiting
on the latch indefinitely. Create the latch before signaling end of stream so
that it's guaranteed to be set before the end-of-stream signal arrives.

PiperOrigin-RevId: 532056472
2023-05-16 09:42:25 +01:00
huangdarwin
2736b11845 Transformer: Add latest input format to DebugTraceUtil
This can provide more information to help debug muxer errors

PiperOrigin-RevId: 531521974
2023-05-15 10:39:57 +01:00
tofunmi
565a013d8d Refactor ChainingGlShaderProgramListener to use FrameConsumptionManager
PiperOrigin-RevId: 531180020
2023-05-12 10:05:48 +01:00
tofunmi
00d5031dcf Disable texture input when input is HDR
Verified by running all the tests in mh (excluding the analysis tests) on a pixel 7 pro

PiperOrigin-RevId: 530655237
2023-05-11 09:54:55 +00:00
tofunmi
7875131e2a Effect: Support input via texture ID
PiperOrigin-RevId: 530624195
2023-05-11 09:49:26 +00:00
tofunmi
477994461d Image transcoding: Add support for heic/heif image formats
PiperOrigin-RevId: 530578549
2023-05-11 09:42:12 +00:00
tofunmi
9baa6f6be5 Fix operations order in RenderingTest
For DefaultVideoFrameProcessorVideoFrameRenderingTest: `registerInputStream` sets the textureManager, so reordering makes sure the textureManager is set before you set the frame info. This is important for texture input, where the frame info provides the width and height.

PiperOrigin-RevId: 529753404
2023-05-08 15:20:14 +00:00
huangdarwin
38d3fb5382 HDR: Don't register unsupported HDR bitmap input.
Minimal fix for HDR test postsubmit breakage

PiperOrigin-RevId: 529734521
2023-05-05 16:58:34 +00:00
claincly
c2da310718 Remove unused variables in DVFP.
PiperOrigin-RevId: 529685584
2023-05-05 16:48:28 +00:00
claincly
83b5f824f8 Add a tracing utility for debugging video related muxer timeout
PiperOrigin-RevId: 529632886
2023-05-05 16:45:38 +00:00
claincly
213740c456 Enable switching input types via InputMultiplexer.
PiperOrigin-RevId: 529624205
2023-05-05 16:43:58 +00:00
huangdarwin
93e3fe418e Effect: Fix concurrent access null pointer exception
A list was being accessed from one thread when it wasn't guaranteed to be empty.

PiperOrigin-RevId: 529102141
2023-05-03 17:16:06 +01:00
claincly
ad2d4f5008 Mark @CallSuper on some BaseGlShaderProgram methods.
PiperOrigin-RevId: 528734663
2023-05-03 17:09:11 +01:00
ibaker
dab0260ae0 Relax some copybara transform regexes
This means that comments like `//copybara:media3-only` are now detected
(no space between `//` and `copybara`) which will ensure that lines like
this are correctly transformed for the media3 and exoplayer2 GitHub
repos:
aa4e008014/library/effect/build.gradle (L33)
PiperOrigin-RevId: 527919649
2023-05-03 17:05:25 +01:00
claincly
349eca7ae2 Add an input switcher to switch between input types.
Also make FinalShaderProgramWrapper always receive internal texture.

This means it does not sample from a input texture, and its input color is
always linear, hence the input type does not matter.

PiperOrigin-RevId: 527869045
2023-05-03 17:02:43 +01:00
tofunmi
4b75397f4e Rollback of 5c02210305
*** Original commit ***

Effect: glFlush instead of glFinish on tex output

This is much faster (~2-3x) than glFlush. While there's a risk that GL commands
queued to the GL server may not be complete by the time non-GL commands access
the texture, this should be unlikely as we only access the texture from GL.

If we see stability issues in the future, we can reconsider and move this back
to glFinish (or GL synchronization mechanisms like fences, which are more
complex)

***

PiperOrigin-RevId: 527848094
2023-05-03 17:01:54 +01:00
huangdarwin
24343f55af effect: Rename VFP frame release to render.
renderOutputFrame actually renders frames to an output surface. We'll soon have
a releaseOutputFrame method, that would release resources associated with an
output time, so rename this to disambiguate the two methods.

Also rename onOutputFrameAvailable to onOutputFrameAvailableForRendering, to
make it clear this is not available for "release"

This change should be a renaming-only change and have no functional differences.

PiperOrigin-RevId: 527844947
2023-05-03 17:00:48 +01:00
tofunmi
3a3322dbc5 Remove volatile from boolean in BitmapTextureManager
The field is only accessed on the GL thread.

PiperOrigin-RevId: 527844674
2023-05-03 16:59:49 +01:00
huangdarwin
5c02210305 Effect: glFlush instead of glFinish on tex output
This is much faster (~2-3x) than glFlush. While there's a risk that GL commands
queued to the GL server may not be complete by the time non-GL commands access
the texture, this should be unlikely as we only access the texture from GL.

PiperOrigin-RevId: 527641520
2023-05-03 16:57:00 +01:00
tofunmi
0902fad55d Effect: split ending frame Processing and getting the output bitmap in tests
PiperOrigin-RevId: 527554408
2023-05-03 16:54:53 +01:00
claincly
12cac0d69f Add a listener once one MediaItem is fully processed
Add `VideoFrameProcessor.registerInputStream()` to signal a new type of input.

And `InputHandler.signalEndOfCurrentInputStream()` to signal to `InputHandler`
partial input stream completion.

Fully processed means after FinalShaderProgramWrapper releases the last frame.

PiperOrigin-RevId: 527356646
2023-04-27 12:28:40 +01:00
huangdarwin
fdeeaba9d8 Effect: Call glFinish before providing VFP output texture
glFinish should be called before reading from a texture, to make sure it's been
properly rendered to.

PiperOrigin-RevId: 527302946
2023-04-27 12:27:40 +01:00
tofunmi
178a323897 Update sdr internal fsh to accept input color transfer
PiperOrigin-RevId: 527271212
2023-04-27 12:22:24 +01:00
huangdarwin
97272c139c Effects: Output to texture without surface in VFP.
Allow the VideoFrameProcessor to output to a texture without an output surface.

Tested by updating texture output tests to no longer output to a surface.

PiperOrigin-RevId: 527244605
2023-04-26 15:51:06 +01:00
tonihei
889f435a49 Update dependencies to latest versions
The only dependencies that are not updated are the ones that need
to be kept in sync with other system (like Android source tree) or
would require a Kotlin dependency in common or exoplayer modules.

As a side effect, some demo apps now need a Kotlin config and some
additional modules require desugaring/multidex logic. To simplify
the setup, the desugaring and multidex steps are added to the common
config.

PiperOrigin-RevId: 527243950
2023-04-26 15:49:57 +01:00
claincly
19b979d817 Allow setting individual offset for bitmaps.
PiperOrigin-RevId: 527001582
2023-04-26 15:47:31 +01:00
tofunmi
8612d2820d Update image transcoding pipeline to signal input COLOR_TRANSFER_SRGB
The production code changes are in transformer, but the tests in effect have also been updated to confirm the is no color regression `inputColorInfo.colorTransfer=C.COLOR_TRANSFER_SRGB`

PiperOrigin-RevId: 526950435
2023-04-26 15:46:21 +01:00
claincly
c539cb8575 Add default impl for some InputHandler methods.
And minor fixes.

PiperOrigin-RevId: 526717927
2023-04-26 15:43:46 +01:00
huangdarwin
324115f6cf Effects: Allow BT2020 colors with SDR transfers
Previously, I assumed that SDR contents must be BT709, and HDR contents must be
BT2020. Turns out BT2020 is just wide-gamut, and SDR contents / transfers may be
represented in BT2020 color spaces.

Relax the check, so that we don't throw when valid BT2020 SMPTE 170M contents
are input into effects.

PiperOrigin-RevId: 526668347
2023-04-26 15:41:33 +01:00
tofunmi
f1d9a0a819 Add nullness-compat-qual dependency for android tests
PiperOrigin-RevId: 526652129
2023-04-26 15:38:22 +01:00
claincly
f3c0256bb4 Abstract the interface of DefaultVideoFrameProcessor's input.
PiperOrigin-RevId: 526642898
2023-04-26 15:37:12 +01:00