34 Commits

Author SHA1 Message Date
huangdarwin
e773031927 Effect: Move BitmapTestUtil to common test util class.
Move BitmapTestUtil from media3.effect to media3.test.utils.

This allows this class to be accessed from both transformer and effect tests.

Refactoring change. No functional change intended.

PiperOrigin-RevId: 501837130
2023-01-17 02:54:54 +00:00
huangdarwin
1a1cdcc350 HDR: Update glsl color transfer errors to use unique colors.
Each error will make the output be a solid color with a unique color.

PiperOrigin-RevId: 501824355
2023-01-17 02:50:57 +00:00
huangdarwin
4ade37c48d HDR: Implement PQ to SDR tone-mapping.
Tested manually on the Pixel 7 and Samsung S10.

PiperOrigin-RevId: 501626354
2023-01-17 02:46:59 +00:00
tofunmi
a786c03a34 Move BitmapLoader to common.
The migration strategy is to deprecate `androidx.media3.session.BitmapLoader` and copy the file into common since BitmapLoader is a public interface that apps could be relying on.

PiperOrigin-RevId: 501266521
2023-01-17 02:03:35 +00:00
andrewlewis
9835e91439 Fix some minor issues/typos
PiperOrigin-RevId: 499808853
2023-01-10 20:08:14 +00:00
kimvde
8a8c81d57f Move video decoding to AssetLoader
PiperOrigin-RevId: 499454273
2023-01-04 19:43:10 +00:00
andrewlewis
bec46b66eb Reduce flakiness of GL release test
Based on experimentation it seems that buffers can occasionally (roughly 1% of test runs) be dropped when rendering off-screen from EGL on the emulator. Specifically, in this test, sometimes after rendering three buffers with distinct timestamps only the first and third buffers' timestamps are handled in the `ImageReader`'s image available callback causing the assertion checking all frames rendered to fail. This behavior seems to be independent of the nanosecond presentation time attached to the buffers (as expected for off-screen rendering).

Introducing a pause of 1 second between rendering each frame reduces the flake rate to around 1/2000. This increases the run time of some of the tests, so this change also removes the 5 second `FRAME_PROCESSING_WAIT_MS` (it seems to be unnecessary when rendering off-screen) and instead uses a latch to wait until the frame processor has handled 'end of stream'.

PiperOrigin-RevId: 499440591
2023-01-04 19:39:12 +00:00
tofunmi
967e44c066 Add TextureOverlay.configure() to configure overlay before frame processing
This change allows an overlay to be sized with respect to the video dimensions.

PiperOrigin-RevId: 497337734
2023-01-04 19:15:18 +00:00
tofunmi
4d1283f43e Add support for customizing overlay anchor point.
PiperOrigin-RevId: 496956035
2023-01-04 18:33:09 +00:00
claincly
8ab8dadf43 Simplify concurrency in FrameProcessingTaskExecutor
This CL replaces concurrent collections and atomic primitives with a single
lock, this way the code is easier to reason about.

PiperOrigin-RevId: 496718057
2022-12-21 17:38:25 +00:00
tofunmi
ce766d7622 Create dynamically created shaders for multiple overlays.
PiperOrigin-RevId: 496379904
2022-12-21 16:55:44 +00:00
claincly
a4bc0959be Support previewing HDR on API33+
FrameProcessor already support using different transfer function for input and
output color. This CL has two major changes:

- Create an eglSurface that recognizes BT.2020 PQ
  - This requires a separate extension that works only after 33
  - So we current throw, if input is HDR, and this extension doesn't work
- Create FrameProcessor with PQ output transfer function

PiperOrigin-RevId: 496023758
2022-12-21 16:28:37 +00:00
claincly
4a3d693c3e Replace Pair with Size in effects.
PiperOrigin-RevId: 495646341
2022-12-21 15:57:19 +00:00
huangdarwin
b836212b39 HDR: Implement GlEffectsFrameProcessor HLG to SDR tone-mapping.
Adds COLOR_TRANSFER_GAMMA_2_2, to match behavior in other HDR to SDR tone mapping
implementations.

PiperOrigin-RevId: 495371736
2022-12-15 16:59:52 +00:00
tofunmi
7a95e08b09 Fix bugs in DrawableOverlay.java.
calls setbounds() on the drawable as the bounds of a drawable must be set for the .draw() method to work as noted in the [Drawable Documentation](https://developer.android.com/reference/android/graphics/drawable/Drawable#draw(android.graphics.Canvas)).

Also fixes createStaticDrawableOverlay() not taking specified overlay settings into account.

PiperOrigin-RevId: 494116077
2022-12-12 14:40:38 +00:00
tofunmi
05c84a6795 Add support for changing translucency of overlays through alpha values
Implements milestone 1.3 of the [overlays implementation plan](https://docs.google.com/document/d/1EcP2GN8k8N74hHZyD0KTqm9oQo5-W1dZMqIVyqVGtlo/edit#bookmark=id.76uzcie1dg9d)

PiperOrigin-RevId: 493290147
2022-12-12 13:19:58 +00:00
tofunmi
369f9bae43 Implement text and drawable overlays
Implements milestone 1.2 of the [overlays implementation plan](https://docs.google.com/document/d/1EcP2GN8k8N74hHZyD0KTqm9oQo5-W1dZMqIVyqVGtlo/edit#bookmark=id.76uzcie1dg9d)

PiperOrigin-RevId: 493282868
2022-12-12 13:15:47 +00:00
ibaker
c006575d43 Add javadoc links to README files
Fix some other link titles and destinations spotted along the way.

#minor-release

PiperOrigin-RevId: 493276172
2022-12-12 13:07:19 +00:00
huangdarwin
f6e346ede5 HDR: Update to use input/output color instead of OETF/EOTF.
These terms are easier to understand, and make sense in the context of
MatrixTextureProcessor now that a MatrixTextureProcessor may have a
different input and output transfer.

PiperOrigin-RevId: 493265980
2022-12-12 13:03:07 +00:00
huangdarwin
cf61e219ef HDR: Add effect interface support for different in/out color transfers
Modify FrameProcessor and MatrixTextureProcessor interfaces to support
different input and output color transfers. Does not implement conversion between
color ranges (ex. HDR and SDR), but should allow for conversion between color
transfers of the same color range (ex. HLG and PQ).

This supports in-app tone mapping, where we need a single FrameProcessor to
input HDR color transfers (ex. HLG/PQ) and output SDR (ex. gamma2.2). This also
supports previewing, where we need a single FrameProcessor to be able to input HLG
and output PQ.

Manually tested by confirming colors still look right on SDR and HDR videos
with a rotation and color affect applied.

PiperOrigin-RevId: 493108678
2022-12-12 12:54:38 +00:00
claincly
88a7d67f4a Change argument order to match API council suggestion
By putting Listener as the last argument.

PiperOrigin-RevId: 493100906
2022-12-12 12:50:21 +00:00
claincly
656b346e6c Allow specifying an Executor in FrameProcessor.Factory.create()
From this CL on, FrameProcessor listeners will be invoked from an Executor that
is passed in when creating the FrameProcessor.

GlTextureProcessor needs to invoke the ErrorListener on the said Executor too.

PiperOrigin-RevId: 493018583
2022-12-12 12:41:47 +00:00
tofunmi
3964176421 Add the processor, GLEffect, texture and bitmap overlays
Implements milestone 1.1 of the [overlays implementation plan](https://docs.google.com/document/d/1EcP2GN8k8N74hHZyD0KTqm9oQo5-W1dZMqIVyqVGtlo/edit#bookmark=id.76uzcie1dg9d)

PiperOrigin-RevId: 491696361
2022-12-12 10:45:31 +00:00
huangdarwin
39f01bfb93 HDR: Apply OETF in HDR external sampler GLSL shader.
This should fix how color matrix transforms look when applied on HDR colors

PiperOrigin-RevId: 491323228
2022-11-29 19:15:52 +00:00
huangdarwin
07d1970f35 HDR: Add COLOR_TRANSFER_LINEAR in C.java.
This is more clear than using Format.NO_VALUE, when we do actually intend for an
output value.

Also, fix @see formatting by using summary fragments instead, and add an error
output for OETF and EOTF transfer functions.

PiperOrigin-RevId: 490910229
2022-11-29 18:52:13 +00:00
claincly
47d63504c2 Destroy eglSurface in FrameProcessor
Previously, FrameProcessor never had the usecase in which the output surface
is replaced, while previewing introduced this usecase.

When switching output surfaces, we need to destroy the EGL Surface linked to the
surface that is being swapped out, because an EGL surface is linked to the EGL
display (which is not destroyed even when releasing FrameProcessor).

A GL exception will be thrown in the following scenario if we don't destroy the
EGL surface:

1. Creates a Surface, the surface is identified by address 0x11
2. Sets Surface(0x11) on FrameProcessor. Eventually an EGL surface is created
  to wrap Surface(0x11)
3. Release FrameProcess, this releases the EGL context
4. Instantiate a new FrameProcessor, sets Surface(0x11) as the output
5. When FrameProcessor creates an EGL surface to wrap Surface(0x11), GL throws
  an exception, becasue Surface(0x11) has previouly been connected to an EGL
  surface.

PiperOrigin-RevId: 489590072
2022-11-21 15:58:13 +00:00
claincly
f2a63e9ddf Fix availableFrameCount semantic
`availableFrameCount` tracks the number of frames that is avilable on the
`SurfaceTexture`, but haven't been used (by `updateTexImage()`) yet. Thus
semantically this counter should only be decremented after calling
`updateTexImage()`, not before it.

Also reworded `getPendingFrameCount()` javadoc, "external texture" is an
internal state that is not publicised anywhere.

PiperOrigin-RevId: 488765174
2022-11-16 12:43:49 +00:00
samrobinson
52f3ee81d4 Handle buffers in DefaultAudioSink with AudioProcessingPipeline.
PiperOrigin-RevId: 488412695
2022-11-16 12:10:21 +00:00
huangdarwin
095c52e7ba GL: Rename getEglSurface to createEglSurface.
Per documentation, eglCreateWindowSurface creates a new EGL window surface.

getEglSurface suggests instead that a pre-existing surface is returned.

https://registry.khronos.org/EGL/sdk/docs/man/html/eglCreateWindowSurface.xhtml

PiperOrigin-RevId: 488377423
2022-11-14 18:08:57 +00:00
ibaker
3fdfe585ff Fix Dackka/Metalava errors in the effects module
Public methods may only refer to public types in their signature. This
change ensures that by switching to a public supertype everywhere.

PiperOrigin-RevId: 485568625
2022-11-08 14:02:00 +00:00
claincly
f5ad4e098d Add GL utility methods to get 4x4 identity and set identity
PiperOrigin-RevId: 483671580
2022-10-31 16:00:16 +00:00
huangdarwin
9e1adee4cb HDR: Support RGBA_1010102 in GlUtil as a parameter.
Make it easier to support use of RGBA_101012 rather than RGBA_8888 for EGL
contexts, displays, and surfaces.

This tangentially supports adding HDR tests, by slightly simplifying the color
selection logic we'd have to add in HDR tests.

PiperOrigin-RevId: 482219428
2022-10-24 10:35:28 +00:00
huangdarwin
c698ec51a2 Effect: Remove unnecessary "this" qualifier
`transformationMatrix` is not ambiguous, as there's no other local
transformationMatrix variable nearby.

PiperOrigin-RevId: 482184602
2022-10-20 03:19:31 +00:00
bachinger
28c4be2f6b Map lib-effect from androidx to exoplayer2
PiperOrigin-RevId: 482179761
2022-10-20 03:15:38 +00:00