533 Commits

Author SHA1 Message Date
tofunmi
896d147444 Overlay: fix the overlay anchor translation
Makes OverlayFrameAnchor works as described in the OverlaySettings documentation. Currently the code does the opposite e.g setting the anchor to (+1,-1) makes the code anchor to the top left rather than the bottom right.

PiperOrigin-RevId: 621585558
2024-04-03 11:27:43 -07:00
huangdarwin
60bb24a930 Overlay: Add javadoc for why we fly overlays vertically.
Otherwise, apps overriding BitmapOverlay.getVertexTransformation may not
realize this is being done, and may ask why bitmaps are flipped vertically from
what they expect

Reference: https://github.com/androidx/media/issues/1128
PiperOrigin-RevId: 621544348
2024-04-03 09:21:23 -07:00
huangdarwin
24e1796c50 Effect: Improve setEnableColorTransfers javadoc.
PiperOrigin-RevId: 619250163
2024-03-26 11:15:34 -07:00
huangdarwin
33a87a451c Test: Remove EnsuresNonNull from media3 editing tests.
nullness checks were removed a while ago from editing, so no need for them now.

Bit neater to have less code to parse too.

Also, removes RequiresNonNull

PiperOrigin-RevId: 616887502
2024-03-18 11:13:10 -07:00
huangdarwin
e446f20a70 Test: Remove nullness for testid
This also makes it more consistent with newer testId tests that don't have nullness, so tests don't look inconsistent, and was pretty easy to do :P

PiperOrigin-RevId: 616856888
2024-03-18 09:47:12 -07:00
claincly
e8612a0c02 Add warning log if no output is set
PiperOrigin-RevId: 616796787
2024-03-18 05:33:17 -07:00
huangdarwin
cbc654a721 GL: Use GLES20.GL_FALSE and GL_TRUE instead of hardcoded value.
PiperOrigin-RevId: 616759833
2024-03-18 02:35:55 -07:00
tofunmi
4d1ede5432 Ultra HDR: release texture after use
PiperOrigin-RevId: 616209205
2024-03-15 12:23:45 -07:00
tofunmi
762065fae0 Add api to find duration after an effect is applied
PiperOrigin-RevId: 616157067
2024-03-15 09:41:40 -07:00
huangdarwin
b126bae93d Effect: Add EGL_EXT_gl_colorspace_bt2020_hlg support, on API 34+
Confirmed that the HLG extension displays properly on Pixel 7 Pro, API 34 and Samsung S24, API 34. On these devices, this fixes the washed out HLG preview issue on API 34, where the PQ ext had a washed out look, and where the HLG ext is supported (this bug didn't occur on API 33, only 34).

More info on manual tests done to sanity-check:
* Test cases: Transformer debug SurfaceView and ExoPlayer.setVideoEffects
* Test inputs: HLG and PQ
* Test devices: Pixel 7 Pro (API 33 & 34), Samsung Galaxy S24 (API 34)
* Added debugging: Logging colorInfo used in GlUtil.createEglSurface

No regressions were seen. HLG extension is used more in API 34, and behavior stays the same on API 33. Only human-visible change without logging is that HLG content looks better on API 34, for Samsung S24 and Pixel 7 Pro.

PiperOrigin-RevId: 616131192
2024-03-15 08:06:49 -07:00
tofunmi
c0ffc94d15 Add GlEffect that asynchronously adjusts input presentation times
Single asset export only effect.

PiperOrigin-RevId: 616114153
2024-03-15 06:56:25 -07:00
tofunmi
f39fe82bba Support ultra HDR in effect
adding sending gainmap to shader program and applying gainmap to base in shader

PiperOrigin-RevId: 616070582
2024-03-15 03:22:54 -07:00
claincly
626a8adfd8 Test effect-enabled playback
This new test is for `ExoPlayer.setVideoEffects()`. It plays the
one-second-long video, applies an overlay that prints the video frame timestamp
onto the frame, captures the output frame and compares the captured output
frame with golden.

PiperOrigin-RevId: 610781590
2024-02-27 09:44:24 -08:00
tofunmi
0480bc31a8 Rename media/bitmap out of media and rename
PiperOrigin-RevId: 610418418
2024-02-26 08:40:19 -08:00
tofunmi
5a892509f7 remove media/bitmap/input_images
move the images into the respective places in the file extension directory so they file the pre-existing structure

PiperOrigin-RevId: 609744673
2024-02-23 09:07:03 -08:00
huangdarwin
601d6ed587 Update GL thread name, per package name
PiperOrigin-RevId: 609686878
2024-02-23 04:29:31 -08:00
huangdarwin
0d1b35477d Test: Use testId throughout media3 tests.
No more instances of this codesearch query after this CL in non-parameterized
tests:
`String\ testId\ \=\ \" f:media3`
PiperOrigin-RevId: 609364413
2024-02-22 07:28:55 -08:00
Copybara-Service
fb7438378d Merge pull request #983 from MGaetan89:min_api_19
PiperOrigin-RevId: 607319987
2024-02-15 07:00:17 -08:00
claincly
ae0da442b1 Fix forward for 216f3fedb8
The original change missed copying the newly added flag in `buildUpon()`, which
caused some test flakes.

PiperOrigin-RevId: 607232373
2024-02-15 00:05:10 -08:00
Gaëtan Muller
973b717914 Remove unnecessary SDK_INT checks 2024-02-13 10:26:19 +00:00
claincly
216f3fedb8 Loosen the requirement to register every input frame
VideoFrameProcessor requires every input frame to be registered before they are
rendered to its input Surface. This CL adds the option to register the input
frame only once. This is useful for camera inputs where not all input frames
come with changing FrameInfo.

PiperOrigin-RevId: 606294894
2024-02-12 10:38:42 -08:00
tofunmi
1a5eb4eecd Support segment-specific speed changes in SpeedChangeEffect
PiperOrigin-RevId: 606204479
2024-02-12 04:03:23 -08:00
ibaker
338aef4830 Fix transformer, effect, muxer and container API dependencies
The public APIs of these modules reference symbols in some of their
dependencies, so these should be API dependencies, not implementation:
> An API dependency is one that contains at least one type that is
> exposed in the library binary interface, often referred to as its ABI
> (Application Binary Interface).

https://docs.gradle.org/current/userguide/java_library_plugin.html#sec:java_library_recognizing_dependencies

Transformer also uses symbols from `lib-common`, but these are already
an API dep of `lib-exoplayer` so no need to duplicate that here.

PiperOrigin-RevId: 605660621
2024-02-09 10:05:54 -08:00
huangdarwin
73c9753229 HDR: Update constant name for clarity.
This matrix converts from BT2020 in RGB, not to XYZ in BT2020,
so put "bt2020" nearer to RGB.

PiperOrigin-RevId: 605654773
2024-02-09 09:45:16 -08:00
huangdarwin
a5e47982f4 HDR: Add color-space conversion for PQ tone-map.
Previously, we missed the BT2020->BT709 color-space conversion.

A user-visible impact of this is that red and green channels used to be
undersaturated, but now are more correctly saturated.

PiperOrigin-RevId: 605411926
2024-02-08 13:33:52 -08:00
huangdarwin
e4145a9953 Update javadoc to be the right DrawableOverlay class.
PiperOrigin-RevId: 604337168
2024-02-05 09:13:40 -08:00
tofunmi
c79f950d26 ultra HDR: send gainmap downstream if HDR output requested
removing the useHdr parameter from queueInputBitmap() it suggests we support changing between HDR and SDR within a stream, which we don't support. instead, identifying whether to use HDR from the shaderprogram which is informed by the inputColorInfo when the stream is registered.

PiperOrigin-RevId: 603681736
2024-02-02 07:33:15 -08:00
tofunmi
271eb88b48 Remove texture copy from SpeedChangeShaderProgram
Introduces PassthroughShaderProgram to make the effect more efficient.

PiperOrigin-RevId: 603670438
2024-02-02 06:30:18 -08:00
huangdarwin
ae85ba9ee9 Effect: Remove extra checkStateNotNull by reordering logic.
PiperOrigin-RevId: 603660773
2024-02-02 05:34:11 -08:00
huangdarwin
a27511cc50 Effect: Remove inputColorInfo from create() methods.
Transformer export and ExoPlayer previewing both read inputColorInfo from
registerInputStream now, instead of maintaining a consistent input color
throughout multiple streams in a sequence.

Therefore, we can remove inputColor-related arguments and methods now.

PiperOrigin-RevId: 603423509
2024-02-01 11:09:57 -08:00
huangdarwin
dd7846ee1c Effect: Use element instead of peek, when throwing if null.
If nullness should result in an exception, we should throw as soon as possible,
so that stack traces are easier to follow.

Did a quick scan of media3.effect+transformer and this covers all uses of
peek that immediately throw there.

PiperOrigin-RevId: 603393366
2024-02-01 09:32:21 -08:00
huangdarwin
cebe6d8ba5 HDR: Don't assume that swapping between BT709 and BT601 needs tone-map.
VideoFrameProcessor treats BT601 and BT709 as roughly equivalent now, so we
shouldn't be making checks that assume BT709 <-> requires tone-mapping.

Also, the color transfer is a better determinant for tone-mapping than color range, so use just the transfer to determine if tone-mapping is required.

PiperOrigin-RevId: 603083100
2024-01-31 10:07:45 -08:00
claincly
dcae49a561 Fix blank video when switching on/off screen
This happens when using `ExoPlayer.setVideoEffects()`.

This CL also fixes the first frame not rendered problem, originally solved in
7e65cce967, but rolled back in 5056dfaa2b because the solution introduces
the flash that is observed in b/292111083.

Before media3 1.1 release, the output size of `VideoFrameProcessor` is not
reported to the app. This was changed later after introducing
`CompositingVideoSinkProvider`, where the video size after processing **is**
reported to the app. After this CL, the size is again, not reported.

PiperOrigin-RevId: 602345087
2024-01-29 04:24:19 -08:00
tonihei
ed5b7004b4 Replace or suppress deprecated usages
Many usages are needed to support other deprecations and some
can be replaced by the recommended direct alternative.

Also replace links to deprecated/redirected dev site

PiperOrigin-RevId: 601795998
2024-01-26 10:06:18 -08:00
tofunmi
7424cffce1 Support changing input colorinfo for images
since we now can support taking in the inputColor upon registering the stream, there is no need to hardcode the image input color anymore. We should now be able to support switching between texture and image input which we couldn't before, but this is untested and not necessary.

PiperOrigin-RevId: 601784149
2024-01-26 09:17:29 -08:00
timpeut
9c1aaa7c43 Suppress nullability warnings
PiperOrigin-RevId: 600891113
2024-01-23 13:34:14 -08:00
tofunmi
acc78125d2 Update METADATA to add clang-format requirement for glsl files
also ran clang-format for good measure

PiperOrigin-RevId: 600759938
2024-01-23 05:49:38 -08:00
tofunmi
cc62f0556c BaseGlShaderProgram: Remove redundant NoSuchElementException catching
PiperOrigin-RevId: 600738198
2024-01-23 03:52:49 -08:00
huangdarwin
616cb943f0 Previewing: Allow inputColorInfo to change, from SDR to HDR.
This also fixes issue introduced by frames being released from a prior version of a
GlShaderProgram

Tested by seeking within a playlist with one SDR then one HDR video.

PiperOrigin-RevId: 599475959
2024-01-18 05:20:18 -08:00
christosts
0e66197419 Rename TimestampIterator argument name in queueInputBitmap()
#minor-release

PiperOrigin-RevId: 599460771
2024-01-18 04:01:27 -08:00
huangdarwin
1ec193883c HDR: Propagate PQ to SDR OOTF to internal input.
Previously, 8f69bb0d9d updated external input (video input)
but not internal input (image/texture input). Update internal input as
well to match.

PiperOrigin-RevId: 599235813
2024-01-17 11:27:38 -08:00
tofunmi
0155ae998b Gaussian Blur: support blurring without drawing sharp image on top
PiperOrigin-RevId: 598626481
2024-01-15 09:39:38 -08:00
christosts
5056dfaa2b Regression fix of ExoPlayer.setVideoEffects()
7e65cce967 introduced a regression on ExoPlayer.setVideoEffects()
where there is flash on the screen after the first few frames are shown.

Before 7e65cce967, the first frames of the content are missed, until
MediaCodecVideoRenderer sends the onVideoSizeChanged() callback. The
first frame is processed but not shown, the onVideoSizeChanged() is
triggered and the renderer receives a video output resolution message as
a response from the UI.

7e65cce967 fixed the missed first frames by setting a surface on the
CompositingVideoSinkProvider before the provider is initialized, and as
as result:
- the first frames are rendered
- the MediaCodecVideoRenderer sends the the onVideoSizeChanged() after
  frames are shown
- the UI sends a video output resolution change
- the MediaCodecVideoRenderer updates the CompositingVideoSinkProvider
  which causes the flash.

The underlying problem is with onVideoSizeChanged() not being
triggered early enough, before the first frame is shown.

Because the flashing is a regression, this commit is reverting
7e65cce967 but keeps the test that was added as ignored, so that the
test is re-enabled as soon as we address the underlying issue.

PiperOrigin-RevId: 597814013
2024-01-12 05:20:46 -08:00
tofunmi
78c419e566 Extension to Gaussian Blur: support changing blur over time
PiperOrigin-RevId: 597809380
2024-01-12 04:53:02 -08:00
huangdarwin
c59711f592 Effect: Remove unused arguments in DefaultShaderProgram factories.
Partially addresses the following TODO, by simplifying the DefaultShaderProgram
API surface.
```
// TODO(b/274109008): Refactor DefaultShaderProgram to create a class just for sampling.
```

PiperOrigin-RevId: 597575575
2024-01-11 09:25:25 -08:00
huangdarwin
b4dfbed73d Update javadoc param name to match argument name
PiperOrigin-RevId: 597554111
2024-01-11 07:50:13 -08:00
ibaker
1202f8690c Bump effect and leanback minSdk to 19
These were overriding the previous minimum of 16 and therefore not
included in e54abaa75d.

PiperOrigin-RevId: 597242364
2024-01-10 07:22:00 -08:00
tofunmi
51d60e1f3a GaussianBlur: make short constructor call the longer one
PiperOrigin-RevId: 597212434
2024-01-10 04:42:35 -08:00
tofunmi
dee2d3af73 GlassianBlurTest: move golden images comment next to the test
PiperOrigin-RevId: 596892569
2024-01-09 04:41:46 -08:00
huangdarwin
c6b51003d3 Effect: Allow updating inputColorInfo between streams in VFP.
Whenever the inputColorInfo updates, update the samplingGlShaderProgram.

Also, allow either SDR or gamma2.2 to be used for HDR->SDR tone-mapping
`outputColorInfo` request. This is required because we can't update the
`outputColorInfo`, but plan to always use gamma2.2 for `outputColorInfo` in the
future.

This allows VideoFrameProcessor to work as is for exoplayer previewing, but
only when not seeking. As we haven't plumbed the per-stream inputColorInfo from
ExoPlayer down to VFP.registerInputStream, follow-up CLs will be needed to
properly support previewing with changing inputColorInfo.

PiperOrigin-RevId: 596627890
2024-01-08 10:09:04 -08:00