Adds a method to FrameProcessor.Listener to be called when an
output frame is available and a method releaseOutputFrame in
FrameProcessor allowing the caller to trigger release of the
oldest available output frame at a given timestamp. Late frames
or frames with unset release times are dropped in the
FinalMatrixTransformationProcessorWrapper.
More than one output frame can become available before they are
released if the penultimate GlTextureProcessor is capable of producing
multiple output frames. Processing continues while waiting for
releaseOutputFrame to be called. Frame release tasks are prioritized
over other tasks.
PiperOrigin-RevId: 468473072
(cherry picked from commit a5d7fdcab52ea6c12fa35efc348b69eccb53f715)
createSupportedTransformationRequest is more accurate than
createFallbackTransformationRequest, as a TransformationRequest will be returned
regardless of whether any fallback is applied.
PiperOrigin-RevId: 466641277
(cherry picked from commit c5a5364673749cb737be3df92c9521fc15207658)
Previously, this feature interpreted SDR signals as HDR when called. Now, only HDR
streams are interpreted as HDR, so the javadoc should be updated. Not yet removing
this method, as there are still some loose ends to finish up (ex. PQ support, e2e
tests).
PiperOrigin-RevId: 466425738
(cherry picked from commit 05fcda510a1dc8ab0a44487f3ddfff7fc2b617e7)
While HDR is most closely tied to the color transfer (ex.
COLOR_TRANSFER_SDR is the only one explicitly mentioning dynamic
range), technically color spaces may be associated with HDR as well,
like BT.2020 commonly being used for HDR rather than BT.709 for SDR.
Therefore, it's more specific to mention just that the transfer is HDR.
PiperOrigin-RevId: 466316960
(cherry picked from commit 35161c748956d96e8e83cee007143615ef6f9d07)
This allows the GlEffectsFrameProcessor to later handle HLG and PQ
differently, or limited and full color range differently.
No functional change intended in this CL.
PiperOrigin-RevId: 466070764
(cherry picked from commit 536d42c865fd692c1d1799082077906287a2c3f0)
In the case where this check fails, the downstream frame processor chain won't be able to handle the incoming (SDR) data anyway as we've already set it up for HDR.
PiperOrigin-RevId: 465584814
(cherry picked from commit 2bd1a93d27fbc1615c55edb200917f5d5a1891b4)
`requestCalculateSsim` more clearly represents the intention of the caller.
Also rephrase the javadoc to simplify it and make it more precise.
PiperOrigin-RevId: 465575578
(cherry picked from commit 2eb59fa1af823ff66fcf384f567a92e55c35b683)
* Transform frame colors using a defined RGBA Matrix to apply filters.
PiperOrigin-RevId: 464523581
(cherry picked from commit d15f2ed388e59e1b378440470347ee6f7f00ed94)
Format.NO_VALUE is a placeholder value for an invalid @C.ColorTransfer, used
for example when the decoder doesn't support this transfer function.
When encountering this invalid value, interpret this as COLOR_TRANSFER_SDR.
Confirmed locally that an exception is thrown when transcoding on p4head, and no exception is thrown when transcoding with this CL.
PiperOrigin-RevId: 464135080
(cherry picked from commit 687a50e9b313e5db37a0b32f5f40559252ab6031)
This will allow effects preview in ExoPlayer to use the
Effect and FrameProcessor interface (and the interfaces
they depend on) without depending on transformer or the
future effects module.
PiperOrigin-RevId: 464060047
(cherry picked from commit 480c129c54777780f298d7e5dbd9fa23a371f660)
* Sets KEY_HDR_STATIC_INFO from MediaFormat in the DefaultCodec.
* Adds checks in mediaparser to ensure color space, range, and transfer are valid
values.
PiperOrigin-RevId: 463921325
(cherry picked from commit 65a2a3a0a073c6dd8b47a044609f4be1abdd19c9)
TransformationException error codes were previously tied to OpenGL
but other FrameProcessor implementations are possible. So this CL
renames the error codes.
Also, remove GL_INIT_FAILED error code, as FrameProcessor
implemenations may initialize resources on a background thread
after the factory method returns, so it's not obvious how to
distinguish between initialization failures and processing failures.
PiperOrigin-RevId: 463704902
(cherry picked from commit 0f96d8615d28cf50f3efa0c34fb31e01fa0d491c)
This allows non-GL effects to be passed to custom FrameProcessor
implementations.
PiperOrigin-RevId: 463696384
(cherry picked from commit 580e44fc47ec5b62da5a133d12048a20d716e94f)
Both are used in the public FrameProcessor interface, so they
should be public too.
PiperOrigin-RevId: 463454859
(cherry picked from commit b1b9ba825f83030c5d5b0617796176f21526dc2e)
This is possible because SSIM increases monotonically with bitrate.
PiperOrigin-RevId: 463434373
(cherry picked from commit 4a0b07b4f7e577d02b6d36f49da2c9f5d0b2e485)
Just a misc nit, since I found the name a bit confusing, and figured
findEncoderWithClosestSupportedFormat might be more descriptive.
PiperOrigin-RevId: 463433646
(cherry picked from commit a7a17dc2bb8bd76988ae32867ff1b8109fcccdb5)
Extract a FrameProcessor.Factory interface from GlEffectsFrameProcessor
and allow it to be customized using a setter on Transformer.Builder.
PiperOrigin-RevId: 463433438
(cherry picked from commit 22822d8e194cedb89bd01fac89a131f145c5042b)
Size requires API 21. Using Pair instead will allow effects to be
used from API 18 during previewing once they are moved out of
transformer.
PiperOrigin-RevId: 463206474
(cherry picked from commit 4adf19939839baa3eaf018fef0d1b58049fd3b91)
This is needed for applying effects to a playlist.
The effects are applied based on the presentation time of the
frame in its corresponding media item and the offset is added
back before encoding.
Each time the offset changes, end of input stream is signalled
to the texture processors. This is needed because the texture
processors can expect monotonically increasing timestamp within
the same input stream but when the offset changes, the timstamps
jump back to 0.
PiperOrigin-RevId: 462714966
(cherry picked from commit 30e5bc9837e2423cd2bb426c5797211e0f6ad76b)
It's always safe to ignore the result of these methods, because the
caller already has a reference to the returned value.
PiperOrigin-RevId: 462388947
(cherry picked from commit 2deb435625c7569cb0bd250848adaa846884dc50)
This allows us to use BT.2020 RGB linear for intermediate shaders, which also
allows us to re-enable PeriodicVignetteProcessor, which should work properly in
linear color-spaces.
Manually tested by adding a GlEffectsWrapper, and confirming that HLG HDR editing still looks correct.
PiperOrigin-RevId: 462265821
(cherry picked from commit 2f977eeec93be5cc71fda57f80362c1bc639c041)
Otherwise, invalid ColorInfo instances generated using faulty
MediaFormat#getInteger values could cause exceptions.
Confirmed that b/239435670 reproduces without this CL, and does not reproduce
with this CL.
PiperOrigin-RevId: 461862191
(cherry picked from commit 87198fe7f8df62a4b9087bc7f2c3cda237f789b2)
* Introduced `useHdr` for `GlEffect#toGlTextureProcessor`, so
`TextureProcessor` implementations can decide how to handle HDR.
* Creating FP16 color textures for HDR input.
Tested via manual testing, adding a no-op GlEffectWrapper to the transformation to
force use of intermediate textures, adding a linear ramp to the fragment shader,
and trying to ascertain that there's a real reduction in posterization when
switching from 4-bit to 8-bit unsigned bytes, and again from 8-bit unsigned bytes
to 16-bit floating point.
PiperOrigin-RevId: 461613117
(cherry picked from commit ba9c9bb96471a9589d391ae498ebb0c194a5b41e)
This may happen when a containers' color transfer incorrectly does not match
the video's color transfer.
An example of a file with such a mismatch is the current Transformer demo HDR10
sample file.
Manually tested by confirming that no errors are emitted for SDR and HLG sample
files, and that errors are emitted for our incorrect HDR10 sample file.
PiperOrigin-RevId: 461583532
(cherry picked from commit 9f7a159bc4c8a82653fdbb6d62e4f79b020cff06)
- Update profile selection logic to pick an HDR-compatible profile when doing HDR editing on H.264/AVC videos.
- Handle doing the capabilities check for all MIME types that support HDR (not just H.265/HEVC).
- Fix a bug where we would pass an HDR input color format to the encoder when using tone-mapping.
- Tweak how `EncoderWrapper` works so decisions at made at construction time.
Manually tested cases:
- Transformation of an SDR video.
- Transformation of an HDR video to AVC (which triggers fallback/tone-mapping on a device that doesn't support HDR editing for AVC).
- Transformation of an HDR video with HDR editing.
PiperOrigin-RevId: 461572973
(cherry picked from commit 604ab7fcdaa759025536feb673a3abb93196a829)
These are providing more variety and complexity.
All files are okay to be public.
PiperOrigin-RevId: 460935247
(cherry picked from commit b87fa45fea56a50b5c035c7ca18eaf19697e60bf)
Also remove VideoEncoderSettings.colorProfile as there are no
concrete use cases for customizing this and it clashes with picking
the color format automatically based on SDR vs. HDR.
PiperOrigin-RevId: 460746987
(cherry picked from commit 7954eeb3c2fa274d9343cbf51963a3cccf3270c7)
Add test that verifies SSIM with API enabled.
#minor-release
PiperOrigin-RevId: 460692420
(cherry picked from commit f903869eb880508812c0dc2e1a8b01ecf0597f45)
Pass the color info and HDR static metadata when configuring the decoder
using MediaFormatUtil.maybeSetColorInfo.
PiperOrigin-RevId: 460424985
(cherry picked from commit 66e12299881b4edbfa60d1fe717a9b6016fac2a5)
This extension is needed for editing HDR input with OpenGL, as the
ExternalTextureProcessor samples raw YUV values from the
external texture for HDR and converts them to RGB itself rather than
relying on the OpenGL driver to do this automatically as for SDR.
PiperOrigin-RevId: 460424154
(cherry picked from commit a5ff4ef17f78b64735d82c6b16f0b69af5a570a3)
This saves an intermediate texture copy step for use-cases
where matrix transformations are the first or only effects
in the chain.
PiperOrigin-RevId: 460239403
(cherry picked from commit 7dc54efdb9c751aff14660fca38889ba2007e4d5)
- Added setter to disable this feature.
- Added accompanying tests.
- Plan to run tests on the same set of settings on H265.
PiperOrigin-RevId: 460238673
(cherry picked from commit 18f4068c06a27ceedce8ee951b3832235a96f75e)
