Upstream timestamps from the decoder are also in microseconds,
so using microseconds here is consistent with that.
PiperOrigin-RevId: 468659099
(cherry picked from commit 4e7f9c575d9ff003f1cc4d4a9f464c6df6b4077a)
Adds a method to FrameProcessor.Listener to be called when an
output frame is available and a method releaseOutputFrame in
FrameProcessor allowing the caller to trigger release of the
oldest available output frame at a given timestamp. Late frames
or frames with unset release times are dropped in the
FinalMatrixTransformationProcessorWrapper.
More than one output frame can become available before they are
released if the penultimate GlTextureProcessor is capable of producing
multiple output frames. Processing continues while waiting for
releaseOutputFrame to be called. Frame release tasks are prioritized
over other tasks.
PiperOrigin-RevId: 468473072
(cherry picked from commit 2c063546a1ac8607eb4632acb88506b9a0d46b9e)
createSupportedTransformationRequest is more accurate than
createFallbackTransformationRequest, as a TransformationRequest will be returned
regardless of whether any fallback is applied.
PiperOrigin-RevId: 466641277
(cherry picked from commit 70972bbab0c884bcc2ef1dfef2177967499c36b6)
Previously, this feature interpreted SDR signals as HDR when called. Now, only HDR
streams are interpreted as HDR, so the javadoc should be updated. Not yet removing
this method, as there are still some loose ends to finish up (ex. PQ support, e2e
tests).
PiperOrigin-RevId: 466425738
(cherry picked from commit 2d2926b8c0b4cded25616558ed74cb30cc2932fd)
While HDR is most closely tied to the color transfer (ex.
COLOR_TRANSFER_SDR is the only one explicitly mentioning dynamic
range), technically color spaces may be associated with HDR as well,
like BT.2020 commonly being used for HDR rather than BT.709 for SDR.
Therefore, it's more specific to mention just that the transfer is HDR.
PiperOrigin-RevId: 466316960
(cherry picked from commit 7075f78eb26daef44275a6fde5362d5139e6df37)
This allows the GlEffectsFrameProcessor to later handle HLG and PQ
differently, or limited and full color range differently.
No functional change intended in this CL.
PiperOrigin-RevId: 466070764
(cherry picked from commit e444eaa5b76074bbf77a6cab6dfe288665a20204)
In the case where this check fails, the downstream frame processor chain won't be able to handle the incoming (SDR) data anyway as we've already set it up for HDR.
PiperOrigin-RevId: 465584814
(cherry picked from commit 0e99d28a79ddca5ba9517ffba73d2c36fc2679a7)
`requestCalculateSsim` more clearly represents the intention of the caller.
Also rephrase the javadoc to simplify it and make it more precise.
PiperOrigin-RevId: 465575578
(cherry picked from commit d90d7d30789ec819d905fd98291f66ab7031f47b)
* Transform frame colors using a defined RGBA Matrix to apply filters.
PiperOrigin-RevId: 464523581
(cherry picked from commit 23f34f8d08855645b342a75b0adfc6c0d600c0ab)
Format.NO_VALUE is a placeholder value for an invalid @C.ColorTransfer, used
for example when the decoder doesn't support this transfer function.
When encountering this invalid value, interpret this as COLOR_TRANSFER_SDR.
Confirmed locally that an exception is thrown when transcoding on p4head, and no exception is thrown when transcoding with this CL.
PiperOrigin-RevId: 464135080
(cherry picked from commit 1a22bcf1e59c1f8e1f8d1fb9d9208a916c92bdb2)
This will allow effects preview in ExoPlayer to use the
Effect and FrameProcessor interface (and the interfaces
they depend on) without depending on transformer or the
future effects module.
PiperOrigin-RevId: 464060047
(cherry picked from commit 06d41c2775137158f831a1b2f19f137efb1e08b0)
* Sets KEY_HDR_STATIC_INFO from MediaFormat in the DefaultCodec.
* Adds checks in mediaparser to ensure color space, range, and transfer are valid
values.
PiperOrigin-RevId: 463921325
(cherry picked from commit 44c42fef2aaa65ba78c52883e3fcec94c8d7ad2d)
TransformationException error codes were previously tied to OpenGL
but other FrameProcessor implementations are possible. So this CL
renames the error codes.
Also, remove GL_INIT_FAILED error code, as FrameProcessor
implemenations may initialize resources on a background thread
after the factory method returns, so it's not obvious how to
distinguish between initialization failures and processing failures.
PiperOrigin-RevId: 463704902
(cherry picked from commit bac323d346c1d3118a0b904caf9926ecb96b912d)
This allows non-GL effects to be passed to custom FrameProcessor
implementations.
PiperOrigin-RevId: 463696384
(cherry picked from commit 66dde42989dbc8fedeee59897a05644d86240cf6)
Both are used in the public FrameProcessor interface, so they
should be public too.
PiperOrigin-RevId: 463454859
(cherry picked from commit 40e96d1534d34312611d8c603b1b7c92e3250938)
This is possible because SSIM increases monotonically with bitrate.
PiperOrigin-RevId: 463434373
(cherry picked from commit 63cddc901028434a9a565a0a8c605b46f223e724)
Just a misc nit, since I found the name a bit confusing, and figured
findEncoderWithClosestSupportedFormat might be more descriptive.
PiperOrigin-RevId: 463433646
(cherry picked from commit 324fc5d7e88d9e4fdb1ecc7e70106d641f8236ec)
Extract a FrameProcessor.Factory interface from GlEffectsFrameProcessor
and allow it to be customized using a setter on Transformer.Builder.
PiperOrigin-RevId: 463433438
(cherry picked from commit ecbe9a502cf62c92005a2ff045ab17988ea2838f)
Size requires API 21. Using Pair instead will allow effects to be
used from API 18 during previewing once they are moved out of
transformer.
PiperOrigin-RevId: 463206474
(cherry picked from commit b994f8bfa06c82fbfe96044b986cd7b8d243acdd)
This is needed for applying effects to a playlist.
The effects are applied based on the presentation time of the
frame in its corresponding media item and the offset is added
back before encoding.
Each time the offset changes, end of input stream is signalled
to the texture processors. This is needed because the texture
processors can expect monotonically increasing timestamp within
the same input stream but when the offset changes, the timstamps
jump back to 0.
PiperOrigin-RevId: 462714966
(cherry picked from commit 46ab06b816d93c2a034c182552fa4483a6d82cba)
It's always safe to ignore the result of these methods, because the
caller already has a reference to the returned value.
PiperOrigin-RevId: 462388947
(cherry picked from commit 21cab6124de7b65f1af016cfc260555fa0b6e738)
This allows us to use BT.2020 RGB linear for intermediate shaders, which also
allows us to re-enable PeriodicVignetteProcessor, which should work properly in
linear color-spaces.
Manually tested by adding a GlEffectsWrapper, and confirming that HLG HDR editing still looks correct.
PiperOrigin-RevId: 462265821
(cherry picked from commit 1ecf1eb7bfdd469fe855edf6126b2822b52ff07f)
Otherwise, invalid ColorInfo instances generated using faulty
MediaFormat#getInteger values could cause exceptions.
Confirmed that b/239435670 reproduces without this CL, and does not reproduce
with this CL.
PiperOrigin-RevId: 461862191
(cherry picked from commit 5533ad1713bd0dc64a6fba485dbbfeb35e177523)
* Introduced `useHdr` for `GlEffect#toGlTextureProcessor`, so
`TextureProcessor` implementations can decide how to handle HDR.
* Creating FP16 color textures for HDR input.
Tested via manual testing, adding a no-op GlEffectWrapper to the transformation to
force use of intermediate textures, adding a linear ramp to the fragment shader,
and trying to ascertain that there's a real reduction in posterization when
switching from 4-bit to 8-bit unsigned bytes, and again from 8-bit unsigned bytes
to 16-bit floating point.
PiperOrigin-RevId: 461613117
(cherry picked from commit f67c1a73f443f411771676b761ceb03e7a0d48e4)
This may happen when a containers' color transfer incorrectly does not match
the video's color transfer.
An example of a file with such a mismatch is the current Transformer demo HDR10
sample file.
Manually tested by confirming that no errors are emitted for SDR and HLG sample
files, and that errors are emitted for our incorrect HDR10 sample file.
PiperOrigin-RevId: 461583532
(cherry picked from commit fd046bd2f63888f8a47c37a426c10212a56be6e0)
- Update profile selection logic to pick an HDR-compatible profile when doing HDR editing on H.264/AVC videos.
- Handle doing the capabilities check for all MIME types that support HDR (not just H.265/HEVC).
- Fix a bug where we would pass an HDR input color format to the encoder when using tone-mapping.
- Tweak how `EncoderWrapper` works so decisions at made at construction time.
Manually tested cases:
- Transformation of an SDR video.
- Transformation of an HDR video to AVC (which triggers fallback/tone-mapping on a device that doesn't support HDR editing for AVC).
- Transformation of an HDR video with HDR editing.
PiperOrigin-RevId: 461572973
(cherry picked from commit 0db07c6791620d06db9f524bf0f47e90df635756)
These are providing more variety and complexity.
All files are okay to be public.
PiperOrigin-RevId: 460935247
(cherry picked from commit d4653c335e2bafa79cc590101bf32481e04b1bcd)
Also remove VideoEncoderSettings.colorProfile as there are no
concrete use cases for customizing this and it clashes with picking
the color format automatically based on SDR vs. HDR.
PiperOrigin-RevId: 460746987
(cherry picked from commit db78a87fdcb095fa6520f3c1bd8b5b419d33de50)
Add test that verifies SSIM with API enabled.
#minor-release
PiperOrigin-RevId: 460692420
(cherry picked from commit 6ec18c8a208842eeb178f97e0680770b2e2a8b29)
Pass the color info and HDR static metadata when configuring the decoder
using MediaFormatUtil.maybeSetColorInfo.
PiperOrigin-RevId: 460424985
(cherry picked from commit 46cd92ac1d80b2e651aa0ac2403e6d52d9c1d767)
This extension is needed for editing HDR input with OpenGL, as the
ExternalTextureProcessor samples raw YUV values from the
external texture for HDR and converts them to RGB itself rather than
relying on the OpenGL driver to do this automatically as for SDR.
PiperOrigin-RevId: 460424154
(cherry picked from commit 413016f7efea9dbb1194558374d9b9a8777a0d8c)
