* Transform frame colors using a defined RGBA Matrix to apply filters.
PiperOrigin-RevId: 464523581
(cherry picked from commit 23f34f8d08855645b342a75b0adfc6c0d600c0ab)
Format.NO_VALUE is a placeholder value for an invalid @C.ColorTransfer, used
for example when the decoder doesn't support this transfer function.
When encountering this invalid value, interpret this as COLOR_TRANSFER_SDR.
Confirmed locally that an exception is thrown when transcoding on p4head, and no exception is thrown when transcoding with this CL.
PiperOrigin-RevId: 464135080
(cherry picked from commit 1a22bcf1e59c1f8e1f8d1fb9d9208a916c92bdb2)
This will allow effects preview in ExoPlayer to use the
Effect and FrameProcessor interface (and the interfaces
they depend on) without depending on transformer or the
future effects module.
PiperOrigin-RevId: 464060047
(cherry picked from commit 06d41c2775137158f831a1b2f19f137efb1e08b0)
* Sets KEY_HDR_STATIC_INFO from MediaFormat in the DefaultCodec.
* Adds checks in mediaparser to ensure color space, range, and transfer are valid
values.
PiperOrigin-RevId: 463921325
(cherry picked from commit 44c42fef2aaa65ba78c52883e3fcec94c8d7ad2d)
TransformationException error codes were previously tied to OpenGL
but other FrameProcessor implementations are possible. So this CL
renames the error codes.
Also, remove GL_INIT_FAILED error code, as FrameProcessor
implemenations may initialize resources on a background thread
after the factory method returns, so it's not obvious how to
distinguish between initialization failures and processing failures.
PiperOrigin-RevId: 463704902
(cherry picked from commit bac323d346c1d3118a0b904caf9926ecb96b912d)
This allows non-GL effects to be passed to custom FrameProcessor
implementations.
PiperOrigin-RevId: 463696384
(cherry picked from commit 66dde42989dbc8fedeee59897a05644d86240cf6)
Both are used in the public FrameProcessor interface, so they
should be public too.
PiperOrigin-RevId: 463454859
(cherry picked from commit 40e96d1534d34312611d8c603b1b7c92e3250938)
This is possible because SSIM increases monotonically with bitrate.
PiperOrigin-RevId: 463434373
(cherry picked from commit 63cddc901028434a9a565a0a8c605b46f223e724)
Just a misc nit, since I found the name a bit confusing, and figured
findEncoderWithClosestSupportedFormat might be more descriptive.
PiperOrigin-RevId: 463433646
(cherry picked from commit 324fc5d7e88d9e4fdb1ecc7e70106d641f8236ec)
Extract a FrameProcessor.Factory interface from GlEffectsFrameProcessor
and allow it to be customized using a setter on Transformer.Builder.
PiperOrigin-RevId: 463433438
(cherry picked from commit ecbe9a502cf62c92005a2ff045ab17988ea2838f)
Size requires API 21. Using Pair instead will allow effects to be
used from API 18 during previewing once they are moved out of
transformer.
PiperOrigin-RevId: 463206474
(cherry picked from commit b994f8bfa06c82fbfe96044b986cd7b8d243acdd)
This is needed for applying effects to a playlist.
The effects are applied based on the presentation time of the
frame in its corresponding media item and the offset is added
back before encoding.
Each time the offset changes, end of input stream is signalled
to the texture processors. This is needed because the texture
processors can expect monotonically increasing timestamp within
the same input stream but when the offset changes, the timstamps
jump back to 0.
PiperOrigin-RevId: 462714966
(cherry picked from commit 46ab06b816d93c2a034c182552fa4483a6d82cba)
It's always safe to ignore the result of these methods, because the
caller already has a reference to the returned value.
PiperOrigin-RevId: 462388947
(cherry picked from commit 21cab6124de7b65f1af016cfc260555fa0b6e738)
This allows us to use BT.2020 RGB linear for intermediate shaders, which also
allows us to re-enable PeriodicVignetteProcessor, which should work properly in
linear color-spaces.
Manually tested by adding a GlEffectsWrapper, and confirming that HLG HDR editing still looks correct.
PiperOrigin-RevId: 462265821
(cherry picked from commit 1ecf1eb7bfdd469fe855edf6126b2822b52ff07f)
Otherwise, invalid ColorInfo instances generated using faulty
MediaFormat#getInteger values could cause exceptions.
Confirmed that b/239435670 reproduces without this CL, and does not reproduce
with this CL.
PiperOrigin-RevId: 461862191
(cherry picked from commit 5533ad1713bd0dc64a6fba485dbbfeb35e177523)
* Introduced `useHdr` for `GlEffect#toGlTextureProcessor`, so
`TextureProcessor` implementations can decide how to handle HDR.
* Creating FP16 color textures for HDR input.
Tested via manual testing, adding a no-op GlEffectWrapper to the transformation to
force use of intermediate textures, adding a linear ramp to the fragment shader,
and trying to ascertain that there's a real reduction in posterization when
switching from 4-bit to 8-bit unsigned bytes, and again from 8-bit unsigned bytes
to 16-bit floating point.
PiperOrigin-RevId: 461613117
(cherry picked from commit f67c1a73f443f411771676b761ceb03e7a0d48e4)
This may happen when a containers' color transfer incorrectly does not match
the video's color transfer.
An example of a file with such a mismatch is the current Transformer demo HDR10
sample file.
Manually tested by confirming that no errors are emitted for SDR and HLG sample
files, and that errors are emitted for our incorrect HDR10 sample file.
PiperOrigin-RevId: 461583532
(cherry picked from commit fd046bd2f63888f8a47c37a426c10212a56be6e0)
- Update profile selection logic to pick an HDR-compatible profile when doing HDR editing on H.264/AVC videos.
- Handle doing the capabilities check for all MIME types that support HDR (not just H.265/HEVC).
- Fix a bug where we would pass an HDR input color format to the encoder when using tone-mapping.
- Tweak how `EncoderWrapper` works so decisions at made at construction time.
Manually tested cases:
- Transformation of an SDR video.
- Transformation of an HDR video to AVC (which triggers fallback/tone-mapping on a device that doesn't support HDR editing for AVC).
- Transformation of an HDR video with HDR editing.
PiperOrigin-RevId: 461572973
(cherry picked from commit 0db07c6791620d06db9f524bf0f47e90df635756)
These are providing more variety and complexity.
All files are okay to be public.
PiperOrigin-RevId: 460935247
(cherry picked from commit d4653c335e2bafa79cc590101bf32481e04b1bcd)
Also remove VideoEncoderSettings.colorProfile as there are no
concrete use cases for customizing this and it clashes with picking
the color format automatically based on SDR vs. HDR.
PiperOrigin-RevId: 460746987
(cherry picked from commit db78a87fdcb095fa6520f3c1bd8b5b419d33de50)
Add test that verifies SSIM with API enabled.
#minor-release
PiperOrigin-RevId: 460692420
(cherry picked from commit 6ec18c8a208842eeb178f97e0680770b2e2a8b29)
Pass the color info and HDR static metadata when configuring the decoder
using MediaFormatUtil.maybeSetColorInfo.
PiperOrigin-RevId: 460424985
(cherry picked from commit 46cd92ac1d80b2e651aa0ac2403e6d52d9c1d767)
This extension is needed for editing HDR input with OpenGL, as the
ExternalTextureProcessor samples raw YUV values from the
external texture for HDR and converts them to RGB itself rather than
relying on the OpenGL driver to do this automatically as for SDR.
PiperOrigin-RevId: 460424154
(cherry picked from commit 413016f7efea9dbb1194558374d9b9a8777a0d8c)
This saves an intermediate texture copy step for use-cases
where matrix transformations are the first or only effects
in the chain.
PiperOrigin-RevId: 460239403
(cherry picked from commit 0615922cb64a649a4b01eb6410680f6f754e2005)
- Added setter to disable this feature.
- Added accompanying tests.
- Plan to run tests on the same set of settings on H265.
PiperOrigin-RevId: 460238673
(cherry picked from commit 6e126000993f5bbb9d1f2fd73df141158e8c7785)
Some other minor nits and adjustments to the API logic.
PiperOrigin-RevId: 459490431
(cherry picked from commit a36967c19bd9fee09b7df534befad7cae1d5453b)
The SDR constant also specified a color space and range, in addition to
C.COLOR_TRANSFER_SDR. However, it turns out that SDR videos may use different color
space and range values, so following prior ExoPlayer conventions to have `null`
mean "generic SDR" is preferable here.
PiperOrigin-RevId: 459296746
(cherry picked from commit 4230ea94e956bacc1bb3b6737686a845bc077dd9)
If the input is HDR (HLG), check encoder capabilities for HDR support
and request tone-mapping to SDR during decoder configuration otherwise.
Capabilities are only checked for API 31 and above, as HDR editing is
not supported before.
As the encoder capabilities check needs to happen before selecting the
encoder to use (as this may depend on the resolution output by the
effects chain), the EncoderWrapper checks all candidate encoders
for the MIME type for HDR capabilities and only requests fallback to
SDR if none of them support it.
When the actual encoder is selected, the wrapper checks that it matches
one of the encoders is checked capabilities for.
PiperOrigin-RevId: 458511599
(cherry picked from commit 474e2f2c77cad4759645ec3320f756757c0882db)
Configure the GL shaders and encoder to take in HDR metadata.
This mostly just consists of passing the Format.colorInfo through
the VideoTranscodingSamplePipeline down to the encoder, rather than passing
the PQ-ness down to the GL step.
Due to b/237674316, this will remove HDR10+ support temporarily to introduce
support for HLG10.
Manually tested to confirm that HLG10 operations that don't affect color display
correctly after this CL with "HDR editing" in the demo checked, and continue to display incorrectly (as before this CL) without the option unchecked.
PiperOrigin-RevId: 458490810
(cherry picked from commit 53db305612a84466b6712cc355dbf918424a968d)
The old getString() will throw because FRAME_RATE can only be float or int.
PiperOrigin-RevId: 458481251
(cherry picked from commit 2b6f62b2a124e3a635ddcba7d9f256c52b564d89)
We used "ALL_COOECS" previously, and it is not necessary because "ALL_CODECS"
additionally the codecs that support tunneling/secure decoding, which there
is no use case in Transformer.
PiperOrigin-RevId: 458470278
(cherry picked from commit 8f89eba59c151f301d00d4fed200fd8f9359c9e2)
