208 Commits

Author SHA1 Message Date
huangdarwin
53978104e6 HDR: For HDR videos, enableHdrEditing by default.
Previously, tone-mapping was the default.

PiperOrigin-RevId: 488362209
2022-11-14 17:31:08 +00:00
huangdarwin
680f0c3af7 HDR: Add public HLG test file.
Add an HLG Pixel 7 Pro video and its associated format for testing.

PiperOrigin-RevId: 488353926
2022-11-14 17:22:26 +00:00
huangdarwin
8bdd2784d3 HDR: Implement ForceInterpretHdrVideoAsSdr
Also, document that we tone map when no HDR features are explicitly set

PiperOrigin-RevId: 487310971
2022-11-10 15:09:41 +00:00
huangdarwin
0b53c934d8 HDR: Update test FileUtil to handle null ColorInfo.
PiperOrigin-RevId: 486706595
2022-11-08 14:25:45 +00:00
huangdarwin
bd4e58d68b HDR Test: Add link to tested method.
PiperOrigin-RevId: 485890141
2022-11-08 14:13:49 +00:00
huangdarwin
faa796d736 HDR: Disable tone mapping on unsupported pixel build ID.
Also, update tests to allow AnyOf error codes, and no longer check exception messages, which caused quite a bit of churn.

PiperOrigin-RevId: 479570861
2022-10-20 01:13:42 +00:00
huangdarwin
36e41059ea HDR: Check isToneMappingApplied in HDR editing test.
Assert that tone mapping is applied when an HDR edit cannot be HDR, but is successfully tone mapped. Meanwhile, assert that fallback, which is applied after codec configuration (which throws the "Tone-mapping requested but not supported by the decoder" error) is not applied when that error is called.

PiperOrigin-RevId: 478762951
2022-10-20 00:45:11 +00:00
huangdarwin
b9a3aa5ca1 HDR: Add tests with HLG10 input.
PiperOrigin-RevId: 478019046
2022-10-20 00:38:08 +00:00
huangdarwin
9786802527 Test: Rename to avoid redundant substrings.
Rename test files to avoid substrings that can be implied by the directory name,
like "Transformation" and "Test"

No functional changes. Renaming-only.

PiperOrigin-RevId: 477724724
2022-10-20 00:27:26 +00:00
huangdarwin
05ce639ea0 HDR: Add period to end of error message.
For tone mapping error messages.

PiperOrigin-RevId: 477447799
2022-10-20 00:20:19 +00:00
huangdarwin
507a1be61d HDR: Add tests for tone mapping.
Also, add checks for output file color.

PiperOrigin-RevId: 477439139
2022-10-20 00:16:37 +00:00
huangdarwin
df58107eae HDR: Assert output C.ColorTransfer for tests.
To confirm that tone mapping did or did not happen.

PiperOrigin-RevId: 476354606
2022-10-19 23:20:59 +00:00
samrobinson
20aa22c9fa Add MssimCalculatorTest to verify SSIM calculations.
As part of this change, MssimCalculator is moved from androidTest/ to main/

PiperOrigin-RevId: 473771344
2022-10-19 22:24:34 +00:00
samrobinson
8f9c9d0f88 Rollback of eb6c1183b6
*** Original commit ***

Handle int instead of byte in SSIM.

The value of pixels are converted to integers at the point of use,
move this logic to the initialisation step.

This is a prerequisite step for testing SSIM calculation, which
will lead on to some SSIM improvements being verifiable.

Tested manually and SSIM values match for the same video
before and after this change.

***

PiperOrigin-RevId: 473259446
2022-10-19 22:06:07 +00:00
samrobinson
eb6c1183b6 Handle int instead of byte in SSIM.
The value of pixels are converted to integers at the point of use,
move this logic to the initialisation step.

This is a prerequisite step for testing SSIM calculation, which
will lead on to some SSIM improvements being verifiable.

Tested manually and SSIM values match for the same video
before and after this change.

PiperOrigin-RevId: 473231779
2022-10-19 22:02:35 +00:00
huangdarwin
8ee2121627 HDR: Add fallback MH tests.
Test that HDR editing succeeds on devices supporting HDR editing, tone maps on
devices supporting tone mapping, and throws exceptions on all other devices.

Also, only restrict HDR editing and tone mapping support to API 31+ only when
transcoding, not for all transformations.

PiperOrigin-RevId: 472958965
2022-10-19 21:44:21 +00:00
claincly
caefa565f7 Remove conditional check for no-longer flaky API version
PiperOrigin-RevId: 472949850
2022-10-19 21:40:42 +00:00
claincly
9a9baa54a5 Remove redundant constant definition
PiperOrigin-RevId: 472522846
2022-10-19 21:25:53 +00:00
claincly
6e45824992 Include CamcorderProfile resolution in encoder capability test
PiperOrigin-RevId: 472459423
2022-10-19 21:07:09 +00:00
andrewlewis
1023254d8e Update color info mismatch test
This should now expect transformation to succeed.

PiperOrigin-RevId: 470950411
2022-10-19 19:55:44 +00:00
andrewlewis
d90d7d3078 Clarify SSIM request method name
`requestCalculateSsim` more clearly represents the intention of the caller.

Also rephrase the javadoc to simplify it and make it more precise.

PiperOrigin-RevId: 465575578
2022-10-19 16:55:11 +00:00
Googler
707b061838 Move effects functionality out of transformer to effects module.
PiperOrigin-RevId: 465038852
2022-10-19 16:00:13 +00:00
leonwind
23f34f8d08 Implement RGBA color matrices processor
* Transform frame colors using a defined RGBA Matrix to apply filters.

PiperOrigin-RevId: 464523581
2022-10-19 15:46:01 +00:00
leonwind
ea213545e0 Migrate Contrast asset to emulator generated one.
PiperOrigin-RevId: 464519030
2022-10-19 15:42:30 +00:00
huangdarwin
1a22bcf1e5 HDR: Have @C.ColorTransfer check consider NO_VALUE as SDR.
Format.NO_VALUE is a placeholder value for an invalid @C.ColorTransfer, used
for example when the decoder doesn't support this transfer function.

When encountering this invalid value, interpret this as COLOR_TRANSFER_SDR.

Confirmed locally that an exception is thrown when transcoding on p4head, and no exception is thrown when transcoding with this CL.

PiperOrigin-RevId: 464135080
2022-10-19 15:38:43 +00:00
Googler
06d41c2775 Move FrameProcessor and related interfaces to common.
This will allow effects preview in ExoPlayer to use the
Effect and FrameProcessor interface (and the interfaces
they depend on) without depending on transformer or the
future effects module.

PiperOrigin-RevId: 464060047
2022-10-19 15:34:54 +00:00
Googler
66dde42989 Add Effect marker interface.
This allows non-GL effects to be passed to custom FrameProcessor
implementations.

PiperOrigin-RevId: 463696384
2022-10-19 14:56:44 +00:00
samrobinson
5f1ee86eb4 Add 8 new videos, representing lower resolutions.
Downsampled from 3840x2160 resolution videos using ffmpeg:

`ffmpeg -i <inputFilename> -filter:v scale=<width>:<height> -c:a copy <outputFilename>`

PiperOrigin-RevId: 463454628
2022-10-19 14:45:21 +00:00
samrobinson
e1b6ed3252 Add H265 as a parameter to SsimMapperTest.
PiperOrigin-RevId: 463452893
2022-10-19 14:41:30 +00:00
leonwind
03ff5246e7 Rename ContrastProcessorPixelTest and make contexts final
PiperOrigin-RevId: 463450395
2022-10-19 14:37:49 +00:00
samrobinson
63cddc9010 Add an SsimMapperTest, which binary searches for 95% ssim.
This is possible because SSIM increases monotonically with bitrate.

PiperOrigin-RevId: 463434373
2022-10-19 14:34:04 +00:00
Googler
ecbe9a502c Allow FrameProcessor.Factory to be set on Transformer.Builder.
Extract a FrameProcessor.Factory interface from GlEffectsFrameProcessor
and allow it to  be customized using a setter on Transformer.Builder.

PiperOrigin-RevId: 463433438
2022-10-19 14:26:43 +00:00
Googler
b994f8bfa0 Replace Size with Pair in effects.
Size requires API 21. Using Pair instead will allow effects to be
used from API 18 during previewing once they are moved out of
transformer.

PiperOrigin-RevId: 463206474
2022-10-19 14:11:40 +00:00
hschlueter
46ab06b816 Allow stream offset passed to GlEffectsFrameProcessor to change.
This is needed for applying effects to a playlist.

The effects are applied based on the presentation time of the
frame in its corresponding media item and the offset is added
back before encoding.

Each time the offset changes, end of input stream is signalled
to the texture processors. This is needed because the texture
processors can expect monotonically increasing timestamp within
the same input stream but when the offset changes, the timstamps
jump back to 0.

PiperOrigin-RevId: 462714966
2022-07-25 22:23:20 +01:00
ibaker
21cab6124d Annotate methods that always return this with @CanIgnoreReturnValue
It's always safe to ignore the result of these methods, because the
caller already has a reference to the returned value.

PiperOrigin-RevId: 462388947
2022-07-25 22:19:11 +01:00
olly
7ef41c7321 Add ContrastProcessor for contrast adjustments.
PiperOrigin-RevId: 462232813
2022-07-21 13:19:55 +00:00
huangdarwin
f67c1a73f4 HDR: Use FP16 color representation for texture processors.
* Introduced `useHdr` for `GlEffect#toGlTextureProcessor`, so
  `TextureProcessor` implementations can decide how to handle HDR.
* Creating FP16 color textures for HDR input.

Tested via manual testing, adding a no-op GlEffectWrapper to the transformation to
force use of intermediate textures, adding a linear ramp to the fragment shader,
and trying to ascertain that there's a real reduction in posterization when
switching from 4-bit to 8-bit unsigned bytes, and again from 8-bit unsigned bytes
to 16-bit floating point.

PiperOrigin-RevId: 461613117
2022-07-21 12:55:36 +00:00
huangdarwin
fd046bd2f6 HDR: Throw when unexpected color transfer encountered.
This may happen when a containers' color transfer incorrectly does not match
the video's color transfer.

An example of a file with such a mismatch is the current Transformer demo HDR10
sample file.

Manually tested by confirming that no errors are emitted for SDR and HLG sample
files, and that errors are emitted for our incorrect HDR10 sample file.

PiperOrigin-RevId: 461583532
2022-07-21 12:52:07 +00:00
samrobinson
d4653c335e Add additional video files.
These are providing more variety and complexity.

All files are okay to be public.

PiperOrigin-RevId: 460935247
2022-07-15 10:23:13 +00:00
samrobinson
6ec18c8a20 Fix assertion error when using high quality targeting API.
Add test that verifies SSIM with API enabled.

#minor-release

PiperOrigin-RevId: 460692420
2022-07-13 18:15:17 +00:00
claincly
fa55078641 Fix a mis-match in encoder priority.
PiperOrigin-RevId: 460500666
2022-07-13 17:51:05 +00:00
hschlueter
f41e6796ce Move DebugViewProvider out of Transformer class.
The GlEffectsFrameProcessor that will be part of the effects module
uses the DebugViewProvider. So it does not make sense for it
to be an inner interface of Transformer.

PiperOrigin-RevId: 458014932
2022-07-07 17:16:30 +00:00
hschlueter
e5527a8add FrameProcessor: Replace SurfaceInfo.Provider with setter.
The FinalMatrixTransformationProcessorWrapper ensures that the
surface is only replaced when it is not being rendered to and vice
versa.

PiperOrigin-RevId: 458007639
2022-07-07 17:12:50 +00:00
hschlueter
bfa663d2b1 Allow changing input pixelWidthHeightRatio for GlEffectsFrameProcessor.
pixelWidthHeightRatio is now passed to setInputFrameInfo instead of
the factory.

PiperOrigin-RevId: 457696703
2022-07-07 16:53:01 +00:00
hschlueter
a230d59f1b Allow FrameProcessor input surface size changes.
This will be useful for downgrading to a lower resolution during
a slow preview and for processing slide-shows once sequential
multi-asset editing is supported.

PiperOrigin-RevId: 457017255
2022-06-27 11:59:48 +01:00
hschlueter
1b5dd92dec Extract FrameProcessor interface from GlEffectsFrameProcessor.
PiperOrigin-RevId: 456814150
2022-06-27 11:53:24 +01:00
hschlueter
d9c63c1e87 Disable flaky transformer emulator test on API 31.
videoEncoderFormatUnsupported_completesWithError() has recently
been flaky on API 31 emulators on presubmit because a different
exception than the expected exception is thrown.
This disables it on those emulators to reduce testing noise
until the underlying problem is investigated and resolved.

PiperOrigin-RevId: 456765512
2022-06-27 11:50:05 +01:00
hschlueter
216fefd669 Rename FrameProcessorChain to GlEffectsFrameProcessor.
This change is just renaming. There is no functional change intended.
The FrameProcessor interface will be created in a follow-up.

PiperOrigin-RevId: 456741628
2022-06-27 11:43:32 +01:00
samrobinson
19bdff96ba Add a Builder for DefaultEncoderFactory.
PiperOrigin-RevId: 456728032
2022-06-27 11:40:04 +01:00
hschlueter
e25bf81195 Use GlTextureProcessor to avoid redundant copy in MediaPipeProcessor.
After this change GlEffects can use any GlTextureProcessor not just
SingleFrameGlTextureProcessor.
MediaPipeProcessor now implements GlTextureProcessor directly which
allows it to reuse MediaPipe's output texture for its output texture
and avoids an extra copy shader step.

PiperOrigin-RevId: 456530718
2022-06-27 11:33:30 +01:00