707 Commits

Author SHA1 Message Date
bachinger
28c4be2f6b Map lib-effect from androidx to exoplayer2
PiperOrigin-RevId: 482179761
2022-10-20 03:15:38 +00:00
kimvde
bd9181e6ba Remove deprecated setOutputMimeType
This is to prepare Muxer to become public

PiperOrigin-RevId: 481893842
2022-10-20 02:53:21 +00:00
samrobinson
b6bd35860c Fix parameter comment block.
#cleanup

PiperOrigin-RevId: 481882181
2022-10-20 02:49:37 +00:00
kimvde
b4d7f066dd Add DefaultMuxer forwarding to FrameworkMuxer
- The naming DefaultMuxer is more consistent with the rest of
Transformer codebase (e.g. DefaultEncoderFactory).
- By hiding the implementation details of DefaultMuxer, the transition
to in-app Muxer will be seamless for apps using DefaultMuxer.
- The current plan is that DefaultMuxer will become the in-app muxer.

PiperOrigin-RevId: 481838790
2022-10-20 02:42:22 +00:00
kimvde
8962f5a3f4 Remove supportsSampleMimeType from Muxer.Factory
- This method is redundant with getSupportedSampleMimeTypes().
- This is to prepare the Muxer class to become public.

PiperOrigin-RevId: 480840902
2022-10-20 01:54:12 +00:00
huangdarwin
faa796d736 HDR: Disable tone mapping on unsupported pixel build ID.
Also, update tests to allow AnyOf error codes, and no longer check exception messages, which caused quite a bit of churn.

PiperOrigin-RevId: 479570861
2022-10-20 01:13:42 +00:00
huangdarwin
36e41059ea HDR: Check isToneMappingApplied in HDR editing test.
Assert that tone mapping is applied when an HDR edit cannot be HDR, but is successfully tone mapped. Meanwhile, assert that fallback, which is applied after codec configuration (which throws the "Tone-mapping requested but not supported by the decoder" error) is not applied when that error is called.

PiperOrigin-RevId: 478762951
2022-10-20 00:45:11 +00:00
huangdarwin
b9a3aa5ca1 HDR: Add tests with HLG10 input.
PiperOrigin-RevId: 478019046
2022-10-20 00:38:08 +00:00
huangdarwin
9786802527 Test: Rename to avoid redundant substrings.
Rename test files to avoid substrings that can be implied by the directory name,
like "Transformation" and "Test"

No functional changes. Renaming-only.

PiperOrigin-RevId: 477724724
2022-10-20 00:27:26 +00:00
huangdarwin
05ce639ea0 HDR: Add period to end of error message.
For tone mapping error messages.

PiperOrigin-RevId: 477447799
2022-10-20 00:20:19 +00:00
huangdarwin
507a1be61d HDR: Add tests for tone mapping.
Also, add checks for output file color.

PiperOrigin-RevId: 477439139
2022-10-20 00:16:37 +00:00
huangdarwin
df58107eae HDR: Assert output C.ColorTransfer for tests.
To confirm that tone mapping did or did not happen.

PiperOrigin-RevId: 476354606
2022-10-19 23:20:59 +00:00
leonwind
afd829e5db Add OETF and EOTF ES2 fragment shaders for non-HDR frames.
* Transform the intermediate color space to linear SDR by applying the SMPTE 170M EOTF and OETF.
* Use linear colors for the color filter pixel tests and update all golden bitmaps.

PiperOrigin-RevId: 476124592
2022-10-19 23:13:26 +00:00
samrobinson
08bb01b218 Add file size to TransformationResult at the end of a Transformation.
Tested:
  * Manually using both path and file descriptor.
PiperOrigin-RevId: 475860978
2022-10-19 23:02:18 +00:00
samrobinson
6dd2a6dac6 Add support for skipping windows in MssimCalculator.
By skipping every other row and column, SSIM calculation time reduces by 10-30%.

PiperOrigin-RevId: 474286702
2022-10-19 22:35:54 +00:00
samrobinson
20aa22c9fa Add MssimCalculatorTest to verify SSIM calculations.
As part of this change, MssimCalculator is moved from androidTest/ to main/

PiperOrigin-RevId: 473771344
2022-10-19 22:24:34 +00:00
samrobinson
8f9c9d0f88 Rollback of eb6c1183b6
*** Original commit ***

Handle int instead of byte in SSIM.

The value of pixels are converted to integers at the point of use,
move this logic to the initialisation step.

This is a prerequisite step for testing SSIM calculation, which
will lead on to some SSIM improvements being verifiable.

Tested manually and SSIM values match for the same video
before and after this change.

***

PiperOrigin-RevId: 473259446
2022-10-19 22:06:07 +00:00
samrobinson
eb6c1183b6 Handle int instead of byte in SSIM.
The value of pixels are converted to integers at the point of use,
move this logic to the initialisation step.

This is a prerequisite step for testing SSIM calculation, which
will lead on to some SSIM improvements being verifiable.

Tested manually and SSIM values match for the same video
before and after this change.

PiperOrigin-RevId: 473231779
2022-10-19 22:02:35 +00:00
huangdarwin
41963fce9b Decoder: Reduce max pending frame count limit down to 5.
PiperOrigin-RevId: 473212543
2022-10-19 21:55:20 +00:00
huangdarwin
8ee2121627 HDR: Add fallback MH tests.
Test that HDR editing succeeds on devices supporting HDR editing, tone maps on
devices supporting tone mapping, and throws exceptions on all other devices.

Also, only restrict HDR editing and tone mapping support to API 31+ only when
transcoding, not for all transformations.

PiperOrigin-RevId: 472958965
2022-10-19 21:44:21 +00:00
claincly
caefa565f7 Remove conditional check for no-longer flaky API version
PiperOrigin-RevId: 472949850
2022-10-19 21:40:42 +00:00
claincly
9a9baa54a5 Remove redundant constant definition
PiperOrigin-RevId: 472522846
2022-10-19 21:25:53 +00:00
claincly
6e45824992 Include CamcorderProfile resolution in encoder capability test
PiperOrigin-RevId: 472459423
2022-10-19 21:07:09 +00:00
huangdarwin
8e14611d60 Minor javadoc and scoping cleanup.
No functional changes.

PiperOrigin-RevId: 472245797
2022-10-19 20:44:40 +00:00
huangdarwin
1a4cd549a4 Rename shouldPassthrough to shouldTranscode
shouldPassthrough's internal checks seem to be check whether we should *not*
pass through, which seemed a bit like a confusing double-negative to me.

shouldTranscode is slightly more clear, by instead returning true when we do
want to transcode.

No functional changes intended.

PiperOrigin-RevId: 471753771
2022-10-19 20:36:58 +00:00
Googler
94713a8ffb Skip transcoding if HDR video does not need encoding.
3b0d2c1586 made `shouldPassthrough` always return false for `enableHdrVideoEditing`:

>We force using `FrameEditor` (no passthrough) to avoid the need to select another edit operation, and use the new shaders. The `EGLContext` and `EGLSurface` also need to be set up differently for this path.

However, this was introduced before the `videoNeedsEncoding` setting was introduced in 3f615040c0. That setting should apply to HDR videos as much as SDR videos.

PiperOrigin-RevId: 471569853
2022-10-19 20:33:15 +00:00
ibaker
e285e70785 Remove media3-only line from exoplayer2 build.gradle file
#minor-release

PiperOrigin-RevId: 470999044
2022-10-19 20:03:06 +00:00
andrewlewis
1023254d8e Update color info mismatch test
This should now expect transformation to succeed.

PiperOrigin-RevId: 470950411
2022-10-19 19:55:44 +00:00
andrewlewis
f1a3a40eb1 Log instead of throwing for transfer mismatch
PiperOrigin-RevId: 470354448
2022-10-19 19:52:00 +00:00
Googler
4e7f9c575d Change onOutputFrameAvailable timestamp from nanos to micros.
Upstream timestamps from the decoder are also in microseconds,
so using microseconds here is consistent with that.

PiperOrigin-RevId: 468659099
2022-10-19 18:57:51 +00:00
Googler
2c063546a1 Allow frame release to be controlled outside FrameProcessor.
Adds a method to FrameProcessor.Listener to be called when an
output frame is available and a method releaseOutputFrame in
FrameProcessor allowing the caller to trigger release of the
oldest available output frame at a given timestamp. Late frames
or frames with unset release times are dropped in the
FinalMatrixTransformationProcessorWrapper.

More than one output frame can become available before they are
released if the penultimate GlTextureProcessor is capable of producing
multiple output frames. Processing continues while waiting for
releaseOutputFrame to be called. Frame release tasks are prioritized
over other tasks.

PiperOrigin-RevId: 468473072
2022-10-19 18:50:08 +00:00
huangdarwin
70972bbab0 HDR: Clarify tone mapping fallback.
createSupportedTransformationRequest is more accurate than
createFallbackTransformationRequest, as a TransformationRequest will be returned
regardless of whether any fallback is applied.

PiperOrigin-RevId: 466641277
2022-10-19 17:46:15 +00:00
huangdarwin
2d2926b8c0 HDR: Update experimental_setEnableHdrEditing javadoc.
Previously, this feature interpreted SDR signals as HDR when called. Now, only HDR
streams are interpreted as HDR, so the javadoc should be updated. Not yet removing
this method, as there are still some loose ends to finish up (ex. PQ support, e2e
tests).

PiperOrigin-RevId: 466425738
2022-10-19 17:38:55 +00:00
Googler
95a7dcaeda Remove times(1) from effect and transformer tests as it is the default.
PiperOrigin-RevId: 466324322
2022-10-19 17:27:57 +00:00
huangdarwin
7075f78eb2 HDR: Rename ColorInfo#isHdr to isTransferHdr.
While HDR is most closely tied to the color transfer (ex.
COLOR_TRANSFER_SDR is the only one explicitly mentioning dynamic
range), technically color spaces may be associated with HDR as well,
like BT.2020 commonly being used for HDR rather than BT.709 for SDR.

Therefore, it's more specific to mention just that the transfer is HDR.

PiperOrigin-RevId: 466316960
2022-10-19 17:24:19 +00:00
huangdarwin
e444eaa5b7 HDR: Input ColorInfo to the FrameProcessor.
This allows the GlEffectsFrameProcessor to later handle HLG and PQ
differently, or limited and full color range differently.

No functional change intended in this CL.

PiperOrigin-RevId: 466070764
2022-10-19 17:16:56 +00:00
andrewlewis
273cf0866a Remove unused log tag constant
PiperOrigin-RevId: 466025324
2022-10-19 17:09:41 +00:00
andrewlewis
0e99d28a79 Throw if tone-mapping is requested but not enabled
In the case where this check fails, the downstream frame processor chain won't be able to handle the incoming (SDR) data anyway as we've already set it up for HDR.

PiperOrigin-RevId: 465584814
2022-10-19 16:58:47 +00:00
andrewlewis
d90d7d3078 Clarify SSIM request method name
`requestCalculateSsim` more clearly represents the intention of the caller.

Also rephrase the javadoc to simplify it and make it more precise.

PiperOrigin-RevId: 465575578
2022-10-19 16:55:11 +00:00
andrewlewis
aafa31e996 Fix frame processor reference
PiperOrigin-RevId: 465562260
2022-10-19 16:44:08 +00:00
huangdarwin
e63d594f49 HDR: Add comment to clarify lack of support HDR10 for AVC
PiperOrigin-RevId: 465360186
2022-10-19 16:33:21 +00:00
andrewlewis
40a28d5aee Avoid encoder default color info when tone-mapping
The encoder color info [defaults](https://cs.android.com/android/platform/superproject/+/master:frameworks/av/media/libstagefright/foundation/ColorUtils.cpp;l=377;drc=891c19ccfa4953b5e5f7b87118e007b994d8074c) to a value that depends on the input resolution, but when tone-mapping is enabled we should always get BT.709 color space. Hard-code this constant for now to avoid behavior depending on the resolution. A future change should use the decoder output media format to populate the color info.

PiperOrigin-RevId: 465070378
2022-10-19 16:14:44 +00:00
Googler
707b061838 Move effects functionality out of transformer to effects module.
PiperOrigin-RevId: 465038852
2022-10-19 16:00:13 +00:00
leonwind
23f34f8d08 Implement RGBA color matrices processor
* Transform frame colors using a defined RGBA Matrix to apply filters.

PiperOrigin-RevId: 464523581
2022-10-19 15:46:01 +00:00
leonwind
ea213545e0 Migrate Contrast asset to emulator generated one.
PiperOrigin-RevId: 464519030
2022-10-19 15:42:30 +00:00
huangdarwin
1a22bcf1e5 HDR: Have @C.ColorTransfer check consider NO_VALUE as SDR.
Format.NO_VALUE is a placeholder value for an invalid @C.ColorTransfer, used
for example when the decoder doesn't support this transfer function.

When encountering this invalid value, interpret this as COLOR_TRANSFER_SDR.

Confirmed locally that an exception is thrown when transcoding on p4head, and no exception is thrown when transcoding with this CL.

PiperOrigin-RevId: 464135080
2022-10-19 15:38:43 +00:00
Googler
06d41c2775 Move FrameProcessor and related interfaces to common.
This will allow effects preview in ExoPlayer to use the
Effect and FrameProcessor interface (and the interfaces
they depend on) without depending on transformer or the
future effects module.

PiperOrigin-RevId: 464060047
2022-10-19 15:34:54 +00:00
huangdarwin
810f451ec6 Transformer: Improve misc javadocs and exception logs.
PiperOrigin-RevId: 463971447
2022-10-19 15:27:24 +00:00
huangdarwin
44c42fef2a HDR: Centralize getting Colorinfo from MediaFormat
* Sets KEY_HDR_STATIC_INFO from MediaFormat in the DefaultCodec.
* Adds checks in mediaparser to ensure color space, range, and transfer are valid
  values.

PiperOrigin-RevId: 463921325
2022-10-19 15:23:34 +00:00
Googler
bac323d346 Generalize frame processing error codes.
TransformationException error codes were previously tied to OpenGL
but other FrameProcessor implementations are possible. So this CL
renames the error codes.
Also, remove GL_INIT_FAILED error code, as FrameProcessor
implemenations may initialize resources on a background thread
after the factory method returns, so it's not obvious how to
distinguish between initialization failures and processing failures.

PiperOrigin-RevId: 463704902
2022-10-19 15:00:33 +00:00