448 Commits

Author SHA1 Message Date
huangdarwin
8548e3519e HDR: Disable tone mapping on unsupported pixel build ID.
Also, update tests to allow AnyOf error codes, and no longer check exception messages, which caused quite a bit of churn.

PiperOrigin-RevId: 479570861
2022-10-17 15:38:30 +00:00
huangdarwin
253486d4af HDR: Check isToneMappingApplied in HDR editing test.
Assert that tone mapping is applied when an HDR edit cannot be HDR, but is successfully tone mapped. Meanwhile, assert that fallback, which is applied after codec configuration (which throws the "Tone-mapping requested but not supported by the decoder" error) is not applied when that error is called.

PiperOrigin-RevId: 478762951
2022-10-17 15:28:24 +00:00
huangdarwin
6e5a4c05b5 HDR: Add tests with HLG10 input.
PiperOrigin-RevId: 478019046
2022-10-17 15:25:17 +00:00
huangdarwin
62b95a7190 Test: Rename to avoid redundant substrings.
Rename test files to avoid substrings that can be implied by the directory name,
like "Transformation" and "Test"

No functional changes. Renaming-only.

PiperOrigin-RevId: 477724724
2022-09-30 18:31:56 +00:00
huangdarwin
1711d7f4c2 HDR: Add period to end of error message.
For tone mapping error messages.

PiperOrigin-RevId: 477447799
2022-09-30 18:30:07 +00:00
huangdarwin
aec4fe7387 HDR: Add tests for tone mapping.
Also, add checks for output file color.

PiperOrigin-RevId: 477439139
2022-09-30 18:27:18 +00:00
huangdarwin
87fd51a39b HDR: Assert output C.ColorTransfer for tests.
To confirm that tone mapping did or did not happen.

PiperOrigin-RevId: 476354606
2022-09-30 18:10:25 +00:00
leonwind
3433758c3b Add OETF and EOTF ES2 fragment shaders for non-HDR frames.
* Transform the intermediate color space to linear SDR by applying the SMPTE 170M EOTF and OETF.
* Use linear colors for the color filter pixel tests and update all golden bitmaps.

PiperOrigin-RevId: 476124592
2022-09-30 18:08:30 +00:00
samrobinson
9ec4e1340a Add file size to TransformationResult at the end of a Transformation.
Tested:
  * Manually using both path and file descriptor.
PiperOrigin-RevId: 475860978
2022-09-30 18:05:19 +00:00
samrobinson
314b312af1 Add support for skipping windows in MssimCalculator.
By skipping every other row and column, SSIM calculation time reduces by 10-30%.

PiperOrigin-RevId: 474286702
2022-09-30 17:58:04 +00:00
samrobinson
8ce42f0670 Add MssimCalculatorTest to verify SSIM calculations.
As part of this change, MssimCalculator is moved from androidTest/ to main/

PiperOrigin-RevId: 473771344
2022-09-30 17:54:22 +00:00
samrobinson
d188babd2d Rollback of 3d5ddf0c42
*** Original commit ***

Handle int instead of byte in SSIM.

The value of pixels are converted to integers at the point of use,
move this logic to the initialisation step.

This is a prerequisite step for testing SSIM calculation, which
will lead on to some SSIM improvements being verifiable.

Tested manually and SSIM values match for the same video
before and after this change.

***

PiperOrigin-RevId: 473259446
2022-09-30 17:49:31 +00:00
samrobinson
3d5ddf0c42 Handle int instead of byte in SSIM.
The value of pixels are converted to integers at the point of use,
move this logic to the initialisation step.

This is a prerequisite step for testing SSIM calculation, which
will lead on to some SSIM improvements being verifiable.

Tested manually and SSIM values match for the same video
before and after this change.

PiperOrigin-RevId: 473231779
2022-09-30 17:48:34 +00:00
huangdarwin
4133bb6070 Decoder: Reduce max pending frame count limit down to 5.
PiperOrigin-RevId: 473212543
2022-09-30 17:46:40 +00:00
huangdarwin
0d8fd3d4fc HDR: Add fallback MH tests.
Test that HDR editing succeeds on devices supporting HDR editing, tone maps on
devices supporting tone mapping, and throws exceptions on all other devices.

Also, only restrict HDR editing and tone mapping support to API 31+ only when
transcoding, not for all transformations.

PiperOrigin-RevId: 472958965
2022-09-30 17:42:50 +00:00
claincly
8b7638bed3 Remove conditional check for no-longer flaky API version
PiperOrigin-RevId: 472949850
2022-09-30 17:41:54 +00:00
claincly
c401fb9771 Remove redundant constant definition
PiperOrigin-RevId: 472522846
2022-09-30 17:36:33 +00:00
claincly
3f6a59f0f7 Include CamcorderProfile resolution in encoder capability test
PiperOrigin-RevId: 472459423
2022-09-30 17:31:35 +00:00
huangdarwin
fa1f09fc51 Minor javadoc and scoping cleanup.
No functional changes.

PiperOrigin-RevId: 472245797
2022-09-30 17:24:54 +00:00
huangdarwin
7085c2fa6e Rename shouldPassthrough to shouldTranscode
shouldPassthrough's internal checks seem to be check whether we should *not*
pass through, which seemed a bit like a confusing double-negative to me.

shouldTranscode is slightly more clear, by instead returning true when we do
want to transcode.

No functional changes intended.

PiperOrigin-RevId: 471753771
2022-09-30 17:21:28 +00:00
Googler
bc88f8be54 Skip transcoding if HDR video does not need encoding.
3b0d2c1586 made `shouldPassthrough` always return false for `enableHdrVideoEditing`:

>We force using `FrameEditor` (no passthrough) to avoid the need to select another edit operation, and use the new shaders. The `EGLContext` and `EGLSurface` also need to be set up differently for this path.

However, this was introduced before the `videoNeedsEncoding` setting was introduced in 3f615040c0. That setting should apply to HDR videos as much as SDR videos.

PiperOrigin-RevId: 471569853
2022-09-30 17:20:32 +00:00
ibaker
932f0d2248 Remove media3-only line from exoplayer2 build.gradle file
#minor-release

PiperOrigin-RevId: 470999044
2022-09-30 17:10:20 +00:00
andrewlewis
daf1e5e2eb Update color info mismatch test
This should now expect transformation to succeed.

PiperOrigin-RevId: 470950411
2022-09-30 17:08:22 +00:00
andrewlewis
dbe6677551 Log instead of throwing for transfer mismatch
PiperOrigin-RevId: 470354448
2022-09-30 17:07:23 +00:00
Googler
0b1c540ff9 Change onOutputFrameAvailable timestamp from nanos to micros.
Upstream timestamps from the decoder are also in microseconds,
so using microseconds here is consistent with that.

PiperOrigin-RevId: 468659099
2022-09-30 16:51:14 +00:00
Googler
a5d7fdcab5 Allow frame release to be controlled outside FrameProcessor.
Adds a method to FrameProcessor.Listener to be called when an
output frame is available and a method releaseOutputFrame in
FrameProcessor allowing the caller to trigger release of the
oldest available output frame at a given timestamp. Late frames
or frames with unset release times are dropped in the
FinalMatrixTransformationProcessorWrapper.

More than one output frame can become available before they are
released if the penultimate GlTextureProcessor is capable of producing
multiple output frames. Processing continues while waiting for
releaseOutputFrame to be called. Frame release tasks are prioritized
over other tasks.

PiperOrigin-RevId: 468473072
2022-09-30 16:49:21 +00:00
huangdarwin
c5a5364673 HDR: Clarify tone mapping fallback.
createSupportedTransformationRequest is more accurate than
createFallbackTransformationRequest, as a TransformationRequest will be returned
regardless of whether any fallback is applied.

PiperOrigin-RevId: 466641277
2022-09-30 16:29:27 +00:00
huangdarwin
05fcda510a HDR: Update experimental_setEnableHdrEditing javadoc.
Previously, this feature interpreted SDR signals as HDR when called. Now, only HDR
streams are interpreted as HDR, so the javadoc should be updated. Not yet removing
this method, as there are still some loose ends to finish up (ex. PQ support, e2e
tests).

PiperOrigin-RevId: 466425738
2022-09-30 16:27:27 +00:00
Googler
1bdc252d2c Remove times(1) from effect and transformer tests as it is the default.
PiperOrigin-RevId: 466324322
2022-09-30 16:24:11 +00:00
huangdarwin
35161c7489 HDR: Rename ColorInfo#isHdr to isTransferHdr.
While HDR is most closely tied to the color transfer (ex.
COLOR_TRANSFER_SDR is the only one explicitly mentioning dynamic
range), technically color spaces may be associated with HDR as well,
like BT.2020 commonly being used for HDR rather than BT.709 for SDR.

Therefore, it's more specific to mention just that the transfer is HDR.

PiperOrigin-RevId: 466316960
2022-09-30 16:23:12 +00:00
huangdarwin
536d42c865 HDR: Input ColorInfo to the FrameProcessor.
This allows the GlEffectsFrameProcessor to later handle HLG and PQ
differently, or limited and full color range differently.

No functional change intended in this CL.

PiperOrigin-RevId: 466070764
2022-09-30 16:21:05 +00:00
andrewlewis
2315a39f52 Remove unused log tag constant
PiperOrigin-RevId: 466025324
2022-09-30 16:19:02 +00:00
andrewlewis
2bd1a93d27 Throw if tone-mapping is requested but not enabled
In the case where this check fails, the downstream frame processor chain won't be able to handle the incoming (SDR) data anyway as we've already set it up for HDR.

PiperOrigin-RevId: 465584814
2022-08-05 16:49:26 +00:00
andrewlewis
2eb59fa1af Clarify SSIM request method name
`requestCalculateSsim` more clearly represents the intention of the caller.

Also rephrase the javadoc to simplify it and make it more precise.

PiperOrigin-RevId: 465575578
2022-08-05 16:02:07 +00:00
andrewlewis
16a325217a Fix frame processor reference
PiperOrigin-RevId: 465562260
2022-08-05 14:49:23 +00:00
huangdarwin
348662283a HDR: Add comment to clarify lack of support HDR10 for AVC
PiperOrigin-RevId: 465360186
2022-08-04 18:18:18 +00:00
andrewlewis
3c4ee51e10 Avoid encoder default color info when tone-mapping
The encoder color info [defaults](https://cs.android.com/android/platform/superproject/+/master:frameworks/av/media/libstagefright/foundation/ColorUtils.cpp;l=377;drc=891c19ccfa4953b5e5f7b87118e007b994d8074c) to a value that depends on the input resolution, but when tone-mapping is enabled we should always get BT.709 color space. Hard-code this constant for now to avoid behavior depending on the resolution. A future change should use the decoder output media format to populate the color info.

PiperOrigin-RevId: 465070378
2022-08-03 16:06:45 +00:00
hschlueter
22725ddfa8 Move effects functionality out of transformer to effects module.
PiperOrigin-RevId: 465038852
2022-08-03 13:19:15 +00:00
leonwind
d15f2ed388 Implement RGBA color matrices processor
* Transform frame colors using a defined RGBA Matrix to apply filters.

PiperOrigin-RevId: 464523581
2022-08-08 08:36:03 +00:00
leonwind
ae1915ea1a Migrate Contrast asset to emulator generated one.
PiperOrigin-RevId: 464519030
2022-08-08 08:35:12 +00:00
huangdarwin
687a50e9b3 HDR: Have @C.ColorTransfer check consider NO_VALUE as SDR.
Format.NO_VALUE is a placeholder value for an invalid @C.ColorTransfer, used
for example when the decoder doesn't support this transfer function.

When encountering this invalid value, interpret this as COLOR_TRANSFER_SDR.

Confirmed locally that an exception is thrown when transcoding on p4head, and no exception is thrown when transcoding with this CL.

PiperOrigin-RevId: 464135080
2022-08-08 08:34:12 +00:00
hschlueter
480c129c54 Move FrameProcessor and related interfaces to common.
This will allow effects preview in ExoPlayer to use the
Effect and FrameProcessor interface (and the interfaces
they depend on) without depending on transformer or the
future effects module.

PiperOrigin-RevId: 464060047
2022-08-08 08:33:22 +00:00
huangdarwin
534740fd46 Transformer: Improve misc javadocs and exception logs.
PiperOrigin-RevId: 463971447
2022-08-08 08:26:35 +00:00
huangdarwin
65a2a3a0a0 HDR: Centralize getting Colorinfo from MediaFormat
* Sets KEY_HDR_STATIC_INFO from MediaFormat in the DefaultCodec.
* Adds checks in mediaparser to ensure color space, range, and transfer are valid
  values.

PiperOrigin-RevId: 463921325
2022-08-08 08:24:51 +00:00
hschlueter
0f96d8615d Generalize frame processing error codes.
TransformationException error codes were previously tied to OpenGL
but other FrameProcessor implementations are possible. So this CL
renames the error codes.
Also, remove GL_INIT_FAILED error code, as FrameProcessor
implemenations may initialize resources on a background thread
after the factory method returns, so it's not obvious how to
distinguish between initialization failures and processing failures.

PiperOrigin-RevId: 463704902
2022-08-08 08:17:46 +00:00
hschlueter
580e44fc47 Add Effect marker interface.
This allows non-GL effects to be passed to custom FrameProcessor
implementations.

PiperOrigin-RevId: 463696384
2022-08-08 08:16:50 +00:00
hschlueter
b28fb13998 Remove redundant interface from FinalMatrixTPWrapper.
PiperOrigin-RevId: 463646211
2022-08-08 08:15:55 +00:00
hschlueter
b1b9ba825f Make FrameInfo and SurfaceInfo public.
Both are used in the public FrameProcessor interface, so they
should be public too.

PiperOrigin-RevId: 463454859
2022-08-08 08:14:59 +00:00
samrobinson
56e50b63ec Add 8 new videos, representing lower resolutions.
Downsampled from 3840x2160 resolution videos using ffmpeg:

`ffmpeg -i <inputFilename> -filter:v scale=<width>:<height> -c:a copy <outputFilename>`

PiperOrigin-RevId: 463454628
2022-08-08 08:14:03 +00:00
samrobinson
60d6654b8d Add H265 as a parameter to SsimMapperTest.
PiperOrigin-RevId: 463452893
2022-08-08 08:13:05 +00:00