659 Commits

Author SHA1 Message Date
samrobinson
bcc2fe4007 Disable the muxer watchdog if all tracks have ended.
PiperOrigin-RevId: 484512661
2022-10-31 11:47:54 +00:00
kimvde
1730334133 Add test for muxer watchdog timer
PiperOrigin-RevId: 484298261
2022-10-31 11:45:55 +00:00
kimvde
16cb5cbc1f Add muxer timer to detect when generating an output sample is too slow
This allows to throw when the Transformer is stuck or is too slow.

PiperOrigin-RevId: 484179037
2022-10-31 11:39:28 +00:00
samrobinson
fc34542864 Move AudioProcessor to common.
PiperOrigin-RevId: 483699606
2022-10-31 11:34:30 +00:00
samrobinson
8723e74b1f Add CanIgnoreReturnValue to AudioProcessor#configure + implementations
Although it can be useful to check the output format, it's not required or needed.

For some AudioProcessor implementations, it is stated/obvious that
the output format will match the input, in which case there is no
a need to check the return value.

#cleanup

PiperOrigin-RevId: 483403679
2022-10-31 11:29:28 +00:00
kimvde
e6d5595545 Remove muxerFactory dependency on playerListener
This listener will need to be passed to the MuxerWrapper to throw when
the Transformer is stuck.

PiperOrigin-RevId: 482433552
2022-10-24 10:46:26 +00:00
kimvde
cf14d0687d Make Muxer public
The reason for making the Muxer public is that we want to add an option
to disable or configure the timer that will throw when the muxer doesn't
receive any data for a given period of time.

PiperOrigin-RevId: 482199360
2022-10-24 10:42:42 +00:00
kimvde
9221b1251c Remove deprecated setOutputMimeType
This is to prepare Muxer to become public

PiperOrigin-RevId: 481893842
2022-10-24 10:32:43 +00:00
samrobinson
7ac7e643ac Fix parameter comment block.
#cleanup

PiperOrigin-RevId: 481882181
2022-10-24 10:31:37 +00:00
kimvde
3399f4ecdf Add DefaultMuxer forwarding to FrameworkMuxer
- The naming DefaultMuxer is more consistent with the rest of
Transformer codebase (e.g. DefaultEncoderFactory).
- By hiding the implementation details of DefaultMuxer, the transition
to in-app Muxer will be seamless for apps using DefaultMuxer.
- The current plan is that DefaultMuxer will become the in-app muxer.

PiperOrigin-RevId: 481838790
2022-10-24 10:29:27 +00:00
kimvde
2786db9e73 Remove supportsSampleMimeType from Muxer.Factory
- This method is redundant with getSupportedSampleMimeTypes().
- This is to prepare the Muxer class to become public.

PiperOrigin-RevId: 480840902
2022-10-17 15:54:39 +00:00
huangdarwin
8548e3519e HDR: Disable tone mapping on unsupported pixel build ID.
Also, update tests to allow AnyOf error codes, and no longer check exception messages, which caused quite a bit of churn.

PiperOrigin-RevId: 479570861
2022-10-17 15:38:30 +00:00
huangdarwin
253486d4af HDR: Check isToneMappingApplied in HDR editing test.
Assert that tone mapping is applied when an HDR edit cannot be HDR, but is successfully tone mapped. Meanwhile, assert that fallback, which is applied after codec configuration (which throws the "Tone-mapping requested but not supported by the decoder" error) is not applied when that error is called.

PiperOrigin-RevId: 478762951
2022-10-17 15:28:24 +00:00
huangdarwin
6e5a4c05b5 HDR: Add tests with HLG10 input.
PiperOrigin-RevId: 478019046
2022-10-17 15:25:17 +00:00
huangdarwin
62b95a7190 Test: Rename to avoid redundant substrings.
Rename test files to avoid substrings that can be implied by the directory name,
like "Transformation" and "Test"

No functional changes. Renaming-only.

PiperOrigin-RevId: 477724724
2022-09-30 18:31:56 +00:00
huangdarwin
1711d7f4c2 HDR: Add period to end of error message.
For tone mapping error messages.

PiperOrigin-RevId: 477447799
2022-09-30 18:30:07 +00:00
huangdarwin
aec4fe7387 HDR: Add tests for tone mapping.
Also, add checks for output file color.

PiperOrigin-RevId: 477439139
2022-09-30 18:27:18 +00:00
huangdarwin
87fd51a39b HDR: Assert output C.ColorTransfer for tests.
To confirm that tone mapping did or did not happen.

PiperOrigin-RevId: 476354606
2022-09-30 18:10:25 +00:00
leonwind
3433758c3b Add OETF and EOTF ES2 fragment shaders for non-HDR frames.
* Transform the intermediate color space to linear SDR by applying the SMPTE 170M EOTF and OETF.
* Use linear colors for the color filter pixel tests and update all golden bitmaps.

PiperOrigin-RevId: 476124592
2022-09-30 18:08:30 +00:00
samrobinson
9ec4e1340a Add file size to TransformationResult at the end of a Transformation.
Tested:
  * Manually using both path and file descriptor.
PiperOrigin-RevId: 475860978
2022-09-30 18:05:19 +00:00
samrobinson
314b312af1 Add support for skipping windows in MssimCalculator.
By skipping every other row and column, SSIM calculation time reduces by 10-30%.

PiperOrigin-RevId: 474286702
2022-09-30 17:58:04 +00:00
samrobinson
8ce42f0670 Add MssimCalculatorTest to verify SSIM calculations.
As part of this change, MssimCalculator is moved from androidTest/ to main/

PiperOrigin-RevId: 473771344
2022-09-30 17:54:22 +00:00
samrobinson
d188babd2d Rollback of 3d5ddf0c42
*** Original commit ***

Handle int instead of byte in SSIM.

The value of pixels are converted to integers at the point of use,
move this logic to the initialisation step.

This is a prerequisite step for testing SSIM calculation, which
will lead on to some SSIM improvements being verifiable.

Tested manually and SSIM values match for the same video
before and after this change.

***

PiperOrigin-RevId: 473259446
2022-09-30 17:49:31 +00:00
samrobinson
3d5ddf0c42 Handle int instead of byte in SSIM.
The value of pixels are converted to integers at the point of use,
move this logic to the initialisation step.

This is a prerequisite step for testing SSIM calculation, which
will lead on to some SSIM improvements being verifiable.

Tested manually and SSIM values match for the same video
before and after this change.

PiperOrigin-RevId: 473231779
2022-09-30 17:48:34 +00:00
huangdarwin
4133bb6070 Decoder: Reduce max pending frame count limit down to 5.
PiperOrigin-RevId: 473212543
2022-09-30 17:46:40 +00:00
huangdarwin
0d8fd3d4fc HDR: Add fallback MH tests.
Test that HDR editing succeeds on devices supporting HDR editing, tone maps on
devices supporting tone mapping, and throws exceptions on all other devices.

Also, only restrict HDR editing and tone mapping support to API 31+ only when
transcoding, not for all transformations.

PiperOrigin-RevId: 472958965
2022-09-30 17:42:50 +00:00
claincly
8b7638bed3 Remove conditional check for no-longer flaky API version
PiperOrigin-RevId: 472949850
2022-09-30 17:41:54 +00:00
claincly
c401fb9771 Remove redundant constant definition
PiperOrigin-RevId: 472522846
2022-09-30 17:36:33 +00:00
claincly
3f6a59f0f7 Include CamcorderProfile resolution in encoder capability test
PiperOrigin-RevId: 472459423
2022-09-30 17:31:35 +00:00
huangdarwin
fa1f09fc51 Minor javadoc and scoping cleanup.
No functional changes.

PiperOrigin-RevId: 472245797
2022-09-30 17:24:54 +00:00
huangdarwin
7085c2fa6e Rename shouldPassthrough to shouldTranscode
shouldPassthrough's internal checks seem to be check whether we should *not*
pass through, which seemed a bit like a confusing double-negative to me.

shouldTranscode is slightly more clear, by instead returning true when we do
want to transcode.

No functional changes intended.

PiperOrigin-RevId: 471753771
2022-09-30 17:21:28 +00:00
Googler
bc88f8be54 Skip transcoding if HDR video does not need encoding.
3b0d2c1586 made `shouldPassthrough` always return false for `enableHdrVideoEditing`:

>We force using `FrameEditor` (no passthrough) to avoid the need to select another edit operation, and use the new shaders. The `EGLContext` and `EGLSurface` also need to be set up differently for this path.

However, this was introduced before the `videoNeedsEncoding` setting was introduced in 3f615040c0. That setting should apply to HDR videos as much as SDR videos.

PiperOrigin-RevId: 471569853
2022-09-30 17:20:32 +00:00
ibaker
932f0d2248 Remove media3-only line from exoplayer2 build.gradle file
#minor-release

PiperOrigin-RevId: 470999044
2022-09-30 17:10:20 +00:00
andrewlewis
daf1e5e2eb Update color info mismatch test
This should now expect transformation to succeed.

PiperOrigin-RevId: 470950411
2022-09-30 17:08:22 +00:00
andrewlewis
dbe6677551 Log instead of throwing for transfer mismatch
PiperOrigin-RevId: 470354448
2022-09-30 17:07:23 +00:00
Googler
0b1c540ff9 Change onOutputFrameAvailable timestamp from nanos to micros.
Upstream timestamps from the decoder are also in microseconds,
so using microseconds here is consistent with that.

PiperOrigin-RevId: 468659099
2022-09-30 16:51:14 +00:00
Googler
a5d7fdcab5 Allow frame release to be controlled outside FrameProcessor.
Adds a method to FrameProcessor.Listener to be called when an
output frame is available and a method releaseOutputFrame in
FrameProcessor allowing the caller to trigger release of the
oldest available output frame at a given timestamp. Late frames
or frames with unset release times are dropped in the
FinalMatrixTransformationProcessorWrapper.

More than one output frame can become available before they are
released if the penultimate GlTextureProcessor is capable of producing
multiple output frames. Processing continues while waiting for
releaseOutputFrame to be called. Frame release tasks are prioritized
over other tasks.

PiperOrigin-RevId: 468473072
2022-09-30 16:49:21 +00:00
huangdarwin
c5a5364673 HDR: Clarify tone mapping fallback.
createSupportedTransformationRequest is more accurate than
createFallbackTransformationRequest, as a TransformationRequest will be returned
regardless of whether any fallback is applied.

PiperOrigin-RevId: 466641277
2022-09-30 16:29:27 +00:00
huangdarwin
05fcda510a HDR: Update experimental_setEnableHdrEditing javadoc.
Previously, this feature interpreted SDR signals as HDR when called. Now, only HDR
streams are interpreted as HDR, so the javadoc should be updated. Not yet removing
this method, as there are still some loose ends to finish up (ex. PQ support, e2e
tests).

PiperOrigin-RevId: 466425738
2022-09-30 16:27:27 +00:00
Googler
1bdc252d2c Remove times(1) from effect and transformer tests as it is the default.
PiperOrigin-RevId: 466324322
2022-09-30 16:24:11 +00:00
huangdarwin
35161c7489 HDR: Rename ColorInfo#isHdr to isTransferHdr.
While HDR is most closely tied to the color transfer (ex.
COLOR_TRANSFER_SDR is the only one explicitly mentioning dynamic
range), technically color spaces may be associated with HDR as well,
like BT.2020 commonly being used for HDR rather than BT.709 for SDR.

Therefore, it's more specific to mention just that the transfer is HDR.

PiperOrigin-RevId: 466316960
2022-09-30 16:23:12 +00:00
huangdarwin
536d42c865 HDR: Input ColorInfo to the FrameProcessor.
This allows the GlEffectsFrameProcessor to later handle HLG and PQ
differently, or limited and full color range differently.

No functional change intended in this CL.

PiperOrigin-RevId: 466070764
2022-09-30 16:21:05 +00:00
andrewlewis
2315a39f52 Remove unused log tag constant
PiperOrigin-RevId: 466025324
2022-09-30 16:19:02 +00:00
andrewlewis
2bd1a93d27 Throw if tone-mapping is requested but not enabled
In the case where this check fails, the downstream frame processor chain won't be able to handle the incoming (SDR) data anyway as we've already set it up for HDR.

PiperOrigin-RevId: 465584814
2022-08-05 16:49:26 +00:00
andrewlewis
2eb59fa1af Clarify SSIM request method name
`requestCalculateSsim` more clearly represents the intention of the caller.

Also rephrase the javadoc to simplify it and make it more precise.

PiperOrigin-RevId: 465575578
2022-08-05 16:02:07 +00:00
andrewlewis
16a325217a Fix frame processor reference
PiperOrigin-RevId: 465562260
2022-08-05 14:49:23 +00:00
huangdarwin
348662283a HDR: Add comment to clarify lack of support HDR10 for AVC
PiperOrigin-RevId: 465360186
2022-08-04 18:18:18 +00:00
andrewlewis
3c4ee51e10 Avoid encoder default color info when tone-mapping
The encoder color info [defaults](https://cs.android.com/android/platform/superproject/+/master:frameworks/av/media/libstagefright/foundation/ColorUtils.cpp;l=377;drc=891c19ccfa4953b5e5f7b87118e007b994d8074c) to a value that depends on the input resolution, but when tone-mapping is enabled we should always get BT.709 color space. Hard-code this constant for now to avoid behavior depending on the resolution. A future change should use the decoder output media format to populate the color info.

PiperOrigin-RevId: 465070378
2022-08-03 16:06:45 +00:00
hschlueter
22725ddfa8 Move effects functionality out of transformer to effects module.
PiperOrigin-RevId: 465038852
2022-08-03 13:19:15 +00:00
leonwind
d15f2ed388 Implement RGBA color matrices processor
* Transform frame colors using a defined RGBA Matrix to apply filters.

PiperOrigin-RevId: 464523581
2022-08-08 08:36:03 +00:00