823 Commits

Author SHA1 Message Date
kimvde
cda969985e Make input media duration volatile
The duration in TransformerInternal.ComponentListener is set on the
Transformer internal thread, and is read on the playback thread. Making
this field volatile ensures that the playback thread reads the updated
value.

PiperOrigin-RevId: 493908385
2022-12-12 11:41:05 +00:00
huangdarwin
c07d503e79 Transformer: Always initialize presentationTime in VTSP.
This shouldn't really have anything to do with HDR.

PiperOrigin-RevId: 493357915
2022-12-12 11:29:00 +00:00
huangdarwin
c37317222c Transformer: Rename error code from output to encoding format.
Rename ERROR_CODE_OUTPUT_FORMAT_UNSUPPORTED to
ERROR_CODE_ENCODING_FORMAT_UNSUPPORTED.

This makes the error code more consistent with ERROR_CODE_DECODING_FAILED on the
decoding side. Also, the error code is in the "Encoding errors (4xxx)" section,
so muxer errors probably should be in the "Muxer errors (7xxx)" section instead.

Additionally, no muxer errors currently seem to use
ERROR_CODE_OUTPUT_FORMAT_UNSUPPORTED, so this should be a safe change.

PiperOrigin-RevId: 493322880
2022-12-12 11:26:43 +00:00
kimvde
0d12de8134 Move slow mo video flattening to AssetLoader
PiperOrigin-RevId: 493300556
2022-12-12 11:25:45 +00:00
kimvde
339205f428 Move progress updates to the AssetLoader
This is necessary to move video slow motion flattening to the
AssetLoader because this step can change the duration. As we use the
duration before flattening to calculate the progress, we must also use
the position before flattening.

PiperOrigin-RevId: 493291990
2022-12-12 11:24:46 +00:00
claincly
cc43ddb528 Re-assign TODOs
PiperOrigin-RevId: 493281221
2022-12-12 11:20:25 +00:00
ibaker
636a4a8538 Add javadoc links to README files
Fix some other link titles and destinations spotted along the way.

#minor-release

PiperOrigin-RevId: 493276172
2022-12-12 11:19:23 +00:00
kimvde
00859a78fd Add comment to explain why cancel exceptions are handled differently
PiperOrigin-RevId: 493260798
2022-12-12 11:17:25 +00:00
huangdarwin
bb2c4c2eff HDR: Add effect interface support for different in/out color transfers
Modify FrameProcessor and MatrixTextureProcessor interfaces to support
different input and output color transfers. Does not implement conversion between
color ranges (ex. HDR and SDR), but should allow for conversion between color
transfers of the same color range (ex. HLG and PQ).

This supports in-app tone mapping, where we need a single FrameProcessor to
input HDR color transfers (ex. HLG/PQ) and output SDR (ex. gamma2.2). This also
supports previewing, where we need a single FrameProcessor to be able to input HLG
and output PQ.

Manually tested by confirming colors still look right on SDR and HDR videos
with a rotation and color affect applied.

PiperOrigin-RevId: 493108678
2022-12-12 11:16:26 +00:00
claincly
02bf7594e2 Change argument order to match API council suggestion
By putting Listener as the last argument.

PiperOrigin-RevId: 493100906
2022-12-12 11:15:23 +00:00
kimvde
7ea74444f3 Avoid sending message on dead thread
PiperOrigin-RevId: 493040714
2022-12-12 11:14:27 +00:00
claincly
900e86ffc8 Allow specifying an Executor in FrameProcessor.Factory.create()
From this CL on, FrameProcessor listeners will be invoked from an Executor that
is passed in when creating the FrameProcessor.

GlTextureProcessor needs to invoke the ErrorListener on the said Executor too.

PiperOrigin-RevId: 493018583
2022-12-12 11:13:30 +00:00
tonihei
634c6161f1 Fix threading of onFallbackApplied callback
The callback is currently triggered on the ExoPlayer playback thread
instead of the app thread that added the listener.

PiperOrigin-RevId: 492474405
2022-12-12 11:08:17 +00:00
kimvde
85c48c481e Make sure that the sample pipeline data is processed regularly.
This is necessary to move video decoding to the AssetLoader. Otherwise,
if the decoder max pending frame count is reached, the AssetLoader will
stop queuing frames to the pipeline, and process data will not be called
anymore.

PiperOrigin-RevId: 492392621
2022-12-12 11:01:36 +00:00
samrobinson
d711bf7b24 Merge feedXFromInput and feedXFromSilence in ATSP.
PiperOrigin-RevId: 492195931
2022-12-12 11:00:31 +00:00
samrobinson
e5727a2cc7 Add an E2E test for changing sample rate with AudioProcessor.
PiperOrigin-RevId: 492160193
2022-12-12 10:54:06 +00:00
samrobinson
0993479efe Add an option to force a silent audio track.
PiperOrigin-RevId: 492154544
2022-12-12 10:53:06 +00:00
claincly
65fe41bc2c Fix Samsung MediaCodec encoder reports incorrect timestmp in EOS
PiperOrigin-RevId: 492023573
2022-12-12 10:48:08 +00:00
huangdarwin
19b4b310fa Transformer: Move getMaxPendingFrameCount logic to constructor.
Also, use the mediaCodecName passed into the constructor to get the
maxPendingFrameCount.

PiperOrigin-RevId: 491985044
2022-12-12 10:46:13 +00:00
claincly
7dee3aec51 Add function to clear the cached list of encoders
EncoderUtil using a static list to cache encoders means it's not possible to
use a different set of encoders for different tests when running all
robolectric tests together.

PiperOrigin-RevId: 491959350
2022-12-12 10:43:03 +00:00
kimvde
eecf7caed0 Move audio decoding to AssetLoader
PiperOrigin-RevId: 491933937
2022-12-12 10:42:04 +00:00
huangdarwin
1aba245a30 HDR: Limit Z Fold 4 c2.qti decoder pending frame count.
Bypass Z Fold 4 HDR10 tone-mapping bug by limiting the max frame count to 12.

This passed with a value of 14, and failed with a value of 15, but I figured I'd use 12 just to be safe.

PiperOrigin-RevId: 491684058
2022-12-12 10:37:55 +00:00
samrobinson
a8e9d158cd Change TransformerInternal durationMs to durationUs.
Follow-up CLs will use this duration for silent audio.

PiperOrigin-RevId: 491670359
2022-11-29 18:53:28 +00:00
claincly
4033013ff8 Move audio MIME type fallback away to ATSP
PiperOrigin-RevId: 491660842
2022-11-29 18:52:17 +00:00
kimvde
09df56f31a Move sample processing to transformer thread
PiperOrigin-RevId: 491623586
2022-11-29 18:49:15 +00:00
claincly
c1ef48f20d Move video encoding MIME type fallback to VTSP
Main change:

- Removed `Codec.EncoderFactory.createForVideoEncoding`'s argument of a list
of allowed MIME types
- Moved the check for whether a video MIME type is supported to VTSP

PiperOrigin-RevId: 491611799
2022-11-29 18:46:55 +00:00
claincly
0f85ce5dbf Fix VideoEncoderSetting.bitrate is ignored when set
PiperOrigin-RevId: 491377695
2022-11-29 18:44:24 +00:00
kimvde
c9b9054c0b Add Transformer internal thread
This thread just starts the player and handles the player callbacks for
now. Sample pipelines are still run on the playback thread.

PiperOrigin-RevId: 491299671
2022-11-29 18:38:22 +00:00
samrobinson
dd96c97bd5 Bump muxer watchdog abort timer default timeout to 10s.
PiperOrigin-RevId: 490756055
2022-11-29 18:27:45 +00:00
kimvde
e12dd1ac5d Surface player release timeouts
Also make sure the demo app doesn't crash when it happens.

PiperOrigin-RevId: 490725959
2022-11-24 15:00:34 +00:00
huangdarwin
7d45b9d83a Tests: Set SSIM decoder KEY_PRIORITY to 1.
This reduces the priority to best effort (from the default that seems to be best effort), and allows us to run SSIM even on 8k24fps video. Without this CL,
start()'ing a second codec may result in a MediaCodec.CodecException.

Tested to confirm that transformation8k24():
* fails deterministically without this CL, or with KEY_PRIORITY set to 0.
* succeeds deterministically after this CL (~18s on Samsung Z Fold 4)

PiperOrigin-RevId: 490570416
2022-11-24 14:52:09 +00:00
huangdarwin
d74eb7500f Transformer: Document when OMX video codecs will no longer exist.
We cannot check this in code, due to DEVICE_INITIAL_SDK_INT being a @SystemApi, and
reflection being a bit risky/unstable.

PiperOrigin-RevId: 490537916
2022-11-24 14:51:06 +00:00
huangdarwin
2693ca138d Transformer: Remove obsolete TODO from TransformationException.
The bug has since been fixed.

The values still could change, as the API is labelled as @UnstableApi, so it's
probably fine to leave the <p> tag mostly as is.

PiperOrigin-RevId: 490509205
2022-11-24 14:47:37 +00:00
huangdarwin
0a176d1f05 Tests: Move codec configure() out of VideoDecodingWrapper constructor.
This allows us to release both codecs used in SSIM when one fails to
configure() or start().

Tested and confirmed that on Samsung Galaxy Z Flip 4, running all
TransformationTest.java tests, tests after transform8k24() fails to start the
2nd codec:
* Before this CL, all fail.
* After this CL, all pass.

PiperOrigin-RevId: 490461560
2022-11-24 14:40:47 +00:00
samrobinson
af1d7b9b1b Fix SpeedChangingAudioProcessor handling of EMPTY_BUFFER.
PiperOrigin-RevId: 490265564
2022-11-24 14:36:14 +00:00
huangdarwin
4d50e0221e Tests: Throw exception when network is needed but not available.
This exception is a bit shorter and more clear (and is more clear that this is a
test issue, as opposed to the prior issue that was thrown as an ExoPlayer
DataSourceException, which may seem like a legitimate Transformer failure)

PiperOrigin-RevId: 490252772
2022-11-22 17:43:08 +00:00
huangdarwin
d0c5168902 HDR Tests: Rename remote URIs to REMOTE instead of ASSET.
Following naming conventions throughout AndroidTestUtil, REMOTE files should have REMOTE instead of ASSET. Update the URI and FORMAT names accordingly.

PiperOrigin-RevId: 490237673
2022-11-22 17:42:01 +00:00
kimvde
d8eec16369 Refactor progress logic to be thread safe
PiperOrigin-RevId: 489984147
2022-11-22 10:10:21 +00:00
kimvde
56cf5e0774 Build TransformationResult only when transformation succeeded
PiperOrigin-RevId: 489442518
2022-11-22 10:04:56 +00:00
kimvde
f658fe79b1 Make sure Muxer is always accessed from playback thread
This thread will become the Transformer internal thread.

PiperOrigin-RevId: 489168435
2022-11-22 09:50:13 +00:00
kimvde
1ee185cb10 Refactor transformation completion
PiperOrigin-RevId: 488929446
2022-11-22 09:44:47 +00:00
huangdarwin
53cff556bf HDR: Use HdrMode IntDef to select HDR output mode.
This is a simpler API to use than the prior tone mapping, HDR editing, and
forceInterpretHdrAsSdr options

PiperOrigin-RevId: 488742391
2022-11-16 12:16:21 +00:00
samrobinson
7b2547271a Utilize AudioProcessingPipeline in Transformer.
Provides an API for applications to set AudioProcessors for use
in Transformer.

PiperOrigin-RevId: 488621242
2022-11-16 12:11:47 +00:00
kimvde
66dc6b3242 Move progress computation to TransformerInternal
AssetLoader will have multiple implementations and be customizable. We
want to remove the responsibility of computing the progress from this
class and centralize the logic in TransformerInternal.

PiperOrigin-RevId: 488608890
2022-11-16 12:09:03 +00:00
huangdarwin
6820704b08 HDR: For HDR videos, enableHdrEditing by default.
Previously, tone-mapping was the default.

PiperOrigin-RevId: 488362209
2022-11-14 17:54:51 +00:00
huangdarwin
0cd1edd834 HDR: Add public HLG test file.
Add an HLG Pixel 7 Pro video and its associated format for testing.

PiperOrigin-RevId: 488353926
2022-11-14 17:52:37 +00:00
kimvde
e1a9ce71df Remove asyncErrorListener
PiperOrigin-RevId: 488310077
2022-11-14 17:51:36 +00:00
samrobinson
83b4d00081 Remove checkNotNull from Transformer build method.
PiperOrigin-RevId: 487811903
2022-11-14 17:50:29 +00:00
huangdarwin
b6d49ec59b HDR: Require transcoding when forceInterpretHdrVideoAsSdr is set.
PiperOrigin-RevId: 487786326
2022-11-14 17:49:28 +00:00
kimvde
c9fe52bf67 Fix bug where streamOffsetUs is passed instead of streamPositionUs
Also make order of streamStartPositionUs and streamOffsetUs consistent

PiperOrigin-RevId: 487511633
2022-11-10 15:01:18 +00:00