833 Commits

Author SHA1 Message Date
claincly
b3571b71b5 Make Codec release frames to frame processor.
PiperOrigin-RevId: 495406734
2022-12-15 16:13:00 +00:00
kimvde
21ae403049 Pass CapturingDecoderFactory to AssetLoader
Otherwise, the decoders are not captured. It works at the moment for the
video decoder because decoding is still done on the sample pipeline but
it will moved to the AssetLoader soon.

PiperOrigin-RevId: 495275575
2022-12-15 16:08:08 +00:00
andrewlewis
f2cf952f68 Output direct buffers from SilentAudioGenerator
`AudioProcessor`s expect direct buffers. This shouldn't make any functional difference in our code, but a custom audio processor might try to access the buffer from JNI in which case a direct byte buffer is more efficient.

PiperOrigin-RevId: 495241669
2022-12-15 16:06:15 +00:00
andrewlewis
3060b97180 Take format for SilentAudioGenerator
This simplifies the caller slightly.

PiperOrigin-RevId: 495234339
2022-12-15 16:05:17 +00:00
claincly
b1425aa6f4 Fix method name to return all encode-able MIME types
PiperOrigin-RevId: 495093939
2022-12-15 16:04:28 +00:00
kimvde
5759eda37e Add AssetLoader.Factory
This is so that apps can customise AssetLoader

PiperOrigin-RevId: 494998497
2022-12-15 16:00:55 +00:00
andrewlewis
beee4732fb Generate complete silent audio frames
`SilentAudioGenerator` could output a fractional audio frame, and this could cause downstream components to throw because of trying to read a complete audio frame but only seeing a partial one.

Calculate the output buffer size based on the frame size (which is a no-op for stereo 16-bit audio) and calculate a total number of frames to output then multiple by the frame size.

PiperOrigin-RevId: 494992941
2022-12-15 15:58:50 +00:00
samrobinson
6c7e892d2f Add codec names to TransformationResult
PiperOrigin-RevId: 494736085
2022-12-15 15:56:02 +00:00
samrobinson
36e52691bb Move SetFrameEditTest into TransformationTest.
PiperOrigin-RevId: 494651985
2022-12-12 11:45:45 +00:00
kimvde
c8d8ff5578 Add AssetLoader interface
This is a step towards allowing apps to inject a custom AssetLoader

PiperOrigin-RevId: 494185078
2022-12-12 11:44:50 +00:00
kimvde
cda969985e Make input media duration volatile
The duration in TransformerInternal.ComponentListener is set on the
Transformer internal thread, and is read on the playback thread. Making
this field volatile ensures that the playback thread reads the updated
value.

PiperOrigin-RevId: 493908385
2022-12-12 11:41:05 +00:00
huangdarwin
c07d503e79 Transformer: Always initialize presentationTime in VTSP.
This shouldn't really have anything to do with HDR.

PiperOrigin-RevId: 493357915
2022-12-12 11:29:00 +00:00
huangdarwin
c37317222c Transformer: Rename error code from output to encoding format.
Rename ERROR_CODE_OUTPUT_FORMAT_UNSUPPORTED to
ERROR_CODE_ENCODING_FORMAT_UNSUPPORTED.

This makes the error code more consistent with ERROR_CODE_DECODING_FAILED on the
decoding side. Also, the error code is in the "Encoding errors (4xxx)" section,
so muxer errors probably should be in the "Muxer errors (7xxx)" section instead.

Additionally, no muxer errors currently seem to use
ERROR_CODE_OUTPUT_FORMAT_UNSUPPORTED, so this should be a safe change.

PiperOrigin-RevId: 493322880
2022-12-12 11:26:43 +00:00
kimvde
0d12de8134 Move slow mo video flattening to AssetLoader
PiperOrigin-RevId: 493300556
2022-12-12 11:25:45 +00:00
kimvde
339205f428 Move progress updates to the AssetLoader
This is necessary to move video slow motion flattening to the
AssetLoader because this step can change the duration. As we use the
duration before flattening to calculate the progress, we must also use
the position before flattening.

PiperOrigin-RevId: 493291990
2022-12-12 11:24:46 +00:00
claincly
cc43ddb528 Re-assign TODOs
PiperOrigin-RevId: 493281221
2022-12-12 11:20:25 +00:00
ibaker
636a4a8538 Add javadoc links to README files
Fix some other link titles and destinations spotted along the way.

#minor-release

PiperOrigin-RevId: 493276172
2022-12-12 11:19:23 +00:00
kimvde
00859a78fd Add comment to explain why cancel exceptions are handled differently
PiperOrigin-RevId: 493260798
2022-12-12 11:17:25 +00:00
huangdarwin
bb2c4c2eff HDR: Add effect interface support for different in/out color transfers
Modify FrameProcessor and MatrixTextureProcessor interfaces to support
different input and output color transfers. Does not implement conversion between
color ranges (ex. HDR and SDR), but should allow for conversion between color
transfers of the same color range (ex. HLG and PQ).

This supports in-app tone mapping, where we need a single FrameProcessor to
input HDR color transfers (ex. HLG/PQ) and output SDR (ex. gamma2.2). This also
supports previewing, where we need a single FrameProcessor to be able to input HLG
and output PQ.

Manually tested by confirming colors still look right on SDR and HDR videos
with a rotation and color affect applied.

PiperOrigin-RevId: 493108678
2022-12-12 11:16:26 +00:00
claincly
02bf7594e2 Change argument order to match API council suggestion
By putting Listener as the last argument.

PiperOrigin-RevId: 493100906
2022-12-12 11:15:23 +00:00
kimvde
7ea74444f3 Avoid sending message on dead thread
PiperOrigin-RevId: 493040714
2022-12-12 11:14:27 +00:00
claincly
900e86ffc8 Allow specifying an Executor in FrameProcessor.Factory.create()
From this CL on, FrameProcessor listeners will be invoked from an Executor that
is passed in when creating the FrameProcessor.

GlTextureProcessor needs to invoke the ErrorListener on the said Executor too.

PiperOrigin-RevId: 493018583
2022-12-12 11:13:30 +00:00
tonihei
634c6161f1 Fix threading of onFallbackApplied callback
The callback is currently triggered on the ExoPlayer playback thread
instead of the app thread that added the listener.

PiperOrigin-RevId: 492474405
2022-12-12 11:08:17 +00:00
kimvde
85c48c481e Make sure that the sample pipeline data is processed regularly.
This is necessary to move video decoding to the AssetLoader. Otherwise,
if the decoder max pending frame count is reached, the AssetLoader will
stop queuing frames to the pipeline, and process data will not be called
anymore.

PiperOrigin-RevId: 492392621
2022-12-12 11:01:36 +00:00
samrobinson
d711bf7b24 Merge feedXFromInput and feedXFromSilence in ATSP.
PiperOrigin-RevId: 492195931
2022-12-12 11:00:31 +00:00
samrobinson
e5727a2cc7 Add an E2E test for changing sample rate with AudioProcessor.
PiperOrigin-RevId: 492160193
2022-12-12 10:54:06 +00:00
samrobinson
0993479efe Add an option to force a silent audio track.
PiperOrigin-RevId: 492154544
2022-12-12 10:53:06 +00:00
claincly
65fe41bc2c Fix Samsung MediaCodec encoder reports incorrect timestmp in EOS
PiperOrigin-RevId: 492023573
2022-12-12 10:48:08 +00:00
huangdarwin
19b4b310fa Transformer: Move getMaxPendingFrameCount logic to constructor.
Also, use the mediaCodecName passed into the constructor to get the
maxPendingFrameCount.

PiperOrigin-RevId: 491985044
2022-12-12 10:46:13 +00:00
claincly
7dee3aec51 Add function to clear the cached list of encoders
EncoderUtil using a static list to cache encoders means it's not possible to
use a different set of encoders for different tests when running all
robolectric tests together.

PiperOrigin-RevId: 491959350
2022-12-12 10:43:03 +00:00
kimvde
eecf7caed0 Move audio decoding to AssetLoader
PiperOrigin-RevId: 491933937
2022-12-12 10:42:04 +00:00
huangdarwin
1aba245a30 HDR: Limit Z Fold 4 c2.qti decoder pending frame count.
Bypass Z Fold 4 HDR10 tone-mapping bug by limiting the max frame count to 12.

This passed with a value of 14, and failed with a value of 15, but I figured I'd use 12 just to be safe.

PiperOrigin-RevId: 491684058
2022-12-12 10:37:55 +00:00
samrobinson
a8e9d158cd Change TransformerInternal durationMs to durationUs.
Follow-up CLs will use this duration for silent audio.

PiperOrigin-RevId: 491670359
2022-11-29 18:53:28 +00:00
claincly
4033013ff8 Move audio MIME type fallback away to ATSP
PiperOrigin-RevId: 491660842
2022-11-29 18:52:17 +00:00
kimvde
09df56f31a Move sample processing to transformer thread
PiperOrigin-RevId: 491623586
2022-11-29 18:49:15 +00:00
claincly
c1ef48f20d Move video encoding MIME type fallback to VTSP
Main change:

- Removed `Codec.EncoderFactory.createForVideoEncoding`'s argument of a list
of allowed MIME types
- Moved the check for whether a video MIME type is supported to VTSP

PiperOrigin-RevId: 491611799
2022-11-29 18:46:55 +00:00
claincly
0f85ce5dbf Fix VideoEncoderSetting.bitrate is ignored when set
PiperOrigin-RevId: 491377695
2022-11-29 18:44:24 +00:00
kimvde
c9b9054c0b Add Transformer internal thread
This thread just starts the player and handles the player callbacks for
now. Sample pipelines are still run on the playback thread.

PiperOrigin-RevId: 491299671
2022-11-29 18:38:22 +00:00
samrobinson
dd96c97bd5 Bump muxer watchdog abort timer default timeout to 10s.
PiperOrigin-RevId: 490756055
2022-11-29 18:27:45 +00:00
kimvde
e12dd1ac5d Surface player release timeouts
Also make sure the demo app doesn't crash when it happens.

PiperOrigin-RevId: 490725959
2022-11-24 15:00:34 +00:00
huangdarwin
7d45b9d83a Tests: Set SSIM decoder KEY_PRIORITY to 1.
This reduces the priority to best effort (from the default that seems to be best effort), and allows us to run SSIM even on 8k24fps video. Without this CL,
start()'ing a second codec may result in a MediaCodec.CodecException.

Tested to confirm that transformation8k24():
* fails deterministically without this CL, or with KEY_PRIORITY set to 0.
* succeeds deterministically after this CL (~18s on Samsung Z Fold 4)

PiperOrigin-RevId: 490570416
2022-11-24 14:52:09 +00:00
huangdarwin
d74eb7500f Transformer: Document when OMX video codecs will no longer exist.
We cannot check this in code, due to DEVICE_INITIAL_SDK_INT being a @SystemApi, and
reflection being a bit risky/unstable.

PiperOrigin-RevId: 490537916
2022-11-24 14:51:06 +00:00
huangdarwin
2693ca138d Transformer: Remove obsolete TODO from TransformationException.
The bug has since been fixed.

The values still could change, as the API is labelled as @UnstableApi, so it's
probably fine to leave the <p> tag mostly as is.

PiperOrigin-RevId: 490509205
2022-11-24 14:47:37 +00:00
huangdarwin
0a176d1f05 Tests: Move codec configure() out of VideoDecodingWrapper constructor.
This allows us to release both codecs used in SSIM when one fails to
configure() or start().

Tested and confirmed that on Samsung Galaxy Z Flip 4, running all
TransformationTest.java tests, tests after transform8k24() fails to start the
2nd codec:
* Before this CL, all fail.
* After this CL, all pass.

PiperOrigin-RevId: 490461560
2022-11-24 14:40:47 +00:00
samrobinson
af1d7b9b1b Fix SpeedChangingAudioProcessor handling of EMPTY_BUFFER.
PiperOrigin-RevId: 490265564
2022-11-24 14:36:14 +00:00
huangdarwin
4d50e0221e Tests: Throw exception when network is needed but not available.
This exception is a bit shorter and more clear (and is more clear that this is a
test issue, as opposed to the prior issue that was thrown as an ExoPlayer
DataSourceException, which may seem like a legitimate Transformer failure)

PiperOrigin-RevId: 490252772
2022-11-22 17:43:08 +00:00
huangdarwin
d0c5168902 HDR Tests: Rename remote URIs to REMOTE instead of ASSET.
Following naming conventions throughout AndroidTestUtil, REMOTE files should have REMOTE instead of ASSET. Update the URI and FORMAT names accordingly.

PiperOrigin-RevId: 490237673
2022-11-22 17:42:01 +00:00
kimvde
d8eec16369 Refactor progress logic to be thread safe
PiperOrigin-RevId: 489984147
2022-11-22 10:10:21 +00:00
kimvde
56cf5e0774 Build TransformationResult only when transformation succeeded
PiperOrigin-RevId: 489442518
2022-11-22 10:04:56 +00:00
kimvde
f658fe79b1 Make sure Muxer is always accessed from playback thread
This thread will become the Transformer internal thread.

PiperOrigin-RevId: 489168435
2022-11-22 09:50:13 +00:00