1471 Commits

Author SHA1 Message Date
claincly
a879bc2154 Rewrite sequence duration matching on MediaSource level
This simplifies the later handling of speed adjusted media.

PiperOrigin-RevId: 675114587
2024-09-16 05:55:00 -07:00
claincly
6c92402fbb Re-enable audio tests
The tests were flaky because CompositionPlayer registers audio sequences'
formats to the audio pipeline in first-come-first-serve order. With the change
in bc8d82355f, the audio format is deterministic.

The test is turned off in 060356ea00

PiperOrigin-RevId: 674261801
2024-09-13 05:44:28 -07:00
Googler
ff656012a8 Update the unisoc blocklist
PiperOrigin-RevId: 674101970
2024-09-12 19:33:13 -07:00
claincly
bc8d82355f Block secondary audio processing until primary format is configured
PiperOrigin-RevId: 673961235
2024-09-12 12:42:09 -07:00
ivanbuper
e88d6fe459 Fix truncation error acumulation for Sonic's resampling algorithm
Sonic would accumulate truncation errors on float to int conversions
that caused the final output sample count to drift noticeably, by
hundreds of samples on streams of a few minutes of length. The fix now
keeps track of the truncation error and compensates for it.

Other small fixes include eliminating lossy operations (e.g. int
division) and using doubles instead of floats for resampling where
helpful.

This CL also introduces `SonicParameterizedTest`, which helps test
resampling on an arbitrary number of randomly generated parameters,
with random sample data. `SonicParameterizedTest` uses `BigDecimal`s
for calculating sample count values, as to avoid precision issues with
large sample counts.

PiperOrigin-RevId: 673852768
2024-09-12 08:14:25 -07:00
kimvde
ce98b7d379 Rename SequencePlayerRenderersWrapper
Also remove reference to SequencePlayerRenderersWrapper in inner
classes to make them effectively static.

PiperOrigin-RevId: 673720251
2024-09-12 00:52:17 -07:00
rohks
f133e8d1f2 Fix preroll sample handling for non-keyframe media start positions
When processing edit lists in MP4 files, the media start position may be a non-keyframe. To ensure proper playback, the decoder must preroll to the preceding keyframe, as these preroll samples are essential for decoding but are not rendered.

Issue: google/ExoPlayer#1659

#cherrypick

PiperOrigin-RevId: 673457615
2024-09-11 11:02:39 -07:00
rohks
d9a678483b Refine sample presentation time validation for negative PTS workaround
In case of negative PTS workaround, instead of disallowing all cases where samples are not in presentation order, we now validate that the first sample's presentation time is the smallest. This adjustment allows for correctly applying an offset to ensure all samples have a presentation time >= 0.

#cherrypick

PiperOrigin-RevId: 673434793
2024-09-11 10:04:16 -07:00
rohks
bb3d055191 Do not drop negative timestamp video buffers during transmuxing
Prevents discarding video buffers with key frame which are required for decoding.

#cherrypick

PiperOrigin-RevId: 673375261
2024-09-11 07:00:02 -07:00
kimvde
8271a5f920 Rename CompositingVideoSinkProvider and PreviewAudioPipeline
The components are mirror components for video and audio so they should
have a matching name

PiperOrigin-RevId: 673357081
2024-09-11 05:49:35 -07:00
sheenachhabra
a7788e0d60 Rename last sample duration behaviour enums
This is to improve readability.

PiperOrigin-RevId: 670563611
2024-09-03 08:15:23 -07:00
dancho
1c61fbadf7 Downscale bitmaps during decoding in Transformer
Limit input image size in Transformer to be less than 4096x4096.
For very large images, this can reduce memory usage substantially,
and stays away from `GL_MAX_TEXTURE_SIZE` - often 4096

PiperOrigin-RevId: 670555939
2024-09-03 07:53:13 -07:00
kimvde
af61c03e09 Move ExoPlayerEffectPlaybackSeekTest out of performance directory
These tests do not test performance. Moving them out of this directory
ensures they are run on emulators and on all physical devices.

PiperOrigin-RevId: 670505992
2024-09-03 04:54:12 -07:00
kimvde
af2f9cb37f Increase ExoplayerEffectPlaybackSeekTest time out
This test will be moved so that it is run on emulators, which are
generally slower to run.

PiperOrigin-RevId: 670495551
2024-09-03 04:13:24 -07:00
sheenachhabra
95f69b649d Use InAppMuxer in tests for API < 25
MediaMuxer doesn't support B-frame before API 25.

The fix is added only in those test which appeared in triage failure.
It can be added to other tests as they are discovered.

PiperOrigin-RevId: 670480966
2024-09-03 03:18:01 -07:00
samrobinson
1d9276863c Ensure force silence SampleConsumerWrapper has the right track type.
PiperOrigin-RevId: 670226122
2024-09-02 08:30:46 -07:00
dancho
207684ca66 Prototype frame extraction based on analyzer mode
Make ByteBufferGlEffect public.
Build a speed test, and end to end test that verify
frames can be copied to CPU-accessible ByteBuffer

PiperOrigin-RevId: 670213343
2024-09-02 07:34:33 -07:00
sheenachhabra
0843444a34 Fix order of elements in FrameworkMuxer.java
PiperOrigin-RevId: 670205944
2024-09-02 07:04:00 -07:00
claincly
060356ea00 Temporarily disable multi-sequence audio playback test
Disable assertions and make sure playback passes.

The flake is caused by having different sequences starting with MediaItems of
different audio format, and it's undefined behaviour as to which one CompositionPlayer chooses to use.

PiperOrigin-RevId: 670195113
2024-09-02 06:16:43 -07:00
sheenachhabra
f0fa7640ca Add support for setting video duration in InAppMuxer
Similar support was previously added in FrameworkMuxer.

PiperOrigin-RevId: 670193986
2024-09-02 06:11:32 -07:00
kimvde
748e4e5230 Fix hanging ExoPlayerPlaybackSeekTest
The test uses a video renderer to count the number of frames and could
time out if some frames were dropped.

PiperOrigin-RevId: 670181693
2024-09-02 05:21:01 -07:00
dancho
b3b4c80641 Release the analyzer mode placeholder surface
PiperOrigin-RevId: 670179138
2024-09-02 05:09:30 -07:00
samrobinson
5eba716410 Add a composition export dump test with a shorter second sequence.
PiperOrigin-RevId: 669348374
2024-08-30 09:20:58 -07:00
ibaker
05cbbffbd7 Add a space in the ParserException message
PiperOrigin-RevId: 668870991
2024-08-29 03:51:40 -07:00
kimvde
3587afc9d7 Move CompositionPlayerSeekTest out of performance directory
These tests do not test performance. Moving them out of this directory
ensures they are run on emulators and on more than 1 physical device.

PiperOrigin-RevId: 668859017
2024-08-29 03:14:12 -07:00
kimvde
8367e420ad Import correct nullable annotation
PiperOrigin-RevId: 668521379
2024-08-28 10:54:29 -07:00
samrobinson
9c5ea4f1ba Assert silent bytes are 0 in AudioGraphInputTest
PiperOrigin-RevId: 668403467
2024-08-28 05:18:37 -07:00
samrobinson
84f4c7bbcc Create and assert against dump files in ParameterizedAudioExportTest.
PiperOrigin-RevId: 668002146
2024-08-27 08:43:27 -07:00
claincly
36d61000fd Reorder the assetions in tests
So it's more clear what's actually broke in the error log.

PiperOrigin-RevId: 667964963
2024-08-27 06:43:58 -07:00
claincly
2b031484fe Allow looping sequences in CompositionPlayer
PiperOrigin-RevId: 666793658
2024-08-23 07:54:13 -07:00
tonihei
6f0cb539d0 Add util method BaseRenderer.getStreamOffsetUs
This simplifies the common pattern of saving this offset in
onStreamChanged to use it at a later point.

PiperOrigin-RevId: 666226263
2024-08-22 00:53:04 -07:00
kimvde
4ba9f59492 Improve logic to wait for a seek in CompositionPlayerSeekTest
The logic was assuming that the shader program was only flushed when
seeking. This is true if a single renderer is used but, with multiple
renderers, the shader program can be flushed at the transition.

This change is necessary to make the test pass for prewarming because 2
video renderers will be used in that case.

PiperOrigin-RevId: 666215967
2024-08-22 00:15:30 -07:00
kimvde
059ad62377 Add timeout to CountDownLatch awaits
This is to avoid tests hanging forever

PiperOrigin-RevId: 665826507
2024-08-21 05:02:18 -07:00
kimvde
51622b6d80 Add BufferingVideoSink
PiperOrigin-RevId: 665733825
2024-08-21 00:15:25 -07:00
kimvde
9b69690856 Move some tests from CompositionPlaybackTest out of performance
directory

PiperOrigin-RevId: 665202382
2024-08-20 00:13:21 -07:00
claincly
677f8ad9f4 Fix edge case: no frame between trim position and the next sync sample
PiperOrigin-RevId: 664841893
2024-08-19 09:32:32 -07:00
kimvde
cd0a7e1143 Document why playback tests are in the performance package
PiperOrigin-RevId: 664792272
2024-08-19 07:14:32 -07:00
claincly
92635342fb Move the test of removeAudio to robolectric
The audio mixing is not deterministic on real device testing because of
threading, but runs deterministically on robolectric tests

PiperOrigin-RevId: 662925912
2024-08-14 08:23:22 -07:00
dancho
879771ded2 Add an experimental flag for renderer dynamic scheduling
Use ExoPlayer dynamic scheduling to reduce the render() interval for
older API devices where `DefaultCodec.getMaxPendingFrameCount()` is set
to 1 in order to prevent frame drops.

Controlled via API on DefaultDecoderFactory.

Add TransformerForegroundSpeedTest that mimics transcoding while the app
is in foreground.

PiperOrigin-RevId: 662925764
2024-08-14 08:20:21 -07:00
claincly
62da288caf Support removing video in previewing
Video playback will be disabled if *any* `EditedMediaItem` removes video. This
is consistent with Transformer.

PiperOrigin-RevId: 662093484
2024-08-12 08:28:16 -07:00
claincly
46eeabb877 Support setRemoveAudio in CompositionPlayer
PiperOrigin-RevId: 662063725
2024-08-12 06:34:44 -07:00
dancho
931b0e25f1 Add a DefaultDecoderFactory option to configure operating rate
This has the largest impact during operations with no encoder, such as
frame extraction. Add a matching performance test.

PiperOrigin-RevId: 661220044
2024-08-09 05:03:02 -07:00
dancho
c1078e3cfa Do not checkState based on input data
App users can choose arbitrary data that might not be
anticipated by developers. Transformer shouldn't `checkState` based on
media data or file type -- report an error for unsupported data instead.

Public API change `ImageAssetLoader` needs to parse MIME type and now accepts
`Context` as parameter.

PiperOrigin-RevId: 660762459
2024-08-08 03:18:27 -07:00
sheenachhabra
0530d663bc Rename muxer/Mp4Utils.java to muxer/Mp4MuxerUtil.java
PiperOrigin-RevId: 660417092
2024-08-07 09:23:53 -07:00
samrobinson
a23f655cf4 Check AssetLoader supports output type required by TransformerInternal.
PiperOrigin-RevId: 659557063
2024-08-05 08:10:23 -07:00
claincly
5165d7df68 Rollback of ffc45820b9
PiperOrigin-RevId: 659520675
2024-08-05 05:31:46 -07:00
dancho
a79b80fcee Support setting H.264 level
Try to follow recommendations in
https://developer.android.com/media/optimize/sharing
more closely

For maximum compatibility the H.264 level should be less than or equal to 4.1

PiperOrigin-RevId: 658755139
2024-08-02 05:06:53 -07:00
dancho
a98a37aa75 Initial Frame Extractor Transformer Factory
Package-private until API is more useable.

Similar to frame analyzer mode: uses ImageReader instead of an encoder,
and no muxer.

PiperOrigin-RevId: 658446675
2024-08-01 10:21:10 -07:00
dancho
ddc86686b7 Experimental flag to limit the number of frames in encoder
Transformer.experimentalSetMaxFramesInEncoder controls max number
of frames in encoder.

VideoFrameProcessor now allows delayed releasing of frames to Surface,
while still using the original presentation time.

VideoSampleExporter can now configure video graphs to not render frames
automatically to output surface.

VideoSampleExporter.VideoGraphWrapper tracks how many frames are ready
to be rendered to Surface, and how many frames are already in-use by encoder.

PiperOrigin-RevId: 658429969
2024-08-01 09:33:16 -07:00
samrobinson
34d3dc926d Use getDecoderNames method, rather than duplicating logic.
PiperOrigin-RevId: 658368479
2024-08-01 05:38:30 -07:00