1064 Commits

Author SHA1 Message Date
samrobinson
aef2fcb900 Plumb EditedMediaItem for audio preview.
PiperOrigin-RevId: 584865962
2023-11-23 05:13:32 -08:00
samrobinson
1524d12634 Avoid using unneeded double ended queue.
These usages have no need for the double ended input functionality. All
other usages across media3 are ConcurrentLinkedQueue.

PiperOrigin-RevId: 584841104
2023-11-23 02:51:20 -08:00
tofunmi
2d77e4d22c Implement trim optimization in Transformer
PiperOrigin-RevId: 584622392
2023-11-22 07:28:34 -08:00
claincly
ad96ca3b30 Make vararg APIs more robust
Composition and EditedMediaItemSequence don't allow empty lists in their main
constructors, so shouldn't the vararg API. This is more inline with Effective
Java item 53.

PiperOrigin-RevId: 583415124
2023-11-17 09:51:39 -08:00
andrewlewis
7b9aa87344 Allow allocating more buffers when transmuxing
When transmuxing, the `EncodedSampleExporter` maintains a queue of input
buffers that get filled with encoded data by the asset loader. The number of
buffers was limited to avoid using more and more memory if producer (asset
loader) gets far ahead of the consumer (exporter).

Previously this limit was fixed at 10 buffers, but increasing the number of
buffers can make some transmux operations much faster. Allow allocating between
a min and max number of buffers, and also set a target allocation size beyond
which new buffers can't be allocated. This allows audio formats which require
many small buffers to be processed more quickly, while preventing allocating
too much memory for hypothetical very high bitrate formats.

'Remove video' edits on local videos in particular get much faster, because
audio buffers are very short and there are lots of them. With a sample 10
minute video, a 'remove video' edit took 2 seconds (36 seconds before this
change).  With a sample 1 minute removing video took 0.25 seconds after this
change (2.5 seconds before).

The speed improvement is smaller for other types of edits that retain the video
track. Transmuxing a 10 minute video retaining the video track took 26 seconds
(40 seconds before).

PiperOrigin-RevId: 583390284
2023-11-17 08:11:25 -08:00
samrobinson
d5fbf0007b Migrate to Util.durationUsToSampleCount in transformer audio.
PiperOrigin-RevId: 582700443
2023-11-15 09:15:10 -08:00
tonihei
b570c72588 Normalize MIME types when accepting user or media input
MIME types are case-insensitive, but none of the many existing
comparisons across our code base take this into account. The
code can be made more robust by normalizing all MIME types at the
moment they are first set into a class/builder and adding toLowerCase
as part of the normalization.

Most concretely, this fixes an issue with playing HLS streams via
the IMA SDK where the stream MIME type is indicated with all lower
case "application/x-mpegurl", which failed the MIME type comparison
in DefaultMediaSourceFactory.

PiperOrigin-RevId: 582317261
2023-11-14 07:40:15 -08:00
andrewlewis
1b13c87825 Refine test skipping logic
Skip wearable devices, which don't have the same CDD requirements on codecs from Android 5 (API 21).

Limit an existing skipping case to pre API 33 devices, so that we can catch failures on Android 13 (API 33) onwards, and add OnePlus 9 Pro which also times out calculating SSIM. Remove TODOs for removing test/SSIM skipping now they are restricted to API version.

PiperOrigin-RevId: 581985554
2023-11-13 09:14:41 -08:00
andrewlewis
c00161fb8f Skip export4K60 on Pixel 3
This device over-reports encoding capabilities, so skip the test.

This is a Pixel device that was upgraded to Android 12 (API 31/32) so the issue should only affect devices that didn't take OTAs.

PiperOrigin-RevId: 581981073
2023-11-13 09:02:38 -08:00
andrewlewis
c763ed7941 Expand blocklist for export8K24 to OnePlus 9 Pro
This is failing with the same error about having insufficient resources.

PiperOrigin-RevId: 581921054
2023-11-13 05:00:15 -08:00
claincly
ef9a76efe2 Fix composition's duration not respecting presentation duration
PiperOrigin-RevId: 580540264
2023-11-08 08:23:54 -08:00
samrobinson
2ae22a412a Cleanup small nits in AudioGraphInput.
PiperOrigin-RevId: 580503934
2023-11-08 05:49:34 -08:00
christosts
dae3e275c4 Increase performance test upper bound threshold
PiperOrigin-RevId: 580497700
2023-11-08 05:22:43 -08:00
samrobinson
76474684f9 Plumb audio through AudioGraph for Composition preview.
The dump file diff as part of this change is because using AudioGraph
means the 2nd item is automatically edited to match the AudioFormat of
the 1st item {44.1KHz mono}, rather than {48KHz stereo}.

Manually verified that for the 2nd item, data output:
* Before: 66936 bytes (16734 frames) output = 348_625us of audio.
* After: 30750 bytes (15375 frames) output = 348_639us of audio.

The small final buffer is caused by SonicAudioProcessor outputting all
pending data when EOS queued, and is WAI.

PiperOrigin-RevId: 580494578
2023-11-08 05:13:08 -08:00
tofunmi
4311cf1d6c Mp4MetadataInfo: add format and time-based I-frame timestamp extraction
PiperOrigin-RevId: 580132463
2023-11-07 04:31:25 -08:00
tofunmi
ba0724ca78 Pass initializationData to EncoderFactory when requested
PiperOrigin-RevId: 580098432
2023-11-07 02:04:39 -08:00
andrewlewis
14fb45626d Add missing visibility modifier
PiperOrigin-RevId: 580092052
2023-11-07 01:33:44 -08:00
sheenachhabra
5db9a66b3b Implement resume API in Transformer
Changes includes:
1. Add resume flow.
2. Change demo app code to resume export.
3. Changes in test infra to trigger resume.
4. E2E Test cases

PiperOrigin-RevId: 579895744
2023-11-06 11:01:04 -08:00
claincly
414b72619b Remove unnecessary memory allocation in OpenGL
PiperOrigin-RevId: 579835031
2023-11-06 07:21:23 -08:00
tofunmi
3253f1b5cd Add microsecond precision to MediaItem.ClippingConfiguration
PiperOrigin-RevId: 578881990
2023-11-02 09:40:11 -07:00
tofunmi
de82aa93f0 Move shouldTrancodeAudio/Video to static helper methods
PiperOrigin-RevId: 578617730
2023-11-01 13:12:56 -07:00
sheenachhabra
5b0be4e8b0 Create unique name for export output file.
With pause/resume functionality coming in, we need the output file
path to be different from the previous one.

PiperOrigin-RevId: 577175540
2023-10-27 06:12:23 -07:00
huangdarwin
346b9257ba Flush: VideoFrameProcessor texture output
PiperOrigin-RevId: 576928149
2023-10-26 11:06:04 -07:00
sheenachhabra
1e2815cade Simplify Mp4ExtractorWrapper implementation
Changes includes:
1. Create static factory method and hide constructor.
2. Move all the fetching logic to init() method.

PiperOrigin-RevId: 576544902
2023-10-25 09:02:02 -07:00
claincly
3204da41fe Add performance test for composition previewing
PiperOrigin-RevId: 576509031
2023-10-25 06:22:16 -07:00
tofunmi
beed1bd76e Pass initial timestamp offset to EncodedSampleExporter
When transmuxing, we usually only need to offset the timestamp by the position of a mediaItem in a sequence.

Trim optimization introduces another type of offset: for the transmux of the second part of the video we need to offset the timestamps by the total duration already trancoded by transformer.

PiperOrigin-RevId: 576134656
2023-10-24 07:39:23 -07:00
claincly
e8adbd9075 Require duration in EditedMediaItem in composition previewing
Adds an additional field for actual presentation duration of the
EditedMediaItem

PiperOrigin-RevId: 576090574
2023-10-24 04:09:14 -07:00
tofunmi
771d203204 ExoAssetLoaderRenderer:Drop negative timestamp buffers when transmuxing
Prevents queuing unneeded buffers when transmuxing. (we already do this when transcoding).

PiperOrigin-RevId: 576080462
2023-10-24 03:21:56 -07:00
claincly
2673b64371 Ignore 8k export test on incapable devices
And add a test to cover 8k transcode to lower resolution

PiperOrigin-RevId: 574809781
2023-10-19 03:58:13 -07:00
andrewlewis
680eed52f3 Add support for flattening SEF files with H.265
PiperOrigin-RevId: 574173120
2023-10-17 09:26:14 -07:00
Daniele Sparano
f14732873f Change isColorValid with isDataSpaceValid 2023-10-17 11:42:33 +00:00
Daniele Sparano
1fe9c3303e Resume toLogString() in ColorInfo; revert DebugTextViewHelper 2023-10-17 11:42:33 +00:00
claincly
dc0bee9307 Perform case insensitive comparison in tests
The model is "Redmi 6A" on the device

PiperOrigin-RevId: 574103629
2023-10-17 04:37:21 -07:00
huangdarwin
b153a271a3 Test: Ignore failing tests fixed in newer API versions.
PiperOrigin-RevId: 570954436
2023-10-05 02:53:15 -07:00
huangdarwin
8953f26a5c Test: Move OpenGL tone mapping capabilities check to utility method.
PiperOrigin-RevId: 570316091
2023-10-03 02:00:55 -07:00
huangdarwin
9ca5d50b2d Test: Ignore failing tests fixed in newer API versions.
PiperOrigin-RevId: 570061196
2023-10-02 07:34:58 -07:00
claincly
34dddfe9d5 Add previewing specific video graph.
PiperOrigin-RevId: 569473178
2023-09-29 06:16:16 -07:00
christosts
0b4638af15 ExportTest: make 8K asset and trim
Move remote 8K file to local and trim to 320ms.

Trim done with ffmpeg:

`ffmpeg -i {remote_file} -t 0.3 -c:v copy -c:a copy 8k24fps_300ms.mp4`

PiperOrigin-RevId: 569449962
2023-09-29 04:07:40 -07:00
claincly
cd6f8a42a5 Move Single/MultiVideoGraph impl to effect
PiperOrigin-RevId: 569188658
2023-09-28 08:27:06 -07:00
huangdarwin
1df2210cf4 Test: Update HDR GL tone-map fallback string.
PiperOrigin-RevId: 568920716
2023-09-27 12:05:30 -07:00
samrobinson
4e10c41bcc Throw Exception if posting to application handler fails.
PiperOrigin-RevId: 568799683
2023-09-27 03:30:30 -07:00
samrobinson
c03a0fb66b Limit SequenceAssetLoader variable scope.
All usages of these variables are within the SampleConsumerWrapper,
so limit them to that scope.

PiperOrigin-RevId: 568582645
2023-09-26 10:38:07 -07:00
samrobinson
0b62d37425 Verify a thread is alive before sending a message to it.
PiperOrigin-RevId: 568515736
2023-09-26 06:09:19 -07:00
tonihei
212f1f8ea8 Update decode-only flag logic in non-MediaCodec-renderers
MediaCodecRenderer has already been updated to not rely on the
input stream to mark its samples as decode-only and instead use
a simple time-based comparison to achieve the same effect.

This change makes the same update for all other renderers that
either use the flag directly or forward to a "decoder" instance.

PiperOrigin-RevId: 568232212
2023-09-25 08:53:19 -07:00
huangdarwin
444cb3fb3a Test: Rename getBitmap to getBitmapAtPresentationTimeUs
PiperOrigin-RevId: 567683139
2023-09-22 12:06:57 -07:00
huangdarwin
8a40952e1b Test: Use timestamp iterator in texture output test.
Simplify tests, before we add some similar tests.

PiperOrigin-RevId: 567666340
2023-09-22 11:02:58 -07:00
huangdarwin
ba8c85a277 Compositor: Add VideoCompositorSettings to Composition.
This allows apps using Transformer to customize how a Composition is used.

PiperOrigin-RevId: 567633129
2023-09-22 08:51:35 -07:00
claincly
d9563b133e Split VideoGraph interface and move VideoGraph to common
PiperOrigin-RevId: 567599249
2023-09-22 06:01:01 -07:00
huangdarwin
0675e48685 HDR: Update HdrMode fallback to use OpenGL tone-mapping.
OpenGL tone-mapping is more reliable and widely supported than
MediaCodec tone-mapping.

PiperOrigin-RevId: 567267389
2023-09-21 04:33:09 -07:00
christosts
7b580d3cf8 Add performance playback test for video effects
PiperOrigin-RevId: 567000714
2023-09-20 10:11:08 -07:00