1208 Commits

Author SHA1 Message Date
ibaker
62f6c64a91 Rename test.mp3 test asset to test-cbr-info-header.mp3
This file is CBR encoded with LAME, so it has an `Info` header (the CBR
equivalent to `Xing`).

A follow-up change will use this file in `Mp3ExtractorTest`.

Issue: androidx/media#878
PiperOrigin-RevId: 595938327
2024-01-05 02:31:20 -08:00
samrobinson
c230414bd3 Set the default audio encoder bitrate in DefaultEncoderFactory.
Matches the approach for video frame rate.

PiperOrigin-RevId: 595385930
2024-01-03 06:37:16 -08:00
sheenachhabra
27ae6d974e Allow creating fragmented MP4 file via InAppMuxer
PiperOrigin-RevId: 594431665
2023-12-29 04:15:01 -08:00
sheenachhabra
1609928242 Throw exception when B-frames are written to Mp4Muxer
Mp4Muxer does not support out of order B-frames. Currently it
silently writes out of order B-frames, producing an invalid file (with
negative sample durations).

Although `Mp4Extractor` is somehow able to process this invalid file and
`Exoplayer` is able to play it but that is unexpected.

The `sample.mp4` test file contains B frames. Other test files does not
contain `H264 video + AAC audio` format hence created a new test file by
running `sample.mp4` via `Transformer` after applying some effects.

PiperOrigin-RevId: 594016144
2023-12-27 08:32:36 -08:00
tofunmi
e3056dacac Rename Mp4MetadataInfo to mp4Info
PiperOrigin-RevId: 593143940
2023-12-22 11:10:30 -08:00
tofunmi
4fc11a98a1 ignore dropSamplesBeforeFirstVideoSample in audio-only streams
PiperOrigin-RevId: 593126632
2023-12-22 08:54:29 -08:00
tofunmi
7e12b9e15f Add rough progress updates in trim optimization
PiperOrigin-RevId: 593116025
2023-12-22 07:37:23 -08:00
tofunmi
1632f37d70 Transformer: Add api to drop audio samples before the first video frame
fix for Issue: androidx/media#829

Manual Testing: Viewed the transformer output file of previously problematic case in Exoplayer, Chrome, VLC, Quicktime and Safari and all showed the issue not to occur anymore. The newly produced output file can be found at https://github.com/androidx/media/assets/42352357/fdf105c1-9550-422f-b088-7900f655ac78

PiperOrigin-RevId: 593104752
2023-12-22 06:18:07 -08:00
simakova
09bde8053d Update method description for setEffects in Composition
PiperOrigin-RevId: 593071081
2023-12-22 02:51:52 -08:00
andrewlewis
ef7842dd88 Fix typo
PiperOrigin-RevId: 593061829
2023-12-22 02:04:30 -08:00
andrewlewis
1845a4ae69 Expand operating rate workaround to T612 chipset
PiperOrigin-RevId: 592916187
2023-12-21 11:52:35 -08:00
huangdarwin
0ed19937d3 Test: Remove setInputColorInfo from VFP test runner.
Instead, for input videos, use the colorInfo provided by the extractor. Similarly, for input images, use sRGB, the only color currently in use.

Textures do still need the input ColorInfo provided though.

PiperOrigin-RevId: 592875967
2023-12-21 09:06:55 -08:00
tofunmi
57d00fcca0 Rename crow_emulator_transformer_output
PiperOrigin-RevId: 592873355
2023-12-21 08:56:48 -08:00
andrewlewis
7d11ee7538 Optionally notify codec init in default decoder factory
Replace the event for notifying fallback to cover codec initialization in
general (but keeping the list of errors).

Add a flag to control whether to try non-primary codecs, with the same
documentation as the similar flag in ExoPlayer's renderer.

Make the class final as it shouldn't be necessary to subclass it.

PiperOrigin-RevId: 592869868
2023-12-21 08:42:44 -08:00
tofunmi
3081ceb179 Remove the matchInitializationData option from TransformerInternal
Essentially a manual revert of ba0724ca78

PiperOrigin-RevId: 592869049
2023-12-21 08:38:58 -08:00
huangdarwin
4774fa1a60 Test: Update tone mapping capabilities check output format.
Checking the output format's mime type may skip tests more often than we'd like,
because we may desire using a lower-spec output mimetype than what's passed in, if
based on the input's HDR mimetype value.

Therefore, update this output format to null, for tone-mapping tests

PiperOrigin-RevId: 592855713
2023-12-21 07:30:07 -08:00
Googler
e25e497227 Make DefaultDecoderFactory try multiple formats before giving up
PiperOrigin-RevId: 592622544
2023-12-20 12:07:19 -08:00
tofunmi
4ce47ccdd3 MuxerWrapper: remove unnecessary volatiles
PiperOrigin-RevId: 592569012
2023-12-20 08:37:03 -08:00
tofunmi
087c07e596 Rename OptimizationResult options
Rename options to be more accurate of what they capture

PiperOrigin-RevId: 592155563
2023-12-19 01:50:40 -08:00
sheenachhabra
bb7aa2fb19 Update updateMetadataEntries() method javadoc
PiperOrigin-RevId: 591858840
2023-12-18 04:45:35 -08:00
sheenachhabra
1303bfbc7c Set language field when creating MediaFormat from a Format
Currently Transformer simply omits the language field from the input
media file when writing data to the output file.

PiperOrigin-RevId: 591287481
2023-12-15 10:06:38 -08:00
huangdarwin
7579693739 Effect: Move VideoFrameProcessor inputColorInfo interface to FrameInfo.
Move `inputColorInfo` from `VideoFrameProcessor`'s `Factory.create` to `FrameInfo`,
input via `registerInputStream`.

Also, manually tested on exoplayer demo that setVideoEffects still works.

PiperOrigin-RevId: 591273545
2023-12-15 09:17:39 -08:00
samrobinson
84471813b6 Update TODO internal bug number.
PiperOrigin-RevId: 590884408
2023-12-14 03:43:16 -08:00
huangdarwin
a15dfd75be GL: Request OpenGL ES 3.0 context on API 29+, with fallback to 2.0.
Despite GL 3.0 not being required on API 29+, it is experimentally
determined to always be supported on our testing devices, on API 29+.

That said, still fall back to OpenGL 2.0 if 3.0 is not supported,
just in case.

PiperOrigin-RevId: 590569772
2023-12-13 06:01:08 -08:00
tofunmi
2fa5430417 Update OptimizationResult to specify different reasons for abandonment
PiperOrigin-RevId: 590530694
2023-12-13 03:22:55 -08:00
tofunmi
37def3679f Trim optimization: fallback on format mismatches
Manual testing: tested manually with pixel 4a

PiperOrigin-RevId: 590284361
2023-12-12 11:53:43 -08:00
tofunmi
00d5b6ec99 TransformerEndToEndTest: replace /*testId=*/ with variable
PiperOrigin-RevId: 589765715
2023-12-11 03:14:34 -08:00
huangdarwin
1ebe099685 Test: Make a method scope private.
For clarity.

PiperOrigin-RevId: 589758781
2023-12-11 02:43:25 -08:00
samrobinson
1471528176 Change AudioGraph #release to #reset, resetting internal state.
PiperOrigin-RevId: 589113634
2023-12-08 06:36:32 -08:00
tofunmi
c5c8e988e8 Abandon trim optimization when transcoding effects are set
PiperOrigin-RevId: 588839072
2023-12-07 10:39:38 -08:00
samrobinson
ab798659d9 Request specific AudioFormat from AudioGraphInputs on creation.
Also adds an alternate way to configure the AudioGraph.

Apps should no longer need to ensure that inputs have the same sample
rate.

PiperOrigin-RevId: 588747431
2023-12-07 04:42:03 -08:00
tofunmi
be5b29d92d Update ExportResult.OptimizationResult options
Added a new ABANDONED option so one can tell the difference between when the optimization has been requested but not applied vs not requested at all. also changed the ordering do better represent the hierarchy of failure modes

PiperOrigin-RevId: 588720513
2023-12-07 02:52:16 -08:00
tofunmi
cd346ca14d Transformer: Add support for transmuxing audio in trim optimization
PiperOrigin-RevId: 588711597
2023-12-07 02:18:21 -08:00
huangdarwin
a98052b3fc Test: Mention expected bitmap in assertion message.
This helped me debug which bitmap was actually failing. Otherwise, you need to clear all bitmaps on the device relating to this test, then adb pull all bitmaps and see the last uploaded one to see, which is much more confusing.

PiperOrigin-RevId: 588374081
2023-12-06 04:17:37 -08:00
samrobinson
757585ce6d Release MuxerWrapper in every test (in an @After block).
PiperOrigin-RevId: 588110566
2023-12-05 10:13:27 -08:00
tofunmi
b0e2fd78af Use getter for presentationDurationUs
Delays failures to when the field is used in preview code, so that exports in the demo app can continue succeed

PiperOrigin-RevId: 588022569
2023-12-05 04:43:57 -08:00
tofunmi
88b15e73b2 fix: update TransformerPauseResumeTest to use isRunningOnEmulator
#minor-release

PiperOrigin-RevId: 586964536
2023-12-01 04:01:52 -08:00
samrobinson
fe272d3c49 Follow practices/guidance in Muxer tests around:
* @Before and @After usage. [1].
* static fields before non-static. [2].

@Before method should typically be paired with an @After method,
focused on ensuring that the component is released regardless of what
the test does.

In tests, inlining final class variables is preferrable [1]. In general
things like the file path should be part of the test (the @Rule means
we don't need before/after) if only used once.

Statically importing values and using them directly is preferable to
having a variable declared as non-final that's effectively final,
because from a readability perspective someone can see (the caps) that
the value is final static and immutable, so doesn't have to check if
it's changed/reassigned.

PiperOrigin-RevId: 586697887
2023-11-30 09:29:18 -08:00
andrewlewis
3204313f13 Remove debug logging
This case is expected and we aren't trying to gather information about it
currently, so we shouldn't log.

PiperOrigin-RevId: 586015969
2023-11-28 10:01:37 -08:00
tofunmi
b598c96c2f Update emulator device names
PiperOrigin-RevId: 585682881
2023-11-27 10:02:02 -08:00
andrewlewis
1c7c06999e Turn off debug trace logging by default
The issue that motivated adding this (frames unexpectedly being dropped by the
decoder) has been addressed, so we can turn off the logging to reduce
unnecessary allocations during transformation. We can easily turn on debug
logging in future as needed by setting `DebugTraceUtil.DEBUG = true`.

Also avoid allocations for string building at logging call sites by passing a
format string for extra info. Formatting the string now only happens when
debugging is turned on.

Tested manually by running transformations in the new state (DEBUG = false) and
with debugging turned on.

PiperOrigin-RevId: 585666349
2023-11-27 08:57:30 -08:00
andrewlewis
e84a13fb54 Restrict operating rate workaround to SM8550
PiperOrigin-RevId: 585613041
2023-11-27 04:42:42 -08:00
andrewlewis
8b38b34b9f Work around codec frame rate issues in Redmi Note 9 Pro
The decoder and encoder won't accept high values for frame rate, so avoid
setting the key when configuring the decoder, and set a default value for the
encoder (where the key is required).

Also skip SSIM calculation for 4k, where the device lacks concurrent decoding
support.

PiperOrigin-RevId: 585604976
2023-11-27 04:05:44 -08:00
andrewlewis
ad40db4489 Avoid value close to overflow for KEY_OPERATING_RATE
Using `Integer.MAX_VALUE` risks causing arithmetic overflow in the codec
implementation.

Issue: androidx/media#810

#minor-release

PiperOrigin-RevId: 585104621
2023-11-24 08:57:59 -08:00
samrobinson
aef2fcb900 Plumb EditedMediaItem for audio preview.
PiperOrigin-RevId: 584865962
2023-11-23 05:13:32 -08:00
samrobinson
1524d12634 Avoid using unneeded double ended queue.
These usages have no need for the double ended input functionality. All
other usages across media3 are ConcurrentLinkedQueue.

PiperOrigin-RevId: 584841104
2023-11-23 02:51:20 -08:00
tofunmi
2d77e4d22c Implement trim optimization in Transformer
PiperOrigin-RevId: 584622392
2023-11-22 07:28:34 -08:00
claincly
ad96ca3b30 Make vararg APIs more robust
Composition and EditedMediaItemSequence don't allow empty lists in their main
constructors, so shouldn't the vararg API. This is more inline with Effective
Java item 53.

PiperOrigin-RevId: 583415124
2023-11-17 09:51:39 -08:00
andrewlewis
7b9aa87344 Allow allocating more buffers when transmuxing
When transmuxing, the `EncodedSampleExporter` maintains a queue of input
buffers that get filled with encoded data by the asset loader. The number of
buffers was limited to avoid using more and more memory if producer (asset
loader) gets far ahead of the consumer (exporter).

Previously this limit was fixed at 10 buffers, but increasing the number of
buffers can make some transmux operations much faster. Allow allocating between
a min and max number of buffers, and also set a target allocation size beyond
which new buffers can't be allocated. This allows audio formats which require
many small buffers to be processed more quickly, while preventing allocating
too much memory for hypothetical very high bitrate formats.

'Remove video' edits on local videos in particular get much faster, because
audio buffers are very short and there are lots of them. With a sample 10
minute video, a 'remove video' edit took 2 seconds (36 seconds before this
change).  With a sample 1 minute removing video took 0.25 seconds after this
change (2.5 seconds before).

The speed improvement is smaller for other types of edits that retain the video
track. Transmuxing a 10 minute video retaining the video track took 26 seconds
(40 seconds before).

PiperOrigin-RevId: 583390284
2023-11-17 08:11:25 -08:00
samrobinson
d5fbf0007b Migrate to Util.durationUsToSampleCount in transformer audio.
PiperOrigin-RevId: 582700443
2023-11-15 09:15:10 -08:00