20942 Commits

Author SHA1 Message Date
ibaker
99679645fc Add a test for MediaCodecRenderer handling of IllegalStateException
This is a regression test for the bug introduced in bb9ff30c3a
which was manually spotted and fixed in 0d2bf49d6a.

Reverting the fix causes this test to fail.

This test is a bit hacky because we have to munge the stack trace of
the `IllegalStateException` to make it look like it was thrown from
inside `MediaCodec`. We deliberately do this 'badly' (e.g. using
`fakeMethod`) to avoid a future reader being confused by a
fake-but-plausible stack trace.

PiperOrigin-RevId: 652820878
2024-07-16 06:47:31 -07:00
tonihei
d4c6e39dfb Further unapplied rotation clean-up
Now the value is guaranteed to be zero (see bb9ff30c3a), we can
remove the rotation handling for it in the UI module. We can also
enforce the documentation more clearly by not even setting the
value to anything other than zero.

PiperOrigin-RevId: 652772091
2024-07-16 03:35:46 -07:00
ktrajkovski
104fcc1c76 Add skeleton of the IAMF JNI wrapper for the native decoder in libiamf.
PiperOrigin-RevId: 652761237
2024-07-16 02:54:21 -07:00
rohks
c60baabb1c Allow changing SNTP client timeout
Added a method to set the timeout for the SNTP request.

Also changed the default timeout to 5s instead of 10s as it seemed quite high.

Issue: androidx/media#1540
PiperOrigin-RevId: 652566008
2024-07-15 12:36:30 -07:00
tonihei
0d2bf49d6a Partially revert MediaCodecException detection
The change in bb9ff30c3a removed the detection util completely,
but even on API21+, the codec exceptions can be thrown as
IllegalStateException and we should reinstate this check to
correctly classify the exceptions.

PiperOrigin-RevId: 652557091
2024-07-15 12:06:22 -07:00
tianyifeng
ec1954c1d5 Release all ListenerHolders when clearing ListenerSet
When removing one listener from the `ListenerSet`, we release the corresponding `ListenerHolder`, which prevents the event queued or sent later than the removal from being invoked. We should also do this in the method `ListenerSet.clear()` where every listener is removed.

PiperOrigin-RevId: 652535216
2024-07-15 10:59:13 -07:00
sheenachhabra
d747f38f59 Skip TransformerPauseResumeTest on vivo 1901
The process crashes unexpectedly on vivo devices.
When test is run individually it completes successfully.

PiperOrigin-RevId: 652496852
2024-07-15 08:56:37 -07:00
ibaker
bb9ff30c3a Remove dead code related to MediaCodec now minSdk is 21
This removes several workarounds that are no longer needed, including
`codecNeedsMonoChannelCountWorkaround` which has been permanently
disabled since the (incomplete) minSdk 19 clean-up in fb7438378d.

PiperOrigin-RevId: 652495578
2024-07-15 08:51:35 -07:00
Googler
1bb8d5f956 Update internal reference.
PiperOrigin-RevId: 652448337
2024-07-15 05:34:32 -07:00
dancho
bfe4824bfd Rollback of 9151dbf9e6
PiperOrigin-RevId: 652443920
2024-07-15 05:14:58 -07:00
Googler
11ca78761e Add support for 3gpp amr-nb audio codec.
To support AMR audio codec(audio/3gpp) add `0x81FF` mode to create damrBox.

Add unit test and an Android end to end test.

PiperOrigin-RevId: 652438693
2024-07-15 04:48:28 -07:00
tofunmi
4da1e26206 Take effects in account when calculating presentationDurationUs
PiperOrigin-RevId: 652425099
2024-07-15 03:40:59 -07:00
andrewlewis
4b7cc80593 Handle muxing with timestamps offset from zero in wrapper
Sources (for example media projection) can populate the `Surface` from
`SurfaceAssetLoader` with timestamps that don't start from zero. But
`MuxerWrapper` assumes the latest sample timestamp can be used as the duration
when it calculates average bitrate and notifies its listener.

This can cause a crash because the calculated average bitrate can be zero if
the denominator duration is large enough.

Use the max minus first sample timestamp across tracks instead to get the
duration.

Side note: the large timestamps from the surface texture when using media
projection arrive unchanged (apart from conversion from ns to us) in effect
implementations and in the muxer wrapper (and are passed to the underlying
muxer). The outputs of media3 muxer and the framework muxer are similar.

PiperOrigin-RevId: 652422674
2024-07-15 03:30:26 -07:00
tonihei
c510ab81bb Update compileSdk to 35
This should have no influence on app behavior and other policies
and just allows code to depend on new API 35 platform symbols.

PiperOrigin-RevId: 652414026
2024-07-15 02:45:55 -07:00
ibaker
268c8cf6a2 Remove dead code from MediaStyleNotificationHelper now minSdk is 21
Tested using session demo app on API 21 emulator, and checked
notification layout looked correct.

PiperOrigin-RevId: 652403407
2024-07-15 01:56:37 -07:00
tonihei
e96ca5a242 Decouple displaySurface from placeholderSurface
We currently use displaySurface == placeholderSurface IF codec != null
as a signal that the codec is configured with a placeholder. In the
future, this placeholder might not be needed and we can decouple this
state a bit better by leaving displaySurface == null in this case and
only using placeholderSurface instead of null when setting a Surface
to the codec.

PiperOrigin-RevId: 652391729
2024-07-15 01:04:48 -07:00
ibaker
d035b745cd Remove dead code from ExoPlayerImpl now minSdk is 21
PiperOrigin-RevId: 651815091
2024-07-12 10:27:49 -07:00
ibaker
6a9ff95bf0 Bump minSdk to 21 and remove resulting simple dead code
All other AndroidX libraries have already increased their min SDK to
21.

This change renames private symbols to remove `V21` suffixes and
similar, but doesn't change public or protected symbols with similar
names, to avoid needless breakages/churn on users of the library.

Some of the dead code removal is more complex, so I've split it out
into follow-up changes to make it easier to review.

PiperOrigin-RevId: 651776556
2024-07-12 08:11:01 -07:00
ibaker
5fa9985ce6 Add H264_ prefix to NalUnitUtil.NAL_UNIT_TYPE_* constants
Also promote all H.265 constants to be public in `NalUnitUtil` with
`H265_` prefixes, for consistency.

A lot of these names are used in h.265 too (and `NalUnitUtil` handles
both), but with different values, so this rename aims to avoid
accidentally using an h.264 value in an h.265 context.

PiperOrigin-RevId: 651774188
2024-07-12 08:02:24 -07:00
ibaker
2c7f2686b7 Remove stray parenthesis from MediaSession.ControllerInfo.toString()
#cherrypick

PiperOrigin-RevId: 651760391
2024-07-12 06:58:33 -07:00
dancho
9151dbf9e6 Avoid decoder input skipping close to media end duration
MediaCodecRenderer might not be able to determine whether a sample
is last in time. Do not use decoder input skipping optimization
for buffers close to media end duration in order to ensure
last frame is rendered.

PiperOrigin-RevId: 651757814
2024-07-12 06:44:51 -07:00
dancho
09239a8a55 Add tests for releasing DefaultVideoFrameProcessor output surface
Add parametrized tests for DefaultVideoFrameProcessor that ensure
output EGL surface can be released, and deallocated.

PiperOrigin-RevId: 651712624
2024-07-12 03:07:20 -07:00
Googler
4c4e24db60 Recognize QC's MV-HEVC decoder.
PiperOrigin-RevId: 651539516
2024-07-11 14:40:39 -07:00
tianyifeng
7aa70a5f2f Add Listener to BasePreloadManager to propagate preload events to apps
PiperOrigin-RevId: 651421044
2024-07-11 08:36:41 -07:00
Googler
735e0cf8a1 Add an MV-HEVC test to Mp4ExtractorParameterizedTest.
PiperOrigin-RevId: 651392787
2024-07-11 06:52:15 -07:00
claincly
1ba2d98fce Remove extra comment line
PiperOrigin-RevId: 651367551
2024-07-11 05:06:17 -07:00
claincly
81f5a5f5f3 Set a longer muxer timeout on emulators
This is because the force EOS workaround for videos is longer than the original
muxer timeout. This way the apps depending on Transformer won't have to manually
set the muxer timeout on emulators.

PiperOrigin-RevId: 651355755
2024-07-11 04:12:59 -07:00
ktrajkovski
b7f141ad2a Replace sample mp4 files related to IAMF and update the dump files.
PiperOrigin-RevId: 651347968
2024-07-11 03:41:56 -07:00
ibaker
0ea555dae0 Stop using SubtitleTranscodingExtractor and deprecate it
The integration with `SubtitleTranscodingExtractorOutput` has been
moved inside the relevant `Extractor` implementations instead.

PiperOrigin-RevId: 651213564
2024-07-10 18:09:53 -07:00
ibaker
240b6fd606 Fix 'duplicate class' error caused by incomplete kt stdlib metadata
When testing locally it seemed that upgrading KGP to `1.9.20` resolved
this, but testing again confirms this was incorrect, so the problem
was reintroduced in 00d1e70a34.

This change reverts back to KGP `1.9.0` (since upgrading didn't help)
and instead implements the workaround suggested in
https://issuetracker.google.com/278545487. The workaround is now in
the `lib-common` `build.gradle` file, which is transitively depended
on by all modules (and therefore all apps that use this library), so
there's no need for any developer action, and so the release note is
removed.

Issue: androidx/media#1262
PiperOrigin-RevId: 651066418
2024-07-10 10:20:12 -07:00
kimvde
21992bff33 Call getFrameReleaseAction from VideoSink when enabled
VideoSink.registerInputFrame is now called for every input frame (not
only the ones that should be rendered to the input surface) because it's
the VideoSink that decides whether it wants the frame to be rendered.

PiperOrigin-RevId: 651049851
2024-07-10 09:33:11 -07:00
dancho
0ff9e0723d Expose fMP4 FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES to DASH
In order for DASH playback to benefit from
FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES, the fMP4 extractor flag
must be set. The smallest API change that allows this is to add an
experimental method to BundledChunkExtractor.

Add a dash end-to-end test to verify that video frames are skipped at
decoder input.

PiperOrigin-RevId: 651046676
2024-07-10 09:23:35 -07:00
ibaker
c64dacf3df Transform double-tap of HEADSETHOOK to skip-to-next
As reported in Issue: androidx/media#1493 we already use `HEADSETHOOK` to start
waiting for a double-tap, but we don't consider it for the second tap.

Similarly, we previously considered `PLAY` for the second tap, but not the first.
Only `HEADSETHOOK` and `PLAY_PAUSE` are
[documented](https://developer.android.com/reference/androidx/media3/session/MediaSession#media-key-events-mapping)
to transform double-tap to `seekToNext`. So this change removes the
`PLAY` detection from the second tap.

#cherrypick

PiperOrigin-RevId: 651017522
2024-07-10 07:52:18 -07:00
tonihei
e8778d77fa Document Renderer and BaseRenderer in more detail
PiperOrigin-RevId: 651008509
2024-07-10 07:24:02 -07:00
Googler
f673ef43b4 Add support for SEI and vexu box parsing.
Stereo view information is stored in the 3D reference displays information SEI and the optional vexu box.  Parsing of the SEI and vexu box is added, and based on the parsed info, proper mapping of primary/secondary view to left/right eye is determined.

PiperOrigin-RevId: 651002190
2024-07-10 07:00:29 -07:00
Googler
34a802ef38 Add support to MPEG4 codec in Mp4Muxer.
Add support for MPEG4 codec to enable muxing video encoded with the mp4v-es codec. Use esdsBox method to generate esds box required for Mp4v box.

PiperOrigin-RevId: 651000744
2024-07-10 06:55:00 -07:00
Googler
cf90d2624d Support for Large CodecSpecificData in ESDS box
Some external media files have CodecSpecificData greater than 128 bytes. Currently, that size
isn't fitting in one byte. Hence, added support to store large CodecSpecificDataSize, as per
ISO standard, by extending to more than one byte as required.

PiperOrigin-RevId: 650972472
2024-07-10 05:16:01 -07:00
tonihei
9d4e43cf55 Support multiple DataSource configurations in DataSourceContractTest
PiperOrigin-RevId: 650967939
2024-07-10 05:01:33 -07:00
ibaker
a202fd0c9c Add parameter comments in ReorderingSeiMessageQueueTest
PiperOrigin-RevId: 650959556
2024-07-10 04:34:24 -07:00
dancho
70a6b5d50d Destroy eglSurface as soon as Surface changes
eglDestroySurface now unbinds surface from the GL
thread to ensure quick release of resources.

PiperOrigin-RevId: 650938712
2024-07-10 03:21:48 -07:00
andrewlewis
dcc3e439e2 Handle AVI containers with no keyframes in index
If there is no keyframe in the index assume that the first chunk has a
keyframe.

PiperOrigin-RevId: 650915467
2024-07-10 01:51:06 -07:00
tonihei
b145a9b35e Limit maximum number of parallel metadata retrievals.
The operation almost always acquires resources straight-away,
for example a new thread or file access. This means starting
many metadata retrievals in parallel easily causes resource
contention. This problem can be alleviated by limiting the
maximum number of parallel retrievals.

Local testing showed a 20% speedup for local file retrievals
when limited to 5 in parallel. Any more parallel retrievals
did not show any improvement or started getting slower again
the more operations are allowed.

PiperOrigin-RevId: 650674685
2024-07-09 10:25:20 -07:00
tonihei
5777e30979 Reuse thread in MetadataRetriever
MetadataRetriever is often used in a loop to retrieve data about many
items in a directory for example. In such cases, it's wasteful to
start a new 'playback' thread for each retrieval, in particular since
this thread is doing almost nothing (just triggering loads and handling
events). This change re-uses an existing thread if one exists already.

PiperOrigin-RevId: 650633343
2024-07-09 08:15:45 -07:00
ibaker
00d1e70a34 Rollback of 91633e6ae3
PiperOrigin-RevId: 650620012
2024-07-09 07:29:57 -07:00
claincly
60359c16da Add a new debug trace log to log the device name / sdk
PiperOrigin-RevId: 650567544
2024-07-09 03:48:52 -07:00
dancho
6650270a4e MediaCodecVideoRenderer skips decoder inputs unused as reference
During a seek, or when playing a media with clipped start,
MCVR encounters preroll decode-only buffers that are not rendered.
Use C.BUFFER_FLAG_NO_OTHER_SAMPLE_DEPENDS_ON_THIS to determine
whether a decode-only buffer is unused as reference.
These buffers can be dropped before the decoder.
When this optimization is triggered, increment
decoderCounters.skippedInputBufferCount.

Tested in ExoPlayer demo app on "One hour frame counter (MP4)"
after enabling extractorsFactory.setMp4ExtractorFlags(
    FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES);
Observe: "sib" increases on each seek.
PiperOrigin-RevId: 650566216
2024-07-09 03:43:06 -07:00
dancho
7d4f623b00 Fix RELEASENOTES merge conflict
PiperOrigin-RevId: 650551156
2024-07-09 02:43:03 -07:00
dancho
439536480b Parse the H264 bitstream of fMP4 files to identify sample dependencies
Changes to FragmentedMp4Extractor to parse additional sample dependency
information and mark output samples as "no other samples depend on this".
Only applies to H.264 tracks.
Controlled by new fMP4 flag: FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES

PiperOrigin-RevId: 650538377
2024-07-09 01:53:51 -07:00
tonihei
2bb719fd54 Remove obsolete TODO
The idea was to not even write any samples to SampleQueues as
they are not needed during the metadata extraction process.
However, this is not easily possible as long as we use our
existing Extractors and MediaSource/Periods for loading given
how deeply ingrained the extraction of samples is in these
classes. For most common formats like MP4, no samples will be
extracted anyway as they can finish the prepare step without
reading any samples.

PiperOrigin-RevId: 650529371
2024-07-09 01:18:01 -07:00
andrewlewis
74c06dc2f4 Add SurfaceAssetLoader
This supports queueing input to Transformer via a `Surface`.

PiperOrigin-RevId: 650318396
2024-07-08 11:33:05 -07:00