The new asset loader is a wrapper around a provided `MediaProjection` and
`SurfaceAssetLoader`.
The test relies on automating the UI to get a `MediaProjection` instance, which
doesn't work reliably on all devices, so it's restricted to Pixel devices from
API 29 onwards.
For now only video is supported. The plan is to add support for audio playback
capture (also using `MediaProjection`) in a later change(s).
PiperOrigin-RevId: 725241962
Parsing AV1 bitstreams allows us to identify frames that are
not used as reference, and improve seeking or frame dropping
behavior.
The AV1 bitstream format is relatively quick to parse
PiperOrigin-RevId: 723462680
Seeking will reset media periods following the current playing period. This includes resetting any current pre-warming media periods. Therefore while seeking, any current pre-warming should be disabled and reset.
#cherrypick
PiperOrigin-RevId: 723439145
When sample batching is disabled, copying of the ByteBuffer data
is not necessary as samples are written as they arrive.
Copying of the BufferInfo is necessary because the info is needed
for writing the moov atom.
The input ByteBuffer can be in little endian order, or have its
position set. AnnexBUtils now ensures big endian order before
inspecting bytes, and supports reading from a non-zero position.
This change reduces the amount of memory allocations by Mp4Muxer
in its default configuration
PiperOrigin-RevId: 723401822
This means we need convert some of the assertions in
ClippingMediaPeriod to contrain the output value to clipped
range instead, because unseekable media will return zero
as a start and seek position in all cases.
PiperOrigin-RevId: 721463824
The exception occurred when an edit list started at a non-sync frame with no preceding sync frame. The fix searches forward for the next sync frame in such cases, preventing the out-of-bounds access.
Issue: androidx/media#2062
#cherrypick
PiperOrigin-RevId: 720642687
The file in Issue: androidx/media#2052 contains a cue with the following timecode:
```
0:00:00:00,0:00:00:00
```
The content of this cue seems to be some 'converted by' metadata, i.e.
it's basically a comment and clearly not intended to be shown on
screen (since it has zero duration).
There is some fiddly logic later in `SsaParser` to support overlapping
cues with the old `Subtitle` structure [1], and this logic gets tripped
up by the start and end time being equal, which results in a
**single**, empty `List<Cue>` being added - which trips up another
assumption that every SSA cue line results in at least two `List<Cue>`
entries (one containing the cue text, and another containing an empty
list to signal the end of the cues).
This fiddly logic is no longer required, because overlapping
`CuesWithTiming` objects can now be merged in `TextRenderer`, so there
is a possible future simplification to `SsaParser` which removes a lot
of this complexity.
[1] Added in <unknown commit>
PiperOrigin-RevId: 718380386
This requires some additional state handling to update the full
state atomically and guess placeholder states while updates are
in progress, using the newly added BackgroundThreadStateHander.
Some tests also needed to be adjusted to account for the fact
that the actual audio system change doesn't happen inline
anymore.
PiperOrigin-RevId: 716702141
This is pre work required to remove `Muxer.java` interface
from the muxer module.
`Mp4Muxer` and `FragmentedMp4Muxer` will no longer implement
the `Muxer` interface.
PiperOrigin-RevId: 716669531
To receive multiple schemes of metadata emitted by a stream, multiple
`MetadataRenderer` instances need to be used. This change makes
`DefaultRenderersFactory` add two metadata renderers by default.
PiperOrigin-RevId: 715790821
This is aligned with the documentation of `MediaCodec` which says the
timestamp of a buffer with `BUFFER_FLAG_END_OF_STREAM` should be
ignored:
https://developer.android.com/reference/android/media/MediaCodec#end-of-stream-handling
Add a test that exercises this by clipping off the end of a sample with
CEA-608 captions, because this creates an EOS-flagged buffer with a
non-EOS timestamp.
Also add a straightforward playback test for the
`fragmented_captions.mp4` sample.
PiperOrigin-RevId: 715716036
Currently, if a seek occurs when the audio sink was providing back pressure, the audio renderers will provide a durationToProgress based on the last non-writable sample prior to the seek.
Solution is to reset the value used for the `getDurationToProgress` when onPositionReset occurs.
This CL also makes sure to reset the value whenever the audiosink is flushed or reset.
PiperOrigin-RevId: 715361834
It captures the information needed for the UI logic related to rendering of the video track (and later images) to the surface.
The video size will be correct only after the Player decoded the video onto the surface and can't be reliably extracted from MediaItem's metadata. The information about the video's true aspect ratio helps inform the UI elements (such as PlayerSurface Composable) how to customize the Modifiers.
Use this state in demo-compose to show off functionality of changing `PlayerSurface`'s aspectRatio
PiperOrigin-RevId: 714104260
When the current MediaMetadata is automatically derived from the
MediaItem and Tracks, the result may change when state.buildUpon()
is used and new input data is provided (e.g. new current index).
To avoid reusing the previously derived MediaMetadata, we need to
reset it to null and re-evaluate it on every call to build()
Issue: androidx/media#1940
PiperOrigin-RevId: 714021424
The initial discontinuity is currently only reported if
the period is prepared at the clip start position. However,
we need the discontinuity whenever we prepare at a non-zero
position (unless we know all samples are sync samples).
PiperOrigin-RevId: 713994155
ExoPlayer assumed 4-bytes for length in two places (by assuming the
length is the same as the 4-byte NAL start code):
1. In `AvcConfig` we transform length-delimited to start-delimited
before writing into `initializationData`, and then skip
'nal unit length field' bytes when parsing from `initializationData`
(when we should skip 'start code length' bytes instead).
2. In `Mp4Extractor.readSample` we modify the local variable
`sampleSize` to fix the difference between length field length and
start code length, but **only on the first attempt to read a
sample**. If we are resuming in the middle of reading a sample (after
a recoverable I/O error), this fix for `sampleSize` is not done,
which means we end up missing the last 2-3 bytes of the sample when
the NAL length is 1-2 bytes.
* This is fixed by moving the `sampleSize` 'fixing' code to outside
the `if (sampleCurrentNalBytesRemaining == 0)` block.
* `FragmentedMp4Extractor` has very similar code, but uses a
field for `sampleSize`, rather than a local, so doesn't look
vulnerable to the same problem (though I haven't totally
tested this).
This change adds a test file with 2-byte NAL lengths, generated by
hacking the media3 muxer to emit 2-byte NAL lengths and transforming
`sample.mp4` using the transformer demo app.
PiperOrigin-RevId: 713709203
If `experimentalSetEnableMediaCodecVideoRendererPrewarming` is set with `true`, `DefaultRenderersFactory` will provide `ExoPlayer` with a secondary `MediaCodecVideoRenderer` for use in pre-warming through its `createSecondaryRenderer` and `buildSecondaryVideoRenderer` implementations. `ExoPlayer` will then pre-process video of subsequent media items to reduce media item transition latency.
PiperOrigin-RevId: 713605911
DASH periods can have a duration that is less than the end of the
last chunk in the period. In these cases, the sample data needs to
be clipped to the declared period duration. This already happens
IF the period duration is known at the point where we start loading
the media chunk. However, if the duration becomes known later or is
reduced (e.g. in a live stream), the existing media chunks are not
clipped. This causes unclean transitions across periods where the
player tries to transition to the next period, but renderers struggle
to output all the remaining surplus samples that should have been
clipped.
This can be fixed by asking ChunkSampleStream to discard surplus
samples that were loaded beyond a clipped duration when evaluating
the sample queue between chunk loads.
Issue: androidx/media#1698
PiperOrigin-RevId: 713288221
It captures some information needed for the UI logic related to rendering of the video track (and later images) to the surface.
Supporting `EVENT_RENDERED_FIRST_FRAME` helps improve the UX by covering the surface with an overlay (scrim/shutter) until the first frame is ready. This helps avoid sudden flickering during MediaItem transitions.
PiperOrigin-RevId: 712889568
We previously parsed an arbitrary number of decimal places, but assumed
the value was in milliseconds, which doesn't make sense if there is
greater or fewer than 3. This change restricts the parsing to match
exactly 3, meaning the millisecond assumption is always true.
The WebVTT spec requires there to be exactly 3 decimal places:
https://www.w3.org/TR/webvtt1/#webvtt-timestamp
The SubRip spec is less clearly defined, but the Wikipedia article
defines it as having exactly 3 decimal places
(https://en.wikipedia.org/wiki/SubRip#Format) and ExoPlayer has always
assumed 3 decimal places (anything else is already handled incorrectly),
so this change just ensures we don't show subtitles at the wrong time.
Issue: androidx/media#1997
PiperOrigin-RevId: 712885023
For the [0.5; 1) speed range, the combination of having a "slow down"
speed (i.e. more output frames than input frames), and Sonic potentially
needing to copy more input frames that are available in the input buffer
can lead to an unexpected underflow.
Specifically, the underflow happens in Sonic#queueEndOfStream() when the
following conditions are met (skipping some minor ones):
1. `inputFrameCount < remainingInputToCopyFrameCount`
2. `0.5f <= speed < 1`.
3. `outputFrameCount <
(inputFrameCount / remainingInputToCopyFrameCount) / 2`.
This underflow caused `SonicAudioProcessor#isEnded()` to return a false
negative (because `getOutputSize() != 0`), which would stall the
`DefaultAudioSink` waiting for processing to end after EOS.
In practical terms, the underflow is relatively easy to reproduce if we
consume all of Sonic's output and then immediately queue EOS without
queueing any more input in between. This should cause both
`inputFrameCount` and `outputFrameCount` to drop to 0.
PiperOrigin-RevId: 711773565
Added a new `usePlatformDiagnostics` in Transformer. This parameter enables/disables forwarding editing events and performance data to the platform.
This is pre-work for metrics support in Transformer. In the following CLs, metrics collection and forwarding will be implemented.
PiperOrigin-RevId: 707930368
