19093 Commits

Author SHA1 Message Date
andrewlewis
9a2bfc4042 Prioritize H.265 and H.264 over other formats
This should make no functional difference because `SampleExporter` always
checks for H.265 and H.264 first. However, in case we ever change that code,
these are used in priority order so it's better to order them accordingly.

PiperOrigin-RevId: 550894935
2023-08-01 14:05:42 +01:00
tonihei
79c8b0e0f8 Set correct start time for newly enabled renderers during playback
We currently pass in the time at which the stream originally started,
but newly enabled renderers should get the current playback position
instead.

PiperOrigin-RevId: 550894630
2023-08-01 14:04:35 +01:00
samrobinson
54093a152e Integrate AudioMixer for audio export.
Adds support for compositions with multiple audio sequences.

PiperOrigin-RevId: 550880626
2023-08-01 14:03:22 +01:00
samrobinson
357c458028 Ensure EOS is queued after processing generated silence with effects.
When generating silence for AudioProcessingPipeline, audio never
queued EOS downstream.

Linked to this, when silence followed an item with audio, the silence
was added to SilentAudioGenerator before the mediaItem reconfiguration
occurred. If the silence had effects, the APP would be flushed after
silence queued EOS, resetting APP.isEnded back to false, so AudioGraph
never ended.

Regression tests reproduce failure without fix, but pass with it.

PiperOrigin-RevId: 550853714
2023-08-01 14:02:09 +01:00
huangdarwin
35bee3299c Remove VisibleForTesting visibility in 8k workaround.
PiperOrigin-RevId: 550843914
2023-08-01 14:01:02 +01:00
Googler
465e7c1277 Rewrite tracing to be more flexible and provide more info.
For each event, the timestamp and presentation time is logged. The trace can
then be dumped to a tsv file and easily imported in a spreadsheet.

PiperOrigin-RevId: 550839156
2023-08-01 13:59:51 +01:00
bachinger
71af48429f Fix flaky test in MediaControllerListenerTest
PiperOrigin-RevId: 550625956
2023-08-01 13:58:49 +01:00
ibaker
449cf55523 Add comment about out-of-order timestamps in AtomParsers.parseStbl
PiperOrigin-RevId: 550596173
2023-08-01 13:57:36 +01:00
huangdarwin
e5e3db954f Remove setting the interrupted flag when unneeded.
When rethrowing InterruptedException, this
Thread.currentThread().interrupted() is not needed.

PiperOrigin-RevId: 550586853
2023-08-01 13:56:35 +01:00
huangdarwin
ef39d317d5 Compositor: Signal onEnded when frames finish compositing.
PiperOrigin-RevId: 550576032
2023-08-01 13:55:21 +01:00
bachinger
a90ec96425 Use MediaUtils.intersect in MediaControllerImplBase
PiperOrigin-RevId: 550566322
2023-08-01 13:54:11 +01:00
bachinger
2c19399d3e Move command button related helper methods to CommandButton
PiperOrigin-RevId: 550558261
2023-08-01 13:53:09 +01:00
claincly
22d30b3d74 Fix some mal formatted TODOs
PiperOrigin-RevId: 550507208
2023-08-01 13:51:58 +01:00
jbibik
f4ad26451f Deprecate MimeTypes.TEXT_EXOPLAYER_CUES for APPLICATION_MEDIA3_CUES
Instead of `text/x-exoplayer-cues`, we will use `application/x-media3-cues`.

The prefix should be `application` not `text` since the encoded form is arbitrary bytes, not necessarily text. The name should not reference `exoplayer`, since the Media3 Extractors (which are not part of `exoplayer`) produce this format.

PiperOrigin-RevId: 550181852
2023-08-01 13:50:50 +01:00
huangdarwin
d7b5ab9f23 Add Thread.currentThread().interrupt() for InterruptedExceptions.
PiperOrigin-RevId: 549975878
2023-08-01 13:49:48 +01:00
huangdarwin
5858723a06 Effect: Add Compositor signalEndOfInputStream and onEnded.
signalEndOfInputStream is needed for when streams have different amounts of
frames, so that if the primary stream finishes after a secondary stream, it
can end without waiting indefinitely for the secondary stream's matching
timestamps.

onEnded mirrors this API on the output side, which will be necessary to
know when to call signalEndOfInput on downstream components (ex. on downstream)
VideoFrameProcessors

PiperOrigin-RevId: 549969933
2023-08-01 13:48:36 +01:00
sheenachhabra
c2615a679f Skip tests if device needs disable 8k workaround
PiperOrigin-RevId: 549967326
2023-08-01 13:47:26 +01:00
huangdarwin
b1cf5fb320 Effect: Reduce Compositor exception try/catch redirection.
The VideoFrameProcessingTaskExecutor already handles wrapping
GlUtil.GlException errors into VideoFrameProcessingExceptions.
Therefore, stop wrapping this ourselves, so that errors are
attributed to the correct line number with less indirection.

PiperOrigin-RevId: 549959258
2023-08-01 13:46:28 +01:00
tofunmi
7eee15ecb4 Update OverlaySettings to explicitly state overlay transformations
Alters the OverlayShaderProgram implementation to support rotations, however we need to apply the transformations separately in order for them to work as expected so the matrix is removed from the interface in favour of explicit methods.

Adds a rotation test to ensure this ability doesn't regress

PiperOrigin-RevId: 549890847
2023-08-01 13:45:22 +01:00
bachinger
d658de5944 Mark media notification controller and filter command buttons
The `MediaNotificationManager` registers an internal controller
to each session. This change marks this controller through its
connection hints and provides an API for apps to hide
implementation details of the marking.

Issue: androidx/media#389
PiperOrigin-RevId: 549712768
2023-08-01 13:44:12 +01:00
jbibik
25253698bc TtmlParser implementation - moved from TtmlDecoder
`TtmlDecoder` which used to be `SimpleSubtitleDecoder` will now be called `TtmlParser` and implement SubtitleParser interface. For backwards compatibility, we will have the same functionality provided by `DelegatingSubtitleDecoder` backed-up by a new `TtmlParser` instance.

PiperOrigin-RevId: 549700490
2023-08-01 13:43:12 +01:00
jbibik
1c1b78e635 Test for Decoder with Mp4WebvttParser corrected
PiperOrigin-RevId: 549629289
2023-08-01 13:42:06 +01:00
jbibik
e11e484bc3 Do not emit empty cues in SsaParser
It is not part of the `SubtitleParser` interface's promise to give events for all changes to do with `Cue` objects. So while in the past, we would have `endTimeUs` of one Cue event being the `startTimeUs` of the next one, now we have that encoded in `durationUs` and can skip event with empty `Cues`.

PiperOrigin-RevId: 549629157
2023-08-01 13:41:11 +01:00
tonihei
85cde93e6c Remove C2Mp3TimestampTracker
This tracker aims to replicate the behavior of a specific codec to
ensure MediaCodecRenderer correctly detects stream and output
format transitions. The class was needed because MediaCodecRenderer
made assumptions about codec behavior this codec did not fulfil (in
particular, changing timestamps and number of samples).

Since then, MediaCodecRenderer was made more robust to this kind of
codec behavior in general and currently has no assumptions that
require any special handling of this codec. This means we can remove
the workaround completely.

PiperOrigin-RevId: 549610989
2023-08-01 13:39:53 +01:00
jbibik
27bda610aa Standardise the use of ParsableByteArray use among SubtitleParsers
PiperOrigin-RevId: 549609028
2023-08-01 13:38:38 +01:00
huangdarwin
9128293236 Test: Use correct test skipping in muxer androidTest.
PiperOrigin-RevId: 549601882
2023-08-01 13:37:31 +01:00
simakova
3d0d5f5c95 Update README file
Add a link to the media dev center and add Transformer to the list of libraries

PiperOrigin-RevId: 549598105
2023-08-01 13:36:21 +01:00
ibaker
1cfaae3450 Enable WebVTT subtitles in MkvPlaybackTest
This is no longer flaky because WebVTT subtitles are decoded on the
loading thread (enabled in this test with
`defaultExractorsFactory.setTextTrackTranscodingEnabled(true)` - and
supported for WebVTT since f0f24aa0d4).

PiperOrigin-RevId: 549594291
2023-08-01 13:35:11 +01:00
tonihei
b4860fada0 Document MediaCodecRenderer's stream and codec behavior assumptions
The class tries to be flexible to support as many different input
and codec behavior combinations as possible. But so far it didn't
spell out its remaining assumptions and explicit non-assumptions,
making it hard to know which behavior to rely on.

PiperOrigin-RevId: 549589347
2023-08-01 13:34:02 +01:00
ibaker
1ccedf8414 Tighten the demo app's handling of DrmInitData for downloads
This code is Widevine specific. `OfflineLicenseHelper.downloadLicense`
requires the passed `Format` to have a `DrmInitData.SchemeData` with
Widevine UUID and non-null `data` field. The demo app tries to check
this in advance (to avoid an exception later), but its checks are
looser than those made by `OfflineLicenseHelper`. This change tightens
the checks to match.

Issue: androidx/media#512
PiperOrigin-RevId: 549587506
2023-08-01 13:31:57 +01:00
jbibik
f4c66d82dd TtmlDecoderTest refactor not to use TtmlNode or TtmlStyle
PiperOrigin-RevId: 549584918
2023-08-01 13:30:39 +01:00
samrobinson
beceb996ac Create ExportException for audio from UnhandledAudioFormatException.
All audio processing export errors are related to a UAFE - passing
audioFormat separately is redundant.

PiperOrigin-RevId: 549427685
2023-07-20 10:35:45 +01:00
rohks
cdb174c91a Add field measured throughput (mtp)
Updated `ExoTrackSelection` to provide the most recent bitrate estimate, enabling the inclusion of measured throughput (mtp) as a CMCD-Request field in Common Media Client Data (CMCD) logging.

Additionally, made changes to the `checkArgument` methods in `CmcdLog` to prevent the use of default values in certain cases.

#minor-release

PiperOrigin-RevId: 549369529
2023-07-20 10:34:32 +01:00
claincly
177187e237 Allow multiple inputs to the processing graphs
The new logic covers the old single-sequence use cases.

PiperOrigin-RevId: 549307918
2023-07-20 10:33:35 +01:00
jbibik
f0f24aa0d4 WebvttParser implementation - moved from WebvttDecoder
`WebvttDecoder` which used to be `SimpleSubtitleDecoder` will now be called `WebvttParser` and implement `SubtitleParser` interface.

For backwards compatibility, we will have the same functionality provided by `DelegatingSubtitleDecoder` backed-up by a new `WebvttParser` instance. `WebvttSubtitle` will still be used behind the scenes to handle overlapping `Cues`.

PiperOrigin-RevId: 549298733
2023-07-20 10:32:13 +01:00
ibaker
5d453fcf37 Add an Extractor to parse subtitles before SampleQueue
The end-to-end test output for the overlapping SRT and SSA subtitles
is currently incorrect. They will be fixed in a future change that
updates `TextRenderer` to support this overlap.

The 'extra' samples visible in the extractor test output files are
'empty cue list' samples produced by `SsaParser`. They will go away
when this implementation is updated to remove this behaviour and rely
on `CuesWithTiming.durationUs` instead (the 'empty list' behaviour is
not required by the `SubtitleParser` interface).

PiperOrigin-RevId: 549264593
2023-07-20 10:30:16 +01:00
tonihei
5c5e7cd257 Update storage access in main demo app
The main demo app was still targeting API 29 to avoid scoped storage
restrictions. It is now updated to 34 (like the rest of the demo apps)
and handles scoped storage as it should handle it.

More specifically:
 - We need to request READ_MEDIA_... permissions instead of
   READ_EXTERNAL_STORAGE from API33.
 - The legacy scoped storage opt-out can be removed
 - READ_MEDIA_... permissions don't allow arbitrary file access
   if the file doesn't end in a typical media file extension, so
   this change adds a remark on the guide page to place samples
   in the app-specific directory.
 - We also don't have to request permissions for the app-specific
   directories.
 - Custom json files can't be placed in arbitray local locations
   because they don't end in a media file extension, as there is
   no way we can request a permission to load them. This means we
   can remove the storage request logic in SampleChooserActivity.

Issue: google/ExoPlayer#6045
PiperOrigin-RevId: 549252474
2023-07-20 10:29:02 +01:00
jbibik
d9daa392d5 Util for Subtitle to CuesWithTiming conversion
PiperOrigin-RevId: 549027279
2023-07-20 10:27:47 +01:00
tonihei
da99f9937d Rollback of 7996766b22
*** Original commit ***

Rollback of b69b33206e

*** Original commit ***

Mark output sample as decode-only based on start time

We currently do the same check on the input timestamps and
expect the output timestamps to match. Some codecs produce
samples with modified timestamps and the logic is a lot safer
when the comparison with the start time is done on the output
side of the codec.

Issue: google/ExoPlayer#11000

***

***

PiperOrigin-RevId: 549019403
2023-07-20 10:26:25 +01:00
huangdarwin
747b31b3c5 Test: Add non-mh androidTest tests to MH_TESTS / triage.
PiperOrigin-RevId: 549005874
2023-07-20 10:25:30 +01:00
tonihei
8064c6df83 Declare foreground service type for DownloadService
This ensures the DownloadService stays functional on Android 14
where defining this type is required. On Android 14 and above,
the app also needs to define the DATA_SYNC permission, which is
added to the demo app as well. In the future, this service type
will no longer be supported and DownloadService needs to be
rewritten with another background scheduling framework.

Issue: google/ExoPlayer#11239
PiperOrigin-RevId: 548994842
2023-07-20 10:23:56 +01:00
samrobinson
035934c6d4 Combine audio mixing logic across Transformer.
AudioMixingUtil#mix handles input & output in float or Int16 PCM. Given
Float and Int16 use different sample ratnes, this util handles
conversion between the two, based on the encoding being mixed to.

Migrate AudioMixer to use the util, removing AudioMixingAlgorithm
interface and implementation. ChannelMixingAudioProcessor will be
migrated after additional performance checks.

PiperOrigin-RevId: 548994584
2023-07-20 10:22:58 +01:00
jbibik
9520180fbb DelegatingSubtitleDecoder tests refactor
PiperOrigin-RevId: 548991428
2023-07-20 10:21:43 +01:00
tonihei
b0ec2cc701 Remove timestamp clipping in MergingMediaPeriod.TimeOffsetSampleStream
This was done because it was deemed correct to only start at timestamp
zero when the code was originally written. However, in case of
prerolling from a keyframe, many samples will get the same timestamp,
which is not correct and interferes with downstream logic
that deals with timestamps.

PiperOrigin-RevId: 548986160
2023-07-20 10:20:22 +01:00
rohks
1b2a2fcde0 Add fields top bitrate(tb) and object duration(d)
Added these CMCD-Object fields to Common Media Client Data (CMCD) logging.

#minor-release

PiperOrigin-RevId: 548950296
2023-07-20 10:19:26 +01:00
ibaker
0fa66534fe Use CuesWithTiming.durationUs in SubripParser instead of empty cues
This fixes two things in one go:
1. In order to indicate 'end of a cue' **without** relying on
   `CuesWithTiming.durationUs`, `SubripParser` should have been emitting
   an empty `List<Cue>` instead of the current behaviour of a
   single-item list with `cue.text=""`.
2. There's no need for the empty cue (or cue list), we can use
   `durationUs` to indicate the end of each list of cues.

There's no real need to ever have a `Cue` with `text=""`, so also
deprecate `Cue.EMPTY`.

PiperOrigin-RevId: 548938874
2023-07-20 10:18:09 +01:00
tonihei
8655429af7 Reset loading period after reading discontinuity that requires loading
Reading a discontinuity from a media period indicates that a position
reset is required. As part of this event, the media period may need
further loading (e.g in a MergingMediaPeriod where one stream reported
a discontinuity and the other need to reload from this position).

This currently fails if the media periods was already fully loaded and
we started loading further items in the playlist. As a result, playback
is stuck forever. We can fix this by detecting that further loading is
needed and resetting the loading period to the current one.

The existing MergingPlaylistPlaybackTest already covers this case
reliably, because it combines all the right preconditions (merging
source, clipping to get a discontinuity and a playlist).

PiperOrigin-RevId: 548735177
2023-07-20 10:17:11 +01:00
Googler
7adc0af8a6 Save to TestStorage when saving Bitmap
PiperOrigin-RevId: 548733979
2023-07-20 10:15:47 +01:00
ibaker
34768c2399 Add MKV test assets with overlapping SSA and SRT subtitles
This is a step towards adding general support for overlapping
subtitles in these formats (and others), both muxed and sideloaded:
* Issue: google/ExoPlayer#10295
* Issue: google/ExoPlayer#4794

This change adds these files to the end-to-end playback tests too, but
the subtitle track is currently disabled because renderer-side subtitle
parsing causes flaky tests (due to an uncontrolled thread in
`SimpleSubtitleDecoder`). The subtitle track will be re-enabled in
a follow-up change when loading-side subtitle parsing is added (so the
tests will no longer be flaky). At this point the overlapping subtitles
**still** won't be supported end-to-end, but a second change will
resolve this will changes in `TextRenderer` - which will change the
end-to-end playback dumps to reflect the overlapping subtitles.

PiperOrigin-RevId: 548705032
2023-07-20 10:14:50 +01:00
huangdarwin
836720ebfa Compositor: Use timestamps for compositing.
For now, just identical timestamps. A follow-up CL will allow for different
framerates. Prior tests already send identical timestamp frames, so no new
tests were added

PiperOrigin-RevId: 548676759
2023-07-20 10:13:28 +01:00