549 Commits

Author SHA1 Message Date
Googler
2052be0f51 Add new setIntsUniform for setting ivec uniforms in shaders.
PiperOrigin-RevId: 567102956
2023-09-20 15:50:43 -07:00
Googler
bfd4b6a188 Move GlProgram.loadAsset to Util and make it public.
PiperOrigin-RevId: 567025091
2023-09-20 11:21:55 -07:00
tofunmi
234f7525f0 Add MimeTypes.EXTERNALLY_LOADED_IMAGE
PiperOrigin-RevId: 566915171
2023-09-20 03:55:36 -07:00
michaelkatz
3cf9c08dca Create AudioOffloadPreferences class
Move audio offload mode related interfaces and definitions from `TrackSelectionParameters` to a new `AudioOffloadModePreferences` class.

PiperOrigin-RevId: 566905017
2023-09-20 03:02:51 -07:00
ibaker
db7c33b01c Deprecate Util.getAudioContentTypeForStreamType
This method isn't used by the library (since <unknown commit>).

It doesn't really work well (e.g. arbitrarily defaults to `MUSIC` when
`UNKNOWN` would be a better default). There's no suggested replacement.

PiperOrigin-RevId: 566676744
2023-09-19 10:42:49 -07:00
jbibik
39c1ac7f3a Move setAudioAttributes from ExoPlayer to Player
PiperOrigin-RevId: 566607528
2023-09-19 06:08:37 -07:00
ibaker
2c0d9ba4c3 Rollback of d58f5fdf7d
PiperOrigin-RevId: 566258299
2023-09-18 04:12:21 -07:00
ibaker
d58f5fdf7d Upgrade Guava dependency to 32.1.2
Android's version of Guava was upgraded in http://r.android.com/2731599

PiperOrigin-RevId: 565396612
2023-09-14 09:50:10 -07:00
ibaker
f8b23da7af Add overflow tests for sample count to duration conversion methods
These methods were updated to use the new overflow-resistant
`scaleLargeValue` method in 885ddb167e.

PiperOrigin-RevId: 565059609
2023-09-13 08:38:14 -07:00
ibaker
d111976125 Change SubtitleParser interface to support incremental output
This change introduces two new types of method to `SubtitleParser`:
1. `parse()` methods that take a `Consumer<CuesWithTiming>` and return `void`
2. `parseToLegacySubtitle` method that returns `Subtitle`

(1) ensures that in the new 'parse before SampleQueue' world we can
write cues to the `SampleQueue` as soon as they're ready - this is
especially important when parsing monolithic text files, e.g. for a
whole movie.

(2) ensures that during the transition, the legacy 'parse after
SampleQueue' behaviour doesn't see any regressions in 'time to first
cue being shown'. Previously we had a single implementation to convert
from `List<CuesWithTiming>` to `Subtitle`, but this relies on the
complete list of cues being available, which can take a long time for
large files in some formats (with ExoPlayer's current parsing logic).
By allowing implementations to customise the way they create a
`Subtitle`, we can directly re-use the existing logic, so that the
'time to first cue being shown' should stay the same.

This change migrates all **usages** to the new methods, but doesn't
migrate any **implementations**. I will migrate the implementations in
follow-up CLs before deleting the old list-returning `parse()` methods.

PiperOrigin-RevId: 565057945
2023-09-13 08:33:29 -07:00
ibaker
abaf3e7aa1 Add Format.cueReplacementBehavior
Previously any `CuesWithTiming.durationUs` could be `TIME_UNSET`,
meaning it should be replaced by the next `CuesWithTiming` instance
(instead of being merged if the durations overlap, which is currently
the expected behavior for all `CuesWithTiming` with a 'real' duration).

This technically allowed a single subtitle track to include a mixture of
`CuesWithTiming` that should be merged, and some that should be
replaced. This is not actually needed for any of the subtitle formats
currently supported by ExoPlayer - in all cases a format expects either
all cues to be merged, or each cue to replace the previous one.

Supporting this mixture of merging and replacing in `TextRenderer` ended
up being very complicated, and it seemed a bit pointless since it's not
actually needed. This change means a given subtitle track either merges
**all** cues (meaning `CuesWithTiming.durationUs = C.TIME_UNSET` is not
allowed), or **every** cue is replaced by the next one (meaning
`CuesWithTiming.durationUs` may be set (to allow for cues to 'time out',
needed for CEA-608), or may be `TIME_UNSET`).

This value will be used in a subsequent change that adds cue-merging
support to `TextRenderer`.

PiperOrigin-RevId: 565028066
2023-09-13 06:09:45 -07:00
ibaker
885ddb167e Add general-purpose overflow-resistant divide+multiply util method
This is equivalent to the existing `scaleLargeTimestamp` method with the
following changes/improvements:
* No longer specific to timestamps (there was nothing inherently
  time-specific about the logic in `scaleLargeTimestamp`, but the name
  and docs suggested it shouldn't be used for non-timestamp use-cases).
* Additional 'perfect division' checks between `value` and `divisor`.
* The caller can now provide a `RoundingMode`.
* Robust against `multiplier == 0`.
* Some extra branches before falling through to (potentially lossy)
  floating-point math, including trying to simplify the fraction with
  greatest common divisor to reduce the chance of overflowing `long`.

This was discussed during review of 6e91f0d4c5

This change also includes some golden test file updates - these
represent a bug fix where floating-point maths had previously resulted
in a timestamp being incorrectly rounded down to the previous
microsecond. These changes are due to the 'some more branches' mentioned
above.

PiperOrigin-RevId: 564760748
2023-09-12 10:22:57 -07:00
ibaker
fe10f5ae18 Add tests for Util.scaleLargeTimestamp
This ensures we don't regress this behaviour when introducing
`scaleLargeValue`.

PiperOrigin-RevId: 564708566
2023-09-12 07:00:22 -07:00
michaelkatz
3742f6b1f5 Allow renderer recovery with disabling offload if failed at first write
If offload fails at first write with [ERROR_DEAD_OBJECT](https://developer.android.com/reference/android/media/AudioTrack#ERROR_DEAD_OBJECT), then try disabling offload mode and try again. This allows recovery in a state where AudioTrack succeeds init in offload mode but writing is failing.

Issue: androidx/media#627
PiperOrigin-RevId: 564402181
2023-09-11 09:04:17 -07:00
huangdarwin
19878b0b8b Effect: Clearly state what callers must do onError.
Instead of invoking "undefined behavior" may occur when a class is used
incorrectly, which is clear but not very helpful to callers, specify what Apps
or users of a class should do when issues occur.

Apply this to Compositor as well, where this otherwise might seem unclear.

PiperOrigin-RevId: 563406219
2023-09-07 06:02:09 -07:00
tofunmi
535af35511 Add heif extractor
PiperOrigin-RevId: 563340591
2023-09-07 00:31:04 -07:00
tofunmi
88b114a997 Add bmpExtractor to DefaultExtractorsFactory
PiperOrigin-RevId: 563076972
2023-09-06 05:41:43 -07:00
tofunmi
45e15d48c4 Add WebpExtractor to DefaultExtractorsFactory
PiperOrigin-RevId: 563063506
2023-09-06 04:26:53 -07:00
claincly
6c2713f153 Renames containsKey to contains so it's more readable
PiperOrigin-RevId: 561916837
2023-09-01 04:25:25 -07:00
tonihei
f6be4bc526 Document C.TIME_UNSET on Player.setMediaItem(.., startPositonMs)
This wasn't properly documented yet (unlike the longer setMediaItems
method that already includes this documentation).

Issue: androidx/media#607
PiperOrigin-RevId: 561910073
2023-09-01 03:44:36 -07:00
ibaker
8af11a980a Remove duplication of various TestUtil methods
In most cases, the places that were trying to avoid a transitive
dependency on `lib-exoplayer` when this duplication was introduced [1]
are already depending on it again, except for `lib-container` where the
dep is added in this change.

In general it seems fine for the tests of module A to depend
(transitively or directly) on module B even where the prod code of
module A **does not** depend on module B.

[1] <unknown commit>

PiperOrigin-RevId: 561660371
2023-08-31 08:23:33 -07:00
ibaker
89f52dcb93 Rollback of 53b882d803
PiperOrigin-RevId: 561587445
2023-08-31 01:52:13 -07:00
ibaker
53b882d803 Add subsampleOffsetUs to Format.toLogString
This will help with debugging subtitle sync issues as it will show up in
track selection details emitted to logcat by `EventLogger`.

PiperOrigin-RevId: 561355836
2023-08-30 09:03:42 -07:00
ibaker
b4ee896847 Remove some spurious } from Player javadoc
Also use `@linkplain` in more places: If I was already touching a javadoc
block, I switched to `@linkplain` throughout the whole block.

PiperOrigin-RevId: 561320273
2023-08-30 06:16:37 -07:00
claincly
930e538aca Allow re-configuring DVFP in a (mostly) non-blocking manner
The first call to method `registerInputStream` doesn't block.

Later successive calls to the method blocks until the previous register call
finishes.

PiperOrigin-RevId: 559694490
2023-08-24 02:55:54 -07:00
tofunmi
404a259295 Remove queueInputBitmap by framerate and duration interfaces
PiperOrigin-RevId: 559384011
2023-08-24 09:25:49 +01:00
christosts
398809e4e2 Replace Queue<Long> with a queue for long primitives
Replace Queue<Long> with LongArrayQueue which provides queue semantics
for long primitives. LongArrayQueue is forked from IntArrayQueue which
in turn was forked from Androidx CircularIntArray.

IntArrayQueue is deleted and we now use CircularIntArray directly from
Androidx Collection.

PiperOrigin-RevId: 559129744
2023-08-24 09:14:44 +01:00
tofunmi
521c210fd1 Create Timestamp iterator
PiperOrigin-RevId: 558738035
2023-08-22 15:27:42 +01:00
rohks
e0d3cad8bb Add fields next object request (nor) and next range request (nrr)
Added this CMCD-Request fields to Common Media Client Data (CMCD) logging.

PiperOrigin-RevId: 558317146
2023-08-22 15:25:03 +01:00
jbibik
2156f94480 Bump media3 version to 1.2.0-alpha01 and update RELEASENOTES
#minor-release

PiperOrigin-RevId: 557487465
2023-08-16 21:55:35 +01:00
tonihei
7b663eea2f Remove MediaPeriodId from common module
It was moved there temporarily to support another temporary move
of ExoPlaybackException. See <unknown commit>.

Since then, ExoPlaybackException has been moved back to ExoPlayer
and we can do the same with MediaPeriodId, which only makes sense
in the context of the ExoPlayer module.

PiperOrigin-RevId: 557159381
2023-08-16 14:30:08 +01:00
tonihei
597de8706d Show play button during playback suppression by default
This changes the default logic of shouldShowPlayButton to show a play
button while the playback is temporarily suppressed. This helps to
provide better UI feedback to the fact that playback stopped and
provides a quick way for users to override the suppression and attempt
to restart playback.

Some apps may want to keep the legacy behavior depending on their app's
needs. Hence, we also add a config parameter to set this behavior both
in MediaSession and our default UI components.

Issue: google/ExoPlayer#11213
PiperOrigin-RevId: 557129171
2023-08-15 17:15:38 +01:00
tianyifeng
1e2a5cd06a Bump version numbers to Media3 1.1.1 and ExoPlayer 2.19.1
PiperOrigin-RevId: 555987303
2023-08-11 20:40:35 +00:00
tofunmi
48de9fa916 Add VFP.queueInputBitmap api accepting timestamps
PiperOrigin-RevId: 555970312
2023-08-11 20:39:20 +00:00
tonihei
1b085474e6 Rephrase format support javadoc to avoid Renderer reference
This is leftover from the fact that these constants were used
for ExoPlayer renderers only initially, but are now more generally
used to indicate format playback support.

PiperOrigin-RevId: 555951001
2023-08-11 20:34:03 +00:00
samrobinson
4d0fb4326a Simplify FakeAudioProcessor by extending BaseAudioProcessor.
Correct createOneSecondDefaultSilenceBuffer - It was returning an
incorrect buffer duration for non-mono channel counts.

PiperOrigin-RevId: 555439387
2023-08-10 12:34:07 +00:00
tofunmi
7a85236655 Update Mimetypes to include supported image mimetypes
PiperOrigin-RevId: 555433856
2023-08-10 12:31:29 +00:00
samturci
821ec49049 Preserve the byte order when using read-only ByteBuffers.
`asReadOnlyBuffer` doesn't copy the original byte order but always sets it to big-endian. Replace calls to it with a utility method that manually sets the byte order after creating the read-only copy.

This fixes `TeeAudioProcessor` providing a `ByteBuffer` always in big-endian and hence causing `AudioBufferSink` to read wrong data.

PiperOrigin-RevId: 554861402
2023-08-10 12:08:19 +00:00
Googler
4fe55b8b63 Return the correct output buffer from audio processing pipeline
PiperOrigin-RevId: 554851456
2023-08-10 12:07:15 +00:00
christosts
cba027c3b4 Util.getMaxPendingFramesCountForMediaCodecDecoders(): Remove unused arguments
PiperOrigin-RevId: 554838494
2023-08-10 12:05:08 +00:00
ibaker
6e91f0d4c5 Use ceiling divide logic in AudioTrackPositionTracker.hasPendingData
This fixes a bug with playing very short audio files, introduced by
fe710871aa

The existing code using floor integer division results in playback never
transitioning to `STATE_ENDED` because at the end of playback for the
short sample clip provided `currentPositionUs=189937`,
`outputSampleRate=16000` and `(189937 * 16000) / 1000000 = 3038.992`,
while `writtenFrames=3039`. This is fixed by using `Util.ceilDivide`
so we return `3039`, which means
`AudioTrackPositionTracker.hasPendingData()` returns `false` (since
`writtenFrames ==
durationUsToFrames(getCurrentPositionUs(/* sourceEnded= */ false))`).

#minor-release

Issue: androidx/media#538
PiperOrigin-RevId: 554481782
2023-08-10 11:57:15 +00:00
tonihei
69656774f9 Only use first frame stream start position check at stream transitions
We currently only force the first frame if the frame timestamp is
greater than the stream *offset*.

This is wrong for two reasons:
 1. The timestamp and the offset are not comparable and it should be
    the stream start position.
 2. The check should only be applied at stream transitions where we
    need to make sure that a new first frame isn't rendered until we
    passed the transition point.

We have to fix both issues together, because fixing just issue (1)
causes seeks to before the start position to no longer render the
frame (and playback will be stuck). A new test covers this case.

We also amend the stream transition test case to actually test what it
promises to test and add a test for prerolling samples at the
beginning, to ensure the first frame is still renderered.

Issue: androidx/media#291
PiperOrigin-RevId: 552858967
2023-08-07 11:21:08 +00:00
tonihei
79e05ad049 Simplify first frame logic
We currently use 3 different booleans to track the state of the first
frame rendering, which implies that there are 8 distinct possible
overall states. However, this is actually a staged process and there
are only 3 different overall states in the current code. This means
it's clearer and easier to reason about if the variables are combined
to a single state value. Overall, this should be a complete no-op.

State mapping:
 - rFFAReset=false, rFFAEnable=false, mayRenderFFAEINS=false
   => FIRST_FRAME_NOT_RENDERED_ONLY_ALLOWED_IF_STARTED
 - rFFAReset=false and/or rFFAEnable=false, mayRenderFFAEINS=any
   => FIRST_FRAME_NOT_RENDERED
 - rFFAReset=true, rFFAEnable=true, mayRenderFFAEINS=any
   => FIRST_FRAME_RENDERED

PiperOrigin-RevId: 552857802
2023-08-07 11:19:56 +00:00
tofunmi
87a020a154 Add PngExtractor to DefaultExtractorsFactory
PiperOrigin-RevId: 552506545
2023-08-01 14:32:58 +01:00
tofunmi
bb699e41c9 Add png extractor
Implementing a basic extractor that reads the whole png file into the trackOutput as one sample.

PiperOrigin-RevId: 551897619
2023-08-01 14:24:47 +01:00
ibaker
a700974d8c Mark x-media3-cues MIME type as text type
PiperOrigin-RevId: 551853336
2023-08-01 14:23:44 +01:00
claincly
c221958889 Remove setInputFrameInfo
After this change, every queued bitmap is treated as an individual input stream
(like a new MediaItems).

This change merges the FrameDropTest and FrameDropPixelTest into one (while maintaining all the test cases)

- This is accomplished by generating bitmaps with timestamps on it in FrameDropTest and compare them with goldens (one may call this a pixel test, please lmk if you want this to be renamed)
- The most part of the change comes from DefaultVideoFrameProcessorVideoFrameRenderingTest. The overall working is
  - We bypass the input manager
  - The TestFrameGenerator generates frames based on timestamps. In this case, we generate frames with timestamps on it
  - The generated frame is sent to texture output and in turn saved to bitmaps
  - We then compare the generated bitmap with the goldens

PiperOrigin-RevId: 551795770
2023-08-01 14:17:59 +01:00
tofunmi
ebbcec9a20 Add imageDurationMs to MediaItem
PiperOrigin-RevId: 551776350
2023-08-01 14:16:48 +01:00
claincly
e77c318d96 Create Util method for SparseArray.contains()
And fixed usages in Transformer and Effect.

PiperOrigin-RevId: 551484860
2023-08-01 14:12:10 +01:00
claincly
22d30b3d74 Fix some mal formatted TODOs
PiperOrigin-RevId: 550507208
2023-08-01 13:51:58 +01:00