525 Commits

Author SHA1 Message Date
claincly
930e538aca Allow re-configuring DVFP in a (mostly) non-blocking manner
The first call to method `registerInputStream` doesn't block.

Later successive calls to the method blocks until the previous register call
finishes.

PiperOrigin-RevId: 559694490
2023-08-24 02:55:54 -07:00
tofunmi
404a259295 Remove queueInputBitmap by framerate and duration interfaces
PiperOrigin-RevId: 559384011
2023-08-24 09:25:49 +01:00
christosts
398809e4e2 Replace Queue<Long> with a queue for long primitives
Replace Queue<Long> with LongArrayQueue which provides queue semantics
for long primitives. LongArrayQueue is forked from IntArrayQueue which
in turn was forked from Androidx CircularIntArray.

IntArrayQueue is deleted and we now use CircularIntArray directly from
Androidx Collection.

PiperOrigin-RevId: 559129744
2023-08-24 09:14:44 +01:00
tofunmi
521c210fd1 Create Timestamp iterator
PiperOrigin-RevId: 558738035
2023-08-22 15:27:42 +01:00
rohks
e0d3cad8bb Add fields next object request (nor) and next range request (nrr)
Added this CMCD-Request fields to Common Media Client Data (CMCD) logging.

PiperOrigin-RevId: 558317146
2023-08-22 15:25:03 +01:00
jbibik
2156f94480 Bump media3 version to 1.2.0-alpha01 and update RELEASENOTES
#minor-release

PiperOrigin-RevId: 557487465
2023-08-16 21:55:35 +01:00
tonihei
7b663eea2f Remove MediaPeriodId from common module
It was moved there temporarily to support another temporary move
of ExoPlaybackException. See <unknown commit>.

Since then, ExoPlaybackException has been moved back to ExoPlayer
and we can do the same with MediaPeriodId, which only makes sense
in the context of the ExoPlayer module.

PiperOrigin-RevId: 557159381
2023-08-16 14:30:08 +01:00
tonihei
597de8706d Show play button during playback suppression by default
This changes the default logic of shouldShowPlayButton to show a play
button while the playback is temporarily suppressed. This helps to
provide better UI feedback to the fact that playback stopped and
provides a quick way for users to override the suppression and attempt
to restart playback.

Some apps may want to keep the legacy behavior depending on their app's
needs. Hence, we also add a config parameter to set this behavior both
in MediaSession and our default UI components.

Issue: google/ExoPlayer#11213
PiperOrigin-RevId: 557129171
2023-08-15 17:15:38 +01:00
tianyifeng
1e2a5cd06a Bump version numbers to Media3 1.1.1 and ExoPlayer 2.19.1
PiperOrigin-RevId: 555987303
2023-08-11 20:40:35 +00:00
tofunmi
48de9fa916 Add VFP.queueInputBitmap api accepting timestamps
PiperOrigin-RevId: 555970312
2023-08-11 20:39:20 +00:00
tonihei
1b085474e6 Rephrase format support javadoc to avoid Renderer reference
This is leftover from the fact that these constants were used
for ExoPlayer renderers only initially, but are now more generally
used to indicate format playback support.

PiperOrigin-RevId: 555951001
2023-08-11 20:34:03 +00:00
samrobinson
4d0fb4326a Simplify FakeAudioProcessor by extending BaseAudioProcessor.
Correct createOneSecondDefaultSilenceBuffer - It was returning an
incorrect buffer duration for non-mono channel counts.

PiperOrigin-RevId: 555439387
2023-08-10 12:34:07 +00:00
tofunmi
7a85236655 Update Mimetypes to include supported image mimetypes
PiperOrigin-RevId: 555433856
2023-08-10 12:31:29 +00:00
samturci
821ec49049 Preserve the byte order when using read-only ByteBuffers.
`asReadOnlyBuffer` doesn't copy the original byte order but always sets it to big-endian. Replace calls to it with a utility method that manually sets the byte order after creating the read-only copy.

This fixes `TeeAudioProcessor` providing a `ByteBuffer` always in big-endian and hence causing `AudioBufferSink` to read wrong data.

PiperOrigin-RevId: 554861402
2023-08-10 12:08:19 +00:00
Googler
4fe55b8b63 Return the correct output buffer from audio processing pipeline
PiperOrigin-RevId: 554851456
2023-08-10 12:07:15 +00:00
christosts
cba027c3b4 Util.getMaxPendingFramesCountForMediaCodecDecoders(): Remove unused arguments
PiperOrigin-RevId: 554838494
2023-08-10 12:05:08 +00:00
ibaker
6e91f0d4c5 Use ceiling divide logic in AudioTrackPositionTracker.hasPendingData
This fixes a bug with playing very short audio files, introduced by
fe710871aa

The existing code using floor integer division results in playback never
transitioning to `STATE_ENDED` because at the end of playback for the
short sample clip provided `currentPositionUs=189937`,
`outputSampleRate=16000` and `(189937 * 16000) / 1000000 = 3038.992`,
while `writtenFrames=3039`. This is fixed by using `Util.ceilDivide`
so we return `3039`, which means
`AudioTrackPositionTracker.hasPendingData()` returns `false` (since
`writtenFrames ==
durationUsToFrames(getCurrentPositionUs(/* sourceEnded= */ false))`).

#minor-release

Issue: androidx/media#538
PiperOrigin-RevId: 554481782
2023-08-10 11:57:15 +00:00
tonihei
69656774f9 Only use first frame stream start position check at stream transitions
We currently only force the first frame if the frame timestamp is
greater than the stream *offset*.

This is wrong for two reasons:
 1. The timestamp and the offset are not comparable and it should be
    the stream start position.
 2. The check should only be applied at stream transitions where we
    need to make sure that a new first frame isn't rendered until we
    passed the transition point.

We have to fix both issues together, because fixing just issue (1)
causes seeks to before the start position to no longer render the
frame (and playback will be stuck). A new test covers this case.

We also amend the stream transition test case to actually test what it
promises to test and add a test for prerolling samples at the
beginning, to ensure the first frame is still renderered.

Issue: androidx/media#291
PiperOrigin-RevId: 552858967
2023-08-07 11:21:08 +00:00
tonihei
79e05ad049 Simplify first frame logic
We currently use 3 different booleans to track the state of the first
frame rendering, which implies that there are 8 distinct possible
overall states. However, this is actually a staged process and there
are only 3 different overall states in the current code. This means
it's clearer and easier to reason about if the variables are combined
to a single state value. Overall, this should be a complete no-op.

State mapping:
 - rFFAReset=false, rFFAEnable=false, mayRenderFFAEINS=false
   => FIRST_FRAME_NOT_RENDERED_ONLY_ALLOWED_IF_STARTED
 - rFFAReset=false and/or rFFAEnable=false, mayRenderFFAEINS=any
   => FIRST_FRAME_NOT_RENDERED
 - rFFAReset=true, rFFAEnable=true, mayRenderFFAEINS=any
   => FIRST_FRAME_RENDERED

PiperOrigin-RevId: 552857802
2023-08-07 11:19:56 +00:00
tofunmi
87a020a154 Add PngExtractor to DefaultExtractorsFactory
PiperOrigin-RevId: 552506545
2023-08-01 14:32:58 +01:00
tofunmi
bb699e41c9 Add png extractor
Implementing a basic extractor that reads the whole png file into the trackOutput as one sample.

PiperOrigin-RevId: 551897619
2023-08-01 14:24:47 +01:00
ibaker
a700974d8c Mark x-media3-cues MIME type as text type
PiperOrigin-RevId: 551853336
2023-08-01 14:23:44 +01:00
claincly
c221958889 Remove setInputFrameInfo
After this change, every queued bitmap is treated as an individual input stream
(like a new MediaItems).

This change merges the FrameDropTest and FrameDropPixelTest into one (while maintaining all the test cases)

- This is accomplished by generating bitmaps with timestamps on it in FrameDropTest and compare them with goldens (one may call this a pixel test, please lmk if you want this to be renamed)
- The most part of the change comes from DefaultVideoFrameProcessorVideoFrameRenderingTest. The overall working is
  - We bypass the input manager
  - The TestFrameGenerator generates frames based on timestamps. In this case, we generate frames with timestamps on it
  - The generated frame is sent to texture output and in turn saved to bitmaps
  - We then compare the generated bitmap with the goldens

PiperOrigin-RevId: 551795770
2023-08-01 14:17:59 +01:00
tofunmi
ebbcec9a20 Add imageDurationMs to MediaItem
PiperOrigin-RevId: 551776350
2023-08-01 14:16:48 +01:00
claincly
e77c318d96 Create Util method for SparseArray.contains()
And fixed usages in Transformer and Effect.

PiperOrigin-RevId: 551484860
2023-08-01 14:12:10 +01:00
claincly
22d30b3d74 Fix some mal formatted TODOs
PiperOrigin-RevId: 550507208
2023-08-01 13:51:58 +01:00
jbibik
f4ad26451f Deprecate MimeTypes.TEXT_EXOPLAYER_CUES for APPLICATION_MEDIA3_CUES
Instead of `text/x-exoplayer-cues`, we will use `application/x-media3-cues`.

The prefix should be `application` not `text` since the encoded form is arbitrary bytes, not necessarily text. The name should not reference `exoplayer`, since the Media3 Extractors (which are not part of `exoplayer`) produce this format.

PiperOrigin-RevId: 550181852
2023-08-01 13:50:50 +01:00
tonihei
5c5e7cd257 Update storage access in main demo app
The main demo app was still targeting API 29 to avoid scoped storage
restrictions. It is now updated to 34 (like the rest of the demo apps)
and handles scoped storage as it should handle it.

More specifically:
 - We need to request READ_MEDIA_... permissions instead of
   READ_EXTERNAL_STORAGE from API33.
 - The legacy scoped storage opt-out can be removed
 - READ_MEDIA_... permissions don't allow arbitrary file access
   if the file doesn't end in a typical media file extension, so
   this change adds a remark on the guide page to place samples
   in the app-specific directory.
 - We also don't have to request permissions for the app-specific
   directories.
 - Custom json files can't be placed in arbitray local locations
   because they don't end in a media file extension, as there is
   no way we can request a permission to load them. This means we
   can remove the storage request logic in SampleChooserActivity.

Issue: google/ExoPlayer#6045
PiperOrigin-RevId: 549252474
2023-07-20 10:29:02 +01:00
tonihei
8064c6df83 Declare foreground service type for DownloadService
This ensures the DownloadService stays functional on Android 14
where defining this type is required. On Android 14 and above,
the app also needs to define the DATA_SYNC permission, which is
added to the demo app as well. In the future, this service type
will no longer be supported and DownloadService needs to be
rewritten with another background scheduling framework.

Issue: google/ExoPlayer#11239
PiperOrigin-RevId: 548994842
2023-07-20 10:23:56 +01:00
samrobinson
035934c6d4 Combine audio mixing logic across Transformer.
AudioMixingUtil#mix handles input & output in float or Int16 PCM. Given
Float and Int16 use different sample ratnes, this util handles
conversion between the two, based on the encoding being mixed to.

Migrate AudioMixer to use the util, removing AudioMixingAlgorithm
interface and implementation. ChannelMixingAudioProcessor will be
migrated after additional performance checks.

PiperOrigin-RevId: 548994584
2023-07-20 10:22:58 +01:00
ibaker
0fa66534fe Use CuesWithTiming.durationUs in SubripParser instead of empty cues
This fixes two things in one go:
1. In order to indicate 'end of a cue' **without** relying on
   `CuesWithTiming.durationUs`, `SubripParser` should have been emitting
   an empty `List<Cue>` instead of the current behaviour of a
   single-item list with `cue.text=""`.
2. There's no need for the empty cue (or cue list), we can use
   `durationUs` to indicate the end of each list of cues.

There's no real need to ever have a `Cue` with `text=""`, so also
deprecate `Cue.EMPTY`.

PiperOrigin-RevId: 548938874
2023-07-20 10:18:09 +01:00
andrewlewis
81ef52763d Fix @RequiresApi annotations in GlUtil
`EGL14` was added in API 17, so move all references into the `Api17` wrapper
class. `GLES30` requires API 18.

Remove some defensive calls to check that a context is current. Checking for GL
errors after calls should catch the case of calling GL methods without a
context (which is also only likely to come up early in development), and these
checks were not done consistently across all methods anyway. That allows
removing the API 17 requirement from `SceneRenderer` (where previously it
wasn't documented).

PiperOrigin-RevId: 548669946
2023-07-20 10:12:14 +01:00
huangdarwin
0c29dacde3 Compositor: Use timestamps to release frames.
Also, implement back-pressure to avoid requesting more than all of the
compositor's texturepool textures

PiperOrigin-RevId: 548179800
2023-07-20 10:04:55 +01:00
huangdarwin
da68b9391d Add underscores to numbers for readability.
No-op change just to marginally improve readability and reduce
counting zeroes.

PiperOrigin-RevId: 547844999
2023-07-14 10:20:48 +01:00
andrewlewis
2bfc6c7740 Add support for extracting Vorbis audio in MP4
PiperOrigin-RevId: 547474958
2023-07-13 15:53:55 +01:00
huangdarwin
c122c0ca05 Compositor: Add support for multiple bitmaps, and add testrunner
* Allow more than one input bitmap at a time.
* Allow Compositor to take in and set up an Executor. Otherwise,
Compositor resources may be created on one thread and accessed on another.
* Add a Compositor TestRunner to reuse test code more.
* Update VideoFrameProcessingTaskExecutor to use a new onError listener, so
that it's more reusable in non-DVFP contexts, like for Compositor.

PiperOrigin-RevId: 547206053
2023-07-13 15:45:30 +01:00
tonihei
5c8b142174 Fix inconsistency in SampleQueue seek for sync-sample-only formats
For sync-sample-only formats, we have an optimization to drop all buffers
with less than the start time when writing them to the queue.

For the same formats, if we set a new start time (=seek), we only seek
to the buffer at or before the start time. This means the first sample
in the queue is different depending on whether we seek to a start time
or set a start time and then write samples. This is inconsistent and
effectively means the first sample depends on a race condition between
the Loader thread (writing samples) and the playback thread (attempting
an initial seek in the already loaded samples).

The effect of this inconsistency is that we have to decode one sample
we don't need (and could have skipped) and that some tests become flaky
if the test setup runs into the mentioned race condition.

The fix is to change the SampleQueue seek method to also seek to
a sample at or after the specified time, to align the behavior to the
case where we write the same samples to an empty queue.

The change also clarifies the Javadoc of
MimeTypes.allSamplesAreSyncSamples to note that this should really only
return true if the samples have no "duration" that matters. Otherwise,
we could reasonably return true for most subtitle formats although it
would break subtitle display because we'd remove samples that start
before the seek time.

PiperOrigin-RevId: 547189941
2023-07-13 15:41:49 +01:00
ibaker
2afcf99591 Expand @UnstableApi javadoc with how to request stable API additions
Issue: androidx/media#503

#minor-release

PiperOrigin-RevId: 547143885
2023-07-13 15:38:08 +01:00
andrewlewis
f657fa8048 Add utility for deleting sync object without throwing
PiperOrigin-RevId: 547138887
2023-07-13 15:37:02 +01:00
andrewlewis
6f4ef51727 Add support for big endian 24/32-bit PCM
Also parse the PCM encoding for lpcm in MP4, and update `MatroskaExtractor`
similarly.

Tested manually in the demo app using an MP4 with 24-bit big endian audio.

PiperOrigin-RevId: 546878505
2023-07-13 15:31:14 +01:00
ibaker
a8520bdee6 Clarify that new ConditionVariable instances are closed
#minor-release

PiperOrigin-RevId: 546803592
2023-07-13 15:25:15 +01:00
andrewlewis
a0bc8d95c2 Improve texture asset loader documentation
Add documentation for threading requirements at the class level (in
addition to existing documentation on the methods) to improve
discoverablility. Also fix a couple of nits in the javadoc (US English
spelling, avoid passive voice) and in `OnInputFrameProcessedListener`.

PiperOrigin-RevId: 546303732
2023-07-13 15:14:21 +01:00
huangdarwin
0b7d9a945c GL: Log error for checkGlError when no gluErrorString is present.
Otherwise, errors like `GL_INVALID_FRAMEBUFFER_OPERATION` will only
report a `null` error string, instead of the proper error string.

PiperOrigin-RevId: 546273328
2023-07-13 15:10:48 +01:00
claincly
2b8c01de47 Remove stale TODO
PiperOrigin-RevId: 545711472
2023-07-13 14:55:23 +01:00
huangdarwin
7a368b9a11 Effect: Remove GlTextureInfo accessor methods.
The extra check was a bit excessive, especially as it's called multiple times
per frame.

PiperOrigin-RevId: 545657102
2023-07-13 14:51:38 +01:00
samrobinson
c33a17d89c Wrap AudioSamplePipeline input and processing within AudioGraph.
This is a no-op refactor.

PiperOrigin-RevId: 545421300
2023-07-05 09:20:12 +00:00
michaelkatz
de4575da28 Add nanoTime method to Clock to support overriding System.nanoTime()
#minor-release

PiperOrigin-RevId: 545237925
2023-07-05 09:15:49 +00:00
huangdarwin
854c5254ec GL: Print egl errors in hex.
The actual errors are all positive hex values. Without this CL, we must first
convert decimal errors to hex ones before figuring out what went wrong.

PiperOrigin-RevId: 544695961
2023-07-05 09:01:36 +00:00
tofunmi
d14122b029 Create queue of sync objects in FinalShaderProgramWrapper
PiperOrigin-RevId: 544683524
2023-07-05 09:00:20 +00:00
ibaker
e8fdd83558 CEA-608: Only truncate to 32 visible characters
We introduced truncation to 32 chars in <unknown commit>
and included indent and offset in the calculation. I think this is
technically correct, but it causes problems with the content in
Issue: google/ExoPlayer#11019 and it doesn't seem a problem to only truncate actual
cue text (i.e. ignore offset and indent).

#minor-release

PiperOrigin-RevId: 544677965
2023-07-05 08:59:08 +00:00